Remote contract opportunity for professionals with strong analytical writing and evaluation skills to design AI assessment tasks, create scoring rubrics, and evaluate AI performance across real-world workflows.
Speech-to-Speech English Voice AI Tester
United States
March 11, 2026
Job description
About the Role
SME Careers is seeking Speech-to-Speech English Voice AI Testers to help evaluate conversational AI systems through real-time voice interactions.
In this role, you will engage in short, natural conversations with an AI model and then assess the model’s performance using structured evaluation questions.
Each interaction typically lasts around five minutes, followed by a brief evaluation where you review the model’s conversational quality, accuracy, and helpfulness.
Your feedback will directly help improve the performance of advanced speech-based AI systems.
About SME Careers
SME Careers is a fast-growing AI data services company and subsidiary of SuperAnnotate. The company provides high-quality training data to some of the world’s largest AI companies and foundation-model research labs.
Contributors help build and evaluate next-generation AI technologies used in conversational agents and voice assistants.
Key Responsibilities
Conduct Speech-Based Conversations
- Speak with the AI model in guided real-world scenarios
- Maintain natural conversations for approximately 5 minutes per session
- Example scenarios include:
- Customer service interactions
- Booking appointments
- Shopping conversations
- General inquiries
Evaluate AI Model Performance
- Assess the model’s responses based on:
- Accuracy
- Relevance
- Helpfulness
- Politeness
- Conversational naturalness
- Task completion ability
Provide Written Feedback
- Write clear, concise evaluation notes
- Identify issues such as:
- Misunderstandings
- Incorrect assumptions
- Unnatural phrasing
- Conversation breakdowns
Follow Evaluation Guidelines
- Apply structured rubrics and evaluation criteria
- Ensure consistency across sessions
- Follow all task instructions and project guidelines
Requirements
- Must currently reside in the United States
- Fluent in English (spoken and written)
- Strong reading and writing skills
- Comfortable speaking naturally on microphone during short sessions
- Detail-oriented with the ability to follow evaluation rubrics
- Reliable internet connection
- Access to a quiet environment for voice conversations
Why Join
This role allows you to contribute directly to improving speech and conversational AI systems used worldwide.
Your feedback helps engineers and researchers refine AI models so they can better understand natural speech and human interaction.
Apply now
You will be redirected to the company's website to complete your application.
Play and evaluate AI-generated game worlds to improve next-gen AI environments ($6–$65/hr).