Speech-to-Speech English Voice AI Tester

SME Careers is seeking Speech-to-Speech English Voice AI Testers to help evaluate and improve conversational AI systems through realistic spoken interactions and structured quality assessments.

In this remote contractor role, contributors will engage in short real-world voice conversations with AI systems and evaluate the model’s:

• Accuracy
• Relevance
• Helpfulness
• Conversational naturalness
• Task completion ability

Typical sessions involve approximately 5-minute scenario-based conversations followed by structured written evaluations and detailed feedback.

This work directly contributes to improving advanced speech and conversational AI systems used by leading AI organizations.

Key Responsibilities

Speech-Based AI Conversations

• Conduct natural voice conversations with AI systems in guided scenarios such as:

  • Shopping interactions
  • Customer support
  • Appointment scheduling
  • Service inquiries

• Maintain realistic conversational flow during testing sessions

AI Performance Evaluation

• Assess AI responses based on:
  • Accuracy
  • Clarity
  • Relevance
  • Politeness
  • Conversational quality
  • Ability to complete tasks

Feedback & Documentation

• Provide concise written feedback identifying:
  • Misunderstandings
  • Incorrect assumptions
  • Awkward phrasing
  • Conversation breakdowns
  • Successful responses

Quality Assurance

• Follow detailed rubrics and evaluation guidelines consistently
• Apply structured scoring standards across all sessions

Required Skills & Qualifications

• Must currently reside in the United States

• Fluent English proficiency:

  • Spoken
  • Written
  • Reading comprehension

• Strong written communication and feedback abilities

• Comfortable speaking naturally and clearly on microphone

• Strong:

  • Attention to detail
  • Consistency
  • Instruction-following skills

• Reliable:

  • Internet connection
  • Quiet workspace
  • Audio setup for voice sessions

Desired Skills & Experience

• Speech AI
• Voice AI
• Conversational AI
• AI evaluation
• QA testing
• Scenario roleplay
• Written feedback and quality analysis

About SME Careers

SME Careers is an AI Data Services company and subsidiary of SuperAnnotate.

The organization provides AI training data and evaluation services for leading AI companies and foundation-model labs worldwide.

Contributors help improve conversational and speech AI systems through structured testing, evaluation, and human feedback workflows.