German Audio Generalist Evaluator Expert
Job description
About the Role
Mercor is seeking a German Audio Generalist Evaluator Expert to support a high-impact audio AI research project with a leading AI research lab.
In this role, you will work on audio transcription, annotation, and evaluation tasks that help train and benchmark advanced language models. The work focuses on analyzing German-language audio and producing structured written outputs that improve multilingual AI systems.
This is a short-term structured engagement ideal for candidates with strong analytical backgrounds and fluency in both German and English.
Key Responsibilities
Transcribe and Optimize Audio & Video
- Listen to, analyze, and transcribe audio and video content in German
- Produce high-quality written outputs in German, with supporting work in English when required
- Ensure clarity, accuracy, and adherence to formatting guidelines
- Capture nuances such as tone, intent, and regional language variations
Define and Document Evaluation Standards
- Establish expectations for correct and high-quality responses in audio tasks
- Develop evaluation rubrics and grading guidelines in German and English
- Document standards to maintain consistency across evaluation teams
- Identify linguistic edge cases and register differences within German
Conduct Model Testing and Grading
- Run prompts through AI models and assess generated responses
- Evaluate outputs for accuracy, completeness, fluency, and clarity
- Provide structured feedback to improve model performance
Support Benchmarking and Quality Assurance
- Participate in quality assurance and review cycles
- Ensure outputs meet Mercor’s quality standards before dataset integration
- Collaborate with project leads to resolve ambiguities and improve task design
Minimum Qualifications
- Strong writing, editing, and analytical skills
- Ability to work independently and manage time effectively
- Native or near-native fluency in German
- Professional fluency in English
- Ability to accurately transcribe and analyze German audio
- Availability to commit 10–20 hours per week
Preferred Qualifications
- College students or recent graduates
- Background in linguistics, humanities, journalism, or social sciences
- Prior experience with transcription, localization, annotation, or AI evaluation
- Familiarity with regional German dialects (e.g., Austrian German, Swiss German)
- Interest in AI research and language model development
Application Process
- Complete a short AI-led interview (approximately 15 minutes)
- If selected, you will receive onboarding instructions
- Begin contributing to the project
Additional Role Details
- Structured project environment with clear guidelines and tools
- Opportunity to gain hands-on experience in AI research workflows
- Direct contribution to benchmarking multilingual AI models
Contract & Payment Terms
- Independent contractor role
- Fully remote work schedule
- Weekly payments via Stripe or Wise
- Project duration may vary depending on performance and project needs
Please note: Mercor cannot support H1-B or STEM OPT candidates at this time.
About Mercor
Mercor partners with leading AI labs and enterprises to train frontier AI models using human expertise.
Contributors collaborate with researchers and engineers to help improve the accuracy, reasoning, and multilingual capabilities of advanced AI systems.
You will be redirected to the company's website to complete your application.
Mercor
Discover more opportunities that match your skills and interests.