Professional Domain Expert (Technology)
Job description
About Mercor
Mercor partners with leading AI labs and enterprises to train frontier models using human expertise. You will collaborate with top researchers and contribute to improving next-generation AI systems in your domain.
Role Overview
Mercor is collaborating with a leading AI lab to contract experienced professionals for an AI model evaluation project.
In this role, you will assess the quality, accuracy, and safety of AI-generated responses across Information Technology (IT) domains. Your work will directly improve the reliability of AI systems used in high-stakes environments where incorrect information can carry significant risks.
Key Responsibilities
- Write realistic prompts that reflect how professionals and users seek domain-specific guidance.
- Evaluate AI-generated responses for factual accuracy, correctness, and practical usefulness.
- Identify fabricated claims, incorrect references, or misleading reasoning.
- Score and rank multiple model responses using structured evaluation rubrics.
- Provide clear, evidence-based written justifications for evaluations.
Ideal Qualifications
- Master’s degree or higher in Computer Science, Information Systems, or a related field.
- Professional experience applying domain expertise in a practical or advisory capacity.
- Familiarity with industry standards, regulations, or best practices.
- Strong written communication and critical reasoning skills.
Work Details
- Commitment: Approximately 20 hours per week
- Work Type: Fully remote and asynchronous
Application Process
- Submit your resume to begin
- Complete a Model Response Evaluation assessment
Contract & Payment Terms
- Independent contractor engagement
- Flexible schedule — work on your own time
- Weekly payments via Stripe or Wise
- Projects may be extended, shortened, or concluded early based on performance and business needs
- No access to confidential or proprietary external data required
Additional Notes
- We consider all qualified applicants without regard to legally protected characteristics
- Reasonable accommodations are available upon request
- Unable to support H1-B or STEM OPT candidates at this time
You will be redirected to the company's website to complete your application.
Mercor
Discover more opportunities that match your skills and interests.