M
Domain Contributor - Evaluation Specialist
Contract type
Freelance
Work mode
Remote
Experience
3 years
Job description
Contractor role supporting an internal project that evaluates and improves advanced AI models by designing domain-specific tasks and assessing model outputs. Experts in history, law, finance, journalism, or tech contribute high-quality, real-world input on a flexible, pay-per-task basis.
At a glance
- Employment
- Contract · Independent contractor
- Work mode
- Remote
What you'll do
- Design nuanced, domain-specific tasks that probe AI model limits and surface reasoning flaws.
- Author questions and scenarios demanding deep subject knowledge, beyond general understanding.
- Assess AI-generated outputs, validating accuracy against authoritative, primary sources in your area of expertise.
- Document clear rationales for model shortcomings, specifying missed reasoning, source errors, or nuances.
- Apply detailed task guidelines and evaluation rubrics consistently to each submission.
- Submit completed deliverables independently on a rolling basis, meeting project requirements for written and verbal clarity.
What we're looking for
- Proven hands-on experience with AI model training, annotation, RLHF, or evaluation.
- Deep expertise in a core domain: history, archival or library science, law (paralegal, legal academic, compliance), finance (equity research, PE/VC, forensic accounting), journalism, or technology/industry history.
- Advanced research skills with a track record of verifying complex claims against authoritative sources.
- Exceptional written and verbal communication skills to articulate findings and feedback.
- Demonstrated ability to precisely follow detailed instructions and evaluation frameworks.
Nice to have
- Background in adversarial or "stomp the model" task design is advantageous.
- Graduate-level research, experience at a Fortune 100 company, or professional fluency in a non-English language a plus.
How to get hired
- 1Onboarding; first tasks expected within 24–48 hours of completing onboarding.
Apply
About micro1
Key facts
- Typically fills roles within 48 hours.