M
AI Safety Specialist - Evaluation Expert
mercor
Contract type
Freelance
Work mode
100% remote
Experience
Senior · 5+ years
Job description
Key details
- Evaluate AI-generated responses for safety, factual accuracy, policy compliance, and overall quality
- Review content involving misinformation, political persuasion, self-harm, violence, cyber, biosecurity, and other sensitive domains
- Apply and refine evaluation rubrics for RLHF, SFT, and AI safety benchmarking
- Identify unsafe outputs, hallucinations, reasoning failures, and policy violations
- Provide structured feedback to improve model alignment and safety performance
- Collaborate with AI researchers and safety teams on ongoing evaluation initiatives
- Company mission
- Mercor connects elite creative and technical talent with leading AI research labs
Benefits
- Information not specified
Requirements & details
- Bachelor's degree or higher in Journalism, Communications, Psychology, Sociology, Public Policy, Law, Biology, Chemistry, Computer Science, or a related discipline
- 5+ years of professional experience in AI Safety, Trust & Safety, journalism, public policy, scientific research, security, or a related field
- Excellent written English, critical thinking, and analytical reasoning skills
- Ability to consistently evaluate nuanced and policy-sensitive scenarios
- Preferred: Experience with AI Safety, RLHF, SFT, Trust & Safety, or AI evaluation
- Preferred: Familiarity with safety policies, content moderation, or evaluation rubric development
- Preferred: Experience reviewing complex, high-risk, or ambiguous content
- Contract type: Contract
- Compensation: $60–$70/hour
- Location: Remote
- Information not specified
- Uncategorized