About the work
Model comparisons are only meaningful when the evaluation is sound. We design benchmark tasks, scoring rubrics and analysis that tell AI teams what a model can and cannot do, with results that hold up when someone checks them.
How it works
You design evaluation cases for a defined capability, write scoring criteria that two reviewers would apply the same way, and run or review model outputs against them. You analyse where models fail, check results for noise or leakage, and write calibration notes that explain how scores should be read.
Who this is for
ML engineers, researchers and data scientists who have built or used evaluation pipelines. You think carefully about what a metric actually measures, notice when a result looks too good, and can explain an evaluation design to an engineering team.
What you’ll do
- Design evaluation cases that target a specific model capability
- Write scoring rubrics with clear pass and fail boundaries
- Run model outputs against benchmarks and analyse failure patterns
- Check evaluation data for leakage, ambiguity and inconsistent labels
- Write calibration notes and summaries of what the results show
Who should apply
- 2+ years of experience in machine learning, evaluation or applied data science
- Working knowledge of ML fundamentals and experiment design
- Proficiency in Python for data analysis
- Experience with benchmarks, metrics or model evaluation pipelines
- Careful, well-documented error analysis
Contract & payment terms
- You will be engaged as an independent contractor, not as an employee.
- This is fully remote work, open to applicants in any location.
- Work is project-based. Availability varies, and an application does not guarantee an assignment.
- Pay, scope, review criteria and access are confirmed at onboarding, after you sign a Non-Disclosure Agreement (NDA) and an Independent Contractor Agreement (ICA).
- Please do not share confidential material from a current or former employer or client.
Clorvin Labs builds coding tasks, evaluation datasets and reproducible environments for AI teams. Our project network brings in experienced specialists when a project needs them.
Learn more about Clorvin Labs