Task design
A concrete problem, expected outputs and clear acceptance criteria. Category and language tags make the collection easier to explore.
Our work / Data & evaluation
The dataset is more than the prompt.
A useful task needs clear instructions, a place to run, a way to judge the result and a record of the checks. Our public library brings those pieces together across software engineering, ML and dataset QA.
100+sample tasks
9technical categories
12language & tool tags
108Clorvin reference-checked tasks
01 / Task datasets
Open the files behind the work.
The Task Library contains original Clorvin tasks across nine categories, from build systems and data processing to debugging, machine learning, security and scientific computing.
Every entry has a new brief, starter code, a reference solution and public fixtures. All 108 reference solutions have been checked locally in Python, with inspectable results and downloadable source packages.
Coverage by category
108 samples · select a category to explore
Build and Dependency Management12Data Processing12Debugging12Games12Machine Learning12Scientific Computing12Security12Software Engineering12System Administration1202 / What the package contains
Enough context to inspect, run and review.
Environments & reference code
Original source files, dependency setup and reference implementations in every Clorvin package. Package pages show exactly what is included.
Verifiers & reward checks
Checks for expected outputs, failure cases and required behaviour. Model rewards should come from recorded verifier results.
Dataset quality
Examples cover split leakage, duplicates, scoring gates, reviewer consensus and consistent replay. Each is available as a downloadable package.
Human feedback & post-training
SFT examples, RLHF preference reviews and domain-specific annotations can be scoped around a partner’s format and review rules.
Traceable handover
An index of task IDs, categories, source links and archive fingerprints. Inspection and execution status stay attached to each sample.
03 / Reference implementations
Follow the checks from input to output.
These two internal engineering walkthroughs include source code, a reference solution and recorded verification checks.
Reconcile a payment ledger without losing a cent.
A replayed transaction should not change an account balance. This exercise combines exact money handling with duplicate detection, so reviewers can inspect both the calculation and the rules around the input.
- Example input
- 4 transaction rows, including a repeated payment and a refund.
- Expected result
- 3 unique entries; the net balance is exactly 0.25.
What the implementation handles
- Adds amounts as integer cents, avoiding floating-point rounding.
- Counts identical IDs once and rejects conflicting versions.
- Returns account balances in a stable order, even if input rows move.
10/10Python reference checks passed
Empty implementation rejected
Empty implementation rejected
Included: task brief, reference solution, verifier, Dockerfile and the recorded Python results.
Internal demonstration · No AI model runsPut late and repeated events in the right order.
Events can arrive late, be replayed or use different timezone offsets. This exercise produces one repeatable timeline from a batch and makes the rules for duplicates and tied times explicit.
- Example input
- 4 event rows with two time offsets, a replay and a timestamp tie.
- Expected result
- 3 unique events ordered by UTC time, then by event ID.
What the implementation handles
- Converts timezone-aware timestamps to a common UTC timeline.
- Removes equivalent replays and rejects conflicting event records.
- Preserves microseconds and uses the event ID to break timestamp ties.
10/10Python reference checks passed
Empty implementation rejected
Empty implementation rejected
Included: task brief, reference solution, verifier, Dockerfile and the recorded Python results.
Internal demonstration · No AI model runsReference verification and model evaluation are separate.
The recorded checks cover included reference implementations. We have not run GPT or Opus on this collection. Task pages include an explicitly labelled illustrative chart; published model benchmarks are available separately with their sources.
Explore published benchmarksWork with Clorvin
Tell us what needs doing.
Share the scope, the expected volume and your timeline.
We’ll work out where our team can help.