How DataFoldIT replaced frontier API calls with two models of its own

The Challenge

DataFoldIT’s recruiters matched candidates to open requisitions with a frontier model. The matches were good, but the bill grew with every requisition, and the model kept missing the firm’s own conventions for seniority levels and skill families. Recruiters were re-checking its work by hand, which defeated the point.

‍

The Approach

Applied Crew rolled out BackContext across the recruiting team’s agent tools and linked every action to what actually happened: submitted, interviewed, placed. BackContext recorded 395K recruiter and agent actions with their outcomes. Only verified outcomes became training and eval data.

‍

Two models, two jobs

We fine-tuned two open-weight models on that data. dfit-match-v3 handles candidate-to-requisition matching. dfit-screen-v2 handles resume screening. Both learned the seniority and skills conventions DataFoldIT’s recruiters actually use, from the placements that actually happened.

‍

Benchmarked on DataFoldIT’s own data

Before anything shipped, both models were tested on a held-out set of DataFoldIT tasks, side by side with Claude Opus 5.5, Claude Fable 5.1, GPT-6 Astra and GPT-6.1 Sol. dfit-match-v3 scored 98.1% match accuracy. GPT-6 Astra scored 92.6% on the same set.

‍

Shipped behind a quality gate

Both models went live behind a quality gate with frontier fallback. When confidence is low, the task routes to a frontier model; that happens on 2.1% of tasks. Every new outcome flows back through BackContext into the next model version.

‍

The Results

DataFoldIT now runs two models of its own in production. Matching is more accurate than the frontier model it replaced, 52× cheaper per task than GPT-6 Astra, and fast enough to run on every requisition, with a median latency of 0.3s.

‍

Key Outcomes

• 98.1% match accuracy on held-out candidate-to-requisition tasks, vs 92.6% for GPT-6 Astra
• 52× lower cost per task than GPT-6 Astra
• 0.3s median latency on every matching request
• 2.1% frontier fallback only on low-confidence tasks
• 395K work records captured with BackContext, feeding every new version

‍

Customer Feedback

“We stopped paying frontier prices for a task our own model now does better. The eval report made the decision easy.” — VP, Recruiting Operations · Staffing