Classification and extraction: the tasks that return the most
Why these in particular
Classification and extraction have a correct answer you can check, a short cheap output, and simple integration with existing systems. They're also the cases where a model replaces repetitive manual work — routing tickets, reading invoices, tagging leads.
By contrast, an open chat interface is hard to measure, expensive, and invites expectations the system won't always meet.
How to build correctly
Define closed categories with a written definition for each, including an “unclear” category. Without it the model will push every case into the nearest category and lie to you in the numbers.
In extraction: define a strict schema, validate in code, and return an empty value instead of a guess. A hallucinated field is more dangerous than a missing one.
Human in the loop
Route low-confidence cases to a human, and use their decisions to grow the evaluation set. That way the system improves through use, with no training project.
Going deeper
Measure each category separately — an overall accuracy of 92 percent can hide a critical category at 60. And build a confusion matrix: it shows not just how many errors there are but which categories swap for each other, which is usually a definition flaw, not a model one.