Where frontier models still fall short

Frontier models struggle with multi-step reasoning in specialized STEM work. Ask one to run a multi-step chemical reaction process, or to solve an engineering problem where step 4 depends on what step 2 returned, and it loses the thread. Agents have a matching failure on long tool-use trajectories. A common case: a single-use, time-limited administrative credential that the agent creates, uses, and then never deletes.

The reason is in the pre-training. These models learn general patterns for predicting the next token, not deep domain knowledge, and post-training is where you close that gap.

1. Model alignment: RLHF and RLVR

RLVR fits tasks with an objective answer, like a mathematics problem. RLHF fits tasks where the answer is a matter of judgment, like writing a novel. Rubric criteria sit between the two and give the reward model something more precise to learn from than a single preference vote, because the model receives a graded score instead of a binary one. On long multi-step tasks, that grading is what keeps the reward model calibrated.

My part starts with the prompts. I write prompts that expose model failures by layering specific, sequential constraints (expertise, content, formatting) and the expectations a domain expert would actually hold. I then write the ideal response for supervised fine-tuning, and grade model responses for reinforcement learning. The applied mathematics background is what lets me build evaluation frameworks that match human intent rather than approximate it.

2. Better reasoning through Chain-of-Thought

I break complex tasks into steps a grader can verify, using Chain-of-Thought frameworks. That turns a technical solution into ground-truth training data. One example: writing out the optimization process for recovering MRI images in fine-grained steps that a model can learn to reproduce and then generalize from.

3. Error analysis and rubric evaluation

I read the YAML files to understand what the user asked for, weigh it against the rubric, and trace agent trajectories to find the specific failure mode. I check the reasoning of both the agent and the world agent, so a failure caused by the world agent doesn't get charged to the agent under review. Then I read the transcript for the function-calling error: a call that never happened, or the right function called with the wrong parameters.

4. What the model can do afterward

After a post-training cycle, models pick up the specialized knowledge for problems they used to fail. Structured multi-step tasks that broke them before go through reliably.