Researchers at Collectively AI and Agentica have launched DeepCoder-14B, a brand new coding mannequin that delivers spectacular efficiency similar to main proprietary fashions like OpenAI’s o3-mini.
Constructed on prime of DeepSeek-R1, this mannequin provides extra flexibility to combine high-performance code era and reasoning capabilities into real-world purposes. Importantly, the groups have absolutely open-sourced the mannequin, its coaching information, code, logs and system optimizations, which may help researchers enhance their work and speed up progress.
Aggressive coding capabilities in a smaller package deal
The analysis group’s experiments present that DeepCoder-14B performs strongly throughout a number of difficult coding benchmarks, together with LiveCodeBench (LCB), Codeforces and HumanEval+.
“Our model demonstrates strong performance across all coding benchmarks… comparable to the performance of o3-mini (low) and o1,” the researchers write in a weblog put up that describes the mannequin.
Apparently, regardless of being skilled totally on coding duties, the mannequin exhibits improved mathematical reasoning, scoring 73.8% on the AIME 2024 benchmark, a 4.1% enchancment over its base mannequin (DeepSeek-R1-Distill-Qwen-14B). This means that the reasoning expertise developed via RL on code might be generalized successfully to different domains.
Credit score: Collectively AI
Essentially the most putting facet is attaining this degree of efficiency with solely 14 billion parameters. This makes DeepCoder considerably smaller and probably extra environment friendly to run than many frontier fashions.
Improvements driving DeepCoder’s efficiency
Whereas creating the mannequin, the researchers solved a few of the key challenges in coaching coding fashions utilizing reinforcement studying (RL).
The primary problem was curating the coaching information. Reinforcement studying requires dependable reward indicators indicating the mannequin’s output is right. Because the researchers level out, “Unlike math—where abundant high-quality, verifiable data is readily available on the Internet—the coding domain suffers from a relative scarcity of such data.”
To deal with this downside, the DeepCoder group applied a strict pipeline that gathers examples from totally different datasets and filters them for validity, complexity and duplication. This course of yielded 24,000 high-quality issues, offering a strong basis for efficient RL coaching.
The group additionally designed an easy reward perform that solely gives a optimistic sign if the generated code passes all sampled unit checks for the issue inside a particular time restrict. Mixed with the high-quality coaching examples, this outcome-focused reward system prevents the mannequin from studying tips like printing memorized solutions for public checks or optimizing for easy edge instances with out fixing the core downside.
The mannequin’s core coaching algorithm relies on Group Relative Coverage Optimization (GRPO), a reinforcement studying algorithm that proved very profitable in DeepSeek-R1. Nonetheless, the group made a number of modifications to the algorithm to make it extra steady and permit the mannequin to proceed enhancing because the coaching extends for an extended time.
GRPO+ allows DeepCoder-14 to proceed for longer durations with out collapsing Credit score: Collectively AI
Lastly, the group prolonged the mannequin’s context window iteratively, first coaching it on shorter reasoning sequences and step by step rising the size. In addition they developed a filtering methodology to keep away from penalizing the mannequin when it created reasoning chains that exceeded the context limits when fixing a tough immediate.
DeepCoder was skilled on 32K context issues however was additionally capable of resolve 64K duties Credit score: Collectively AI
The researchers clarify the core concept: “To preserve long-context reasoning while enabling efficient training, we incorporated overlong filtering… This technique masks out truncated sequences during training so that models aren’t penalized for generating thoughtful but lengthy outputs that exceed the current context limit.”
The coaching was step by step scaled from a 16K to a 32K context window, and the ensuing mannequin might additionally resolve issues that required as much as 64K tokens.
Optimizing long-context RL coaching
Coaching massive fashions with RL, particularly on duties requiring lengthy generated sequences like coding or advanced reasoning, is computationally intensive and gradual. A serious bottleneck is the “sampling” step, the place the mannequin generates probably 1000’s of tokens per instance within the batch. Variations in response size imply some responses end a lot later than others, leaving GPUs idle and slowing down your entire coaching loop.
To speed up this, the group developed verl-pipeline, an optimized extension of the open-source verl library for reinforcement studying from human suggestions (RLHF). The important thing innovation, which they name “One-Off Pipelining,” rearranges the response sampling and mannequin updates to cut back the bottlenecks and accelerator idle time.
One-Off Pipelining
Their experiments confirmed that one-off pipelining supplied as much as a 2x speedup for coding RL duties in comparison with baseline implementations. This optimization was essential for coaching DeepCoder inside an inexpensive timeframe (2.5 weeks on 32 H100s) and is now open-sourced as a part of verl-pipeline for the neighborhood to make use of and construct upon.
Enterprise influence
The researchers have made all of the artifacts for coaching and working DeepCoder-14B out there on GitHub and Hugging Face below a permissive license.
“By fully sharing our dataset, code, and training recipe, we empower the community to reproduce our work and make RL training accessible to all,” the researchers write.
DeepCoder-14B powerfully illustrates a broader, accelerating pattern within the AI panorama: the rise of extremely succesful but environment friendly and overtly accessible fashions.
For the enterprise world, this shift signifies extra choices and better accessibility of superior fashions. Reducing-edge efficiency is now not solely the area of hyperscalers or these prepared to pay premium API charges. Fashions like DeepCoder can empower organizations of all sizes to leverage subtle code era and reasoning, customise options to their particular wants, and securely deploy them inside their environments.
This pattern can decrease the barrier to entry for AI adoption and foster a extra aggressive and revolutionary ecosystem, the place progress is pushed via open supply collaboration.
Every day insights on enterprise use instances with VB Every day
If you wish to impress your boss, VB Every day has you lined. We provide the inside scoop on what corporations are doing with generative AI, from regulatory shifts to sensible deployments, so you possibly can share insights for optimum ROI.
An error occured.
