StarCoder

StarCoder is a large language model (LLM) designed specifically for code generation, developed by the BigCode collaborative (Hugging Face & ServiceNow). It features a 15.5 B parameter decoder-only architecture with Multi‑Query Attention and Fill‑in‑the‑Middle training on a trillion tokens from “The Stack,” covering 80+ programming languages . A refined version fine-tuned on 35 B Python tokens achieves ~40 % pass@1 on HumanEval, outperforming many commercial models .The second iteration, StarCoder2, offers models of 3B, 7B, and 15B parameters, trained on 3–4 T tokens across 600+ programming languages using advanced attention (Group Query) and extended 16K token context . The flagship 15B model reaches or surpasses the capabilities of larger proprietary models despite being more lightweight .