OpenAI Improves Coding in Low-Resource Languages
OpenAI has announced significant model updates and evaluations designed to improve software development and migration for low-resource programming languages.
Radwan AbdulhadiEditor and technology writer
Addressing Critical Infrastructure Languages
OpenAI has rolled out model improvements aimed at boosting software engineering capabilities for low-resource programming languages. Languages such as COBOL, Fortran, and Visual Basic possess less code and documentation for models to learn from compared to widely used options like Python and JavaScript. Despite this limitation, they continue to support critical global infrastructure and essential services. According to OpenAI News, COBOL underpins over 40% of online banking systems, 80% of in-person credit card transactions, and 95% of ATM transactions, while Fortran plays vital roles in weather forecasting and nuclear reactor safety analysis.
Maintaining and modernizing this legacy software presents unique challenges for development teams. In many cases, documentation is incomplete, original developers have departed, and essential business rules reside exclusively within the code. Guided by feedback from enterprise customers and developers, OpenAI sought to measure and improve how models handle bug fixes, maintenance tasks, and migrations while preserving system behavior.
Introducing MultiLangBench and Migration Evaluations
To systematically track progress, OpenAI developed two distinct evaluations to test complementary skills: building new software and preserving the behavior of existing systems. MultiLangBench tasks models with building a SQLite-compatible database engine from scratch across 20 different programming languages. To ensure a uniform comparison, the candidate programs receive an initial database and a SQL query, and a reference SQLite implementation verifies whether the resulting database matches expected contents.
The second evaluation focuses on migrating COBOL programs to alternative languages such as C++, Java, Kotlin, Python, or Rust. Successful migrations require strict byte-for-byte parity on observable outputs, demanding that models thoroughly understand both underlying business logic and specific language behaviors, such as COBOL's fixed-point arithmetic.
Performance Gains Across Models
Latest model iterations have demonstrated substantial improvements across both evaluation frameworks, yielding higher accuracy alongside reduced API costs. On MultiLangBench, GPT-6 Astra achieved an average test pass rate of 87.4% across the 20 tested programming languages at maximum reasoning effort, improving significantly from the 26.8% recorded by GPT-5.6 Sol. Furthermore, improvements extended to every tested language, including notable gains in COBOL, OCaml, Fortran, and Visual Basic.
For cost-conscious workflows, GPT-6 Luna offers an alternative that also outperforms previous generations across every language tested. Luna reached an average test pass rate of 40.3% while operating at an estimated average API cost of $0.77 per sample, representing an approximate 78% cost reduction compared to GPT-5.6 Sol.
COBOL Migration Accuracy and Cost Efficiency
On the dedicated COBOL migration evaluation, GPT-6.1 Sol delivered major accuracy jumps over older models. At low reasoning effort, GPT-6.1 Sol achieved an 86.6% accuracy rate, compared to just 5.4% for GPT-5.6 Sol while running at a lower API cost. When configured for higher reasoning effort, the model reached 97.9% accuracy, bringing the evaluation close to complete saturation.
Developers can inspect the evaluation methodology and review specific performance breakdowns. OpenAI is making the MultiLangBench evaluation setup and test suite publicly available on GitHub, allowing engineering teams to identify performance gaps relevant to their specific operational requirements.
Enterprise Software Engineering Outlook
These benchmark developments reflect ongoing efforts to help models apply core software engineering competencies across a wider variety of programming languages. These capabilities include interpreting unfamiliar repositories, formulating change plans, utilizing build tools, executing tests, and debugging errors.
Developers can test the updated models directly through the API on coding tasks relevant to their technology stacks. OpenAI continues to invite customer feedback regarding model performance, persistent operational challenges, and target languages for future training and evaluation cycles.
Sources
- OpenAI NewsAdvancing Coding in Low-Resource Programming Languages
Continue chronologically
Related entity coverage
Related stories
OpenAI Says Its Models Can Help Banks Migrate From Old COBOL
OpenAI announces that its models now feature improved capabilities for helping financial institutions migrate legacy COBOL software to modern languages.
OpenAI Boosts Security Bug Bounty Rewards and Bonuses
OpenAI announces higher bug bounty rewards, new report quality bonuses, a tiered loyalty program, and priority campaigns for security researchers.
OpenAI Introduces LifeVal Benchmark for Everyday Tasks
OpenAI has introduced LifeVal, a new benchmark designed to measure how effectively artificial intelligence assists people with daily tasks.
OpenAI Adds Stricter Limits For Teens Asking About Firearms
OpenAI announces stricter restrictions for teens asking ChatGPT about weapons and firearms, applying tighter limits even when questions appear harmless.
