Code and Software Engineering Datasets for LLM Training: A Buyer's Map

Eight kinds of code data and what each one trains

Sort code data by the engineering record it comes from, because the record decides the training unit, the use and the main risk.

The issue-to-fix row follows the shape SWE-bench made standard: 2,294 problems drawn from real GitHub issues and the pull requests that resolved them across 12 Python repositories, with each fix checked by tests [1]. A private equivalent means the issue text, base commit, reference diff and an environment where the tests run, as listed on the software engineering histories page.

Two rows are thin in public data. SQL quality depends on real schemas: Spider 2.0 builds its tasks on enterprise databases that often have more than 1,000 columns, on systems such as BigQuery and Snowflake, and some solutions exceed 100 lines of SQL [2]; question-to-SQL pairs are covered in text-to-SQL training data. Public log collections such as Loghub gather runtime logs from distributed systems, supercomputers, operating systems and other software [3], not build-and-test runs tied to the commit that fixed them.

Why public repositories no longer settle the question

Public code is cheap for pre-training but unreliable for evaluation and thin on the work coding agents do. The SWE-Bench Pro authors argue that widely used open-source repositories, especially permissively licensed ones, are prime candidates for the web-crawled corpora used in pre-training, so benchmarks built from public GitHub repositories are hard to keep clean [4]. They built public and held-out sets only from repositories under strong copyleft (GPL) licenses and added a commercial set from codebases acquired from startups, publishing results while that code stays private [4].

In a post dated 23 February 2026, OpenAI said it had stopped reporting SWE-bench Verified because score gains increasingly reflected exposure to the benchmark during training; it recommends SWE-bench Pro for now and calls Pro's contamination considerably lower but not perfect [5]. The LiveBench authors cite evidence that model performance on Codeforces problems drops sharply for problems released after a model's training cutoff [6].

Open baselines exist. The Stack collects 3.1 TB of permissively licensed source code in 30 programming languages, filtered by repository license, with an "Am I in The Stack" search and a removal process [7]. The Stack v2, built from the Software Heritage archive, is distributed as a gated Hugging Face dataset; check its dataset card for access terms and download steps [8]. The Common Pile v0.1 includes code among the 30 openly licensed and public-domain sources in its 8 TB collection [9].

Developer community content is gated too: Stack Exchange posts are licensed CC BY-SA, yet in 2024 the company put its data dump behind a login and an agreement not to use the content to train AI models [10]. Public license labels are weak evidence: the Data Provenance Initiative found licenses omitted for more than 70% of popular datasets on hosting sites and error rates above 50% [11].

Synthetic code, the third route, carries its provider's output terms; as of October 2026, Anthropic's help center, for example, says its terms do not allow outputs to train models that compete with its own [12]. Compare the routes in open code datasets vs licensed private code and synthetic vs licensed real code.

What each training stage needs from a code record

Specify the training stage first: pre-training buys deduplicated tokens, fine-tuning buys linked task pairs, reinforcement learning buys runnable environments, and evaluation buys tasks no model has seen.

Test quality decides the value of RL and evaluation tasks. When it introduced SWE-bench Verified, a 500-task human-validated subset built with the SWE-bench authors, OpenAI cited overly specific unit tests, underspecified problem descriptions and unreliable environment setup in the original tasks, and shipped a Docker-based harness [13]. Ask for the same screening on private tasks; see held-out evaluation sets from private repositories and the LLM evaluation datasets hub.

Forks, vendored dependencies and copied snippets make code heavily duplicated. In natural-language corpora, Lee et al. found that deduplicated training data cut memorized output about tenfold [14]; see deduplicating code training data and data sourcing for post-training teams.

Seven risks that separate licensable code from risky code

Private code is worth licensing only if the supplier can show it owns the code and has removed what it may not share, and the buyer can keep it from resurfacing verbatim.

Verbatim extraction is documented: Carlini et al. recovered hundreds of verbatim training sequences from GPT-2, including code and UUIDs [15].

N-gram overlap is the most common decontamination technique, with thresholds that vary by lab, such as 13-gram matches or 50-character overlaps [16]. The LMSYS team showed that rephrased test items slip past n-gram checks, and its LLM-based decontaminator found benchmark overlap in corpora including The Stack [17].

Keep provenance per repository, because model developers' disclosure duties depend on knowing where training data came from. As of October 2026, under the EU AI Act, providers of general-purpose AI models must publish a summary of their training content; the European Commission published a template for this on 24 July 2025, which calls for listing main data collections and other sources [18]. California's AB 2013 has required developers of generative AI systems made available to Californians to post training-data documentation since 1 January 2026, including whether datasets contain copyrighted or licensed material or personal information [19].

This page is general information, not legal advice. Confirm requirements with counsel for your jurisdiction and use case.

Who holds private code, and who must approve its release

Private code sits with three kinds of holders, and each has a different approval chain. Software product companies usually own their code but embed third-party components. In-house teams at non-software businesses hold internal tools and legacy estates; customizations of a vendor platform need a check of the vendor's terms. Agencies and contractors often build code their clients own, so the client's authorization comes first.

SourceX sources operational datasets from US companies, including engineering records, and manages the commercial process, from the licensing agreement to ongoing purchases. Buyers describe the data rather than the businesses: SourceX looks for US companies that hold it, and every release is approved by the supplying company. Datasets are sourced on request, not held in stock, and a request does not guarantee a matching dataset.

Every dataset goes through rights review, which checks that the business owns or may share the records and that required consents are in place, and is delivered under a license that defines the records, permitted uses, term and delivery. Personal details such as names and emails are removed or replaced before delivery, with the method recorded and a processed sample checked, though no de-identification method is perfect. SourceX does not source scraped public web content, does not train AI models and does not publish prices. You can describe the private code or engineering records you need.

Record-level detail is on the pages for proprietary codebases with full git history, coding agent training data, licensing source code, licensing code reviews and pull requests and whether AI labs buy code.

Mistakes code data buyers make

These mistakes come from measuring the wrong unit, accepting the wrong shape of data or trusting an unchecked label.

• Pricing by lines of code. Counts that include vendored, generated and minified files inflate volume; see what drives the price of licensed code data.
• Accepting a snapshot when you need history. A tree of the latest files has no commits, no issue links and no base commits for tasks; see delivering repositories with full history.
• Trusting the repository LICENSE file for every file. Copied snippets and vendored directories carry their own terms, and dataset-level labels are often missing or wrong [11].
• Comparing scores across benchmark versions. Task sets and harnesses differ between SWE-bench, SWE-bench Verified, and SWE-bench Pro, so scores do not transfer. OpenAI stopped reporting on SWE-bench Verified due to flawed tests and contamination, recommending SWE-bench Pro instead [13][5].

Start here: code data guides by buying step

The guides below follow the order of a purchase, from writing the request to settling rights. For other data types, start from the SourceX guide to AI data.

Need private code or engineering histories for your model?

Describe the languages, repository types, history depth, linked records (issues, reviews, CI runs), test requirements, volume and permitted uses you need. SourceX looks for US companies that hold that code, checks their licensing permissions, and manages the license and delivery. Send SourceX your code dataset specification.