Openly Licensed Text Corpora for Commercial LLM Training: What They Cover and Where They Stop

This guide sits under text datasets for LLM training and catalogs the corpora themselves. License-compatibility rules and volume planning are covered in licensed text corpora for LLM pre-training.

The four corpora that define the openly licensed baseline

As of October 2026, four corpora are the reference points for a license-clean text baseline, and each was built with a different filter. Knowing the filter matters more than the headline size, because the filter determines what your legal reviewers still have to check.

The Common Pile result is the strongest evidence that the approach is viable: the authors report that their 7B-parameter Comma v0.1 models are competitive with models trained on unlicensed text at similar compute budgets [1]. That is a parity claim at one scale, not a guarantee for frontier-scale runs or for domains the corpus barely covers.

How license filters differ: SA, NC and ND are the deciding clauses

The practical difference between "open" corpora is which Creative Commons modules they let through, and the strictest commercial filter excludes NC, SA and ND. GPT-NL is explicit that it drops NC and SA material to keep the corpus commercially usable [3], while broader "openly licensed" definitions can retain share-alike text. If your model weights or outputs could be argued to be adaptations, share-alike terms raise questions your counsel should answer before mixing.

NoDerivatives is the quiet trap. Recent work on low-resource corpus licensing notes that an ND clause can be read to forbid tokenization, annotation and other transformations, which means an ND component can be unusable for training even when commercial use is permitted [8]. Check ND-licensed subsets separately rather than trusting a corpus-level label.

NC is the obvious exclusion, but it hides in mixed collections. The PubMed Central Open Access Subset carries machine-readable Creative Commons or similar licenses that allow broader reuse than the rest of PMC [7], yet that subset mixes commercial and non-commercial terms. Filter on the per-article license field, not on membership in the subset; the open-access scientific full-text guide covers those license tiers.

Dataset-level licenses do not settle rights in the underlying text

A permissive license on a dataset card covers the compilation, not necessarily every document inside it. Web-derived corpora such as C4 and RefinedWeb publish dataset-level licenses, but those terms come from the distributor and do not convert the copyrighted pages they contain into openly licensed text. They do not belong in a license-clean baseline unless your counsel has accepted a separate legal theory for them.

The reverse also happens. Pile of Law collects about 256GB of court opinions, filings, agency publications, statutes and casebooks from 35 sources [5]; much of the underlying government text is public domain, yet the compiled dataset card carries its own terms. Read both layers.

Label errors are common enough to plan for. The Data Provenance Initiative audited more than 1,800 text datasets and found license omission above 70% and license error rates above 50% on popular hosting sites [6]. Treat the card as a lead, then verify against the original source's terms; the copyright-status classification guide gives a four-way scheme for recording the result.

Build the baseline with a per-document license manifest

The safest way to use open corpora commercially is to carry license metadata down to the document, so a later filter change or takedown can be applied without re-crawling. Most openly licensed corpora preserve a source and license field; keep them through deduplication and tokenization rather than dropping them at the shard stage.

Illustrative example: invented to show structure; it does not describe an available dataset.

{
  "doc_id": "cp-v0.1:pubmed-oa:000184223",
  "corpus": "common-pile-v0.1",
  "upstream_source": "pmc-oa",
  "upstream_url": "https://example.org/article/000184223",
  "license_spdx": "CC-BY-4.0",
  "license_evidence": "article XML <license> element",
  "nc": false, "sa": false, "nd": false,
  "attribution_required": true,
  "attribution_string": "Author A, Author B (2021), Journal X",
  "language": "en",
  "first_published": "2021-05-14",
  "copyright_status": "licensed-open",
  "filter_version": "baseline-2026-10-a",
  "dedup_cluster": "minhash-7f3c91"
}

Use SPDX identifiers (CC-BY-4.0, CC-BY-SA-4.0, CC0-1.0, MIT) so policy filters are machine-checkable. Keep filter_version so you can show which rule set produced a given training mix. The metadata fields guide lists the fields to require when you later add licensed corpora on top.

Commercial-use filter checklist

• Drop any document whose license includes NC, unless counsel has approved a specific exception.
• Route SA and ND documents to a separate review queue instead of silently including them.
• Reject documents with no license evidence beyond the dataset card.
• Preserve attribution strings for CC BY material and confirm how you will satisfy attribution.
• Record first_published so you can show the temporal coverage and find post-cutoff gaps.
• Rerun near-duplicate detection across corpora; Common Pile and Common Corpus can overlap on digitized public-domain books.

Where open corpora stop: the coverage gaps to license

Open corpora are structurally thin wherever the text is commercially valuable, recent or private, and those gaps decide what you license next. Public-domain collections lean on older books and government documents, and openly licensed collections lean on research papers, code and encyclopedic text [1][4]. Neither source type produces much of how businesses actually write.

Gaps that pre-training teams typically find after building the baseline:

• Professional and business registers. Support transcripts, sales emails, engineering tickets, contracts, invoices and internal procedures are almost never released under open licenses. See proprietary text beyond web crawls.
• Post-cutoff content. Public-domain text outside government works is mostly old, and openly licensed sources update unevenly, so recent terminology and events are underrepresented.
• Domain-specific non-English text. German Commons and GPT-NL show that open non-English corpora are possible, but per-language volumes trail web alternatives [2][3], and domain text in those languages is thinner still. The non-English text licensing guide covers the commercial route.
• Dialogue and multi-turn interaction. Transcripts in open corpora are mostly lectures and proceedings, not two-party working conversations.

Before buying, test whether a candidate licensed corpus actually adds novelty over what you already hold; the method is in testing licensed data for overlap with public web corpora.

Documentation duties apply regardless of license

Using only open text does not remove disclosure duties; it makes them easier to meet. Under EU AI Act Article 53(1)(c), general-purpose model providers must maintain a copyright compliance policy that respects machine-readable reservations under Article 4(3) of the DSM Directive, and Article 53(1)(d) requires a public summary of training content [9]. The Commission's template for that summary is dated 24 July 2025 [10]. As of October 2026 these duties apply, with AI Office enforcement for new models from 2 August 2026.

In California, AB 2013 requires developers of generative AI systems made available to Californians to post documentation about training data, with a deadline of January 1, 2026 [11]. A per-document manifest like the one above lets you answer both regimes from the same records. For the broader trade-off between open, licensed and scraped inputs, see licensed vs synthetic vs scraped data and public vs proprietary data.

This page is general information, not legal advice. Confirm requirements with counsel for your jurisdiction and use case.

Filling gaps with licensed operational text

Once the open baseline is fixed, the remaining gaps are usually operational text held by companies, and that requires a negotiated license rather than a download. SourceX sources operational datasets from US companies on request, including support and sales histories, engineering records, documents, and finance and legal workflows; nothing is held in stock and a request does not guarantee a match. Each dataset is rights-reviewed for ownership and consents and delivered under a license defining records, uses, term and delivery. You can describe the text your baseline lacks without naming specific companies. Foundation-model pre-training teams can scope these requests by register, language and date range.

Sourcing the text open corpora do not cover

SourceX helps AI teams wherever they are based find US companies that hold the operational text an open baseline lacks, then manages the process from assessment through licensing and ongoing purchases. Every release is approved by the supplying company, and personal details are removed or replaced before delivery. Start a buyer request at sourcex.si/buyers.