Privacy and preparation
International dataset entity names: keeping links across languages
By SourceX Editorial · Reviewed by Noah Loul ·
10 sections
Short answer
Link records to a stable entity key, never to a name string. Give each company one internal ID, attach an official identifier such as an LEI or a registry number where one exists, and store every name as a dated variant with its language tag, script, name type and source. Matching then runs on the key, so a rename, translation or transliteration adds a variant instead of creating a second company.
Countries and languages
International dataset entity
Key takeaways
- Join records on a stable entity key, not on any name string, and keep each record's name as written.
- Store every former, alternative-language, transliterated and trading name as a typed, dated variant with a BCP 47 language tag.
- Anchor keys to authority identifiers such as an LEI or a registry number stored with its jurisdiction.
- Merge two rows only on a shared authority identifier or recorded evidence, never on name similarity alone.
- Prove links survived masking by matching entity and join counts before and after packaging.
How do you keep international dataset entity names linked when a company is renamed or translated?
Stop joining on names. Every company gets one entity key that never changes, and every spelling, translation, transliteration and former name becomes a row in a variant table that points to that key. Records keep the name exactly as it was written at the time; the key does the linking.
This matters most when records cross borders. A supplier invoiced in German in 2019, renamed in 2022 and listed in an English-language CRM by its trading name can look like three companies to a buyer who receives the export. If the buyer trains an entity resolution or spend classification model on those rows, the model learns the split as if it were true.
The rest of this guide gives the mapping fields, the identifiers worth anchoring to, a worked example with an exception, the assumptions that can reverse a match, and the tests to run before a package leaves the building.
How does one company end up with several names?
A name string changes for reasons that have nothing to do with the entity changing. Knowing which kind of change you are looking at decides how the variant is typed and whether two rows may be merged.
At a glance
How does one company end up with several names
- Legal rename: the registered name changes on a date. Both names are correct, each for its own period, and records written before the date should keep the old one.
- Alternative-language legal name: some entities hold a registered name in more than one language. The global LEI format types this separately from the primary legal name and requires its language code to differ from the legal name's language.
- Transliteration: the same name written in another script, such as a Japanese katakana name written in Latin letters. Unicode's ICU documentation stresses that transliteration converts letters, not meaning, and that an inverse transform should not be assumed to give an exact reversal.
- Encoding differences: an accented letter can be stored as one code point or as a base letter plus a combining mark. Unicode Standard Annex 15 defines normalization forms such as NFC so that canonically equivalent strings get one representation; without normalization a byte comparison treats them as different names.
- Trading and short names: sales teams type the brand, the abbreviation or the name on the letterhead. These are aliases, useful for matching, and never evidence that a legal entity exists.
Which identifiers can anchor the entity key?
Anchor to an identifier issued by an authority when one exists, and record where it came from. The Legal Entity Identifier is defined in ISO 17442, a point schema.org also makes in its leiCode property, and the LEI record carries the legal name, other names and dated legal entity events. In the United States, the SEC treats a filer's CIK as permanent and notes that one CIK can be associated with several names after company name changes.
Internal system IDs are useful anchors inside one system and weak ones across systems. A vendor ID in an ERP survives a rename, but two vendor IDs for the same supplier are common after a migration or an acquisition, so they still need a reviewed merge.
In practice most international packages use a ladder. Take the authority identifier where one exists, fall back to a registry number paired with its issuing jurisdiction, and use a reviewed internal key for everything else. Record which rung each entity sits on, because a buyer judging match quality will treat an LEI-anchored link differently from one a steward confirmed by reading invoices.
| Identifier | What it anchors | Survives a rename? | Watch for |
|---|---|---|---|
| LEI (ISO 17442) | A legal entity in the global LEI system | Yes; former names are kept as typed other names | Many smaller private companies have none; check the record's status before relying on it |
| National company registry number | A registration in one jurisdiction | Usually yes within that registry | Not unique across countries; always store it with the issuing jurisdiction |
| SEC CIK | An EDGAR filer account | Yes; the SEC says one CIK may carry several names after name changes | Covers SEC filers only; a CIK is not proof of a current legal name |
| Tax ID | A taxpayer in one country | Usually yes | Often confidential or personal for sole traders; keep it out of a licensed package unless counsel clears it |
| ERP or CRM record ID | A row in one system | Yes inside that system | Duplicates after migrations and acquisitions; merges need review |
| Web domain | A brand presence | Often not | Rebrands, regional domains and resold domains make it an alias, never a key |
Which fields belong in the entity identity and language variant map?
The mapping is one table of name variants and one table of links from source records to entity keys. The fields below are the variant table. Each row is one string as it actually appeared, with enough context that a reviewer in another country can tell what it is without asking the person who built it.
At a glance
Which fields belong in the entity identity and language variant map
Name types borrow from the LEI format, which separates previous legal names, alternative-language legal names and trading names from transliterations, and splits transliterations into those the entity supplied and those generated automatically. Language tags follow BCP 47 (RFC 5646), which uses a four-letter script subtag such as Latn so that a tag like sr-Latn says Serbian written in Latin script.
| Field | What to record | Rule | Illustrative value |
|---|---|---|---|
| entity_key | Your stable internal ID for the legal entity | Never reused, never derived from the name | ENT-000412 |
| name_text | The string exactly as the source wrote it | Store as found and in NFC-normalized form; never edit the original | Nordhafen Präzisionstechnik GmbH |
| name_type | legal, previous_legal, alternative_language_legal, trading, transliterated_preferred, transliterated_auto, internal_alias | One type per row; trading names never become legal | previous_legal |
| language_tag | BCP 47 tag including script when it matters | Tag the language of the original before any transliteration | de-Latn |
| transliteration_source | Who produced a Latin or other-script form, and with which transform or rule | Required for transliterated rows; blank otherwise is an error | entity supplied, letterhead 2023 |
| valid_from and valid_to | The legal effective dates of the name | Use the effective date, not the date you learned of it | 2016-03-01 to 2022-06-30 |
| recorded_on | When the change entered your source or the registry | Kept separate from the effective date | 2022-07-14 |
| jurisdiction | Country and, where relevant, region of registration | ISO 3166 style codes; required for any registry ID | DE |
| evidence | Registry extract, LEI record, contract or invoice that shows the name | At least one per legal or previous_legal row | Registry extract dated 2022-07-14 |
| review_status | proposed, confirmed, rejected, with reviewer and date | Only confirmed rows drive links | confirmed, data steward, 2026-10-02 |
How do you record a rename without rewriting history?
Keep historical records as they were written and let the entity key carry the continuity. An invoice from 2019 should still say the 2019 name, because a model learning invoice extraction or vendor matching needs to see the name a clerk actually saw.
Separate two dates for every rename. The LEI format distinguishes the date a legal entity event became legally effective from the date it was recorded in the system, and the two can differ. Your own records have the same gap: the ERP vendor master is often updated weeks after the legal change, and transactions in between carry a mix of names.
Never overwrite the name field in place to make the history look tidy. If a cleanup script replaced old names with the current one, recover the original strings from audit logs or document images before packaging, or flag the affected period so the buyer knows the names were rewritten.
- Close the old legal name with valid_to and add the new one with valid_from.
- Keep the old name as a previous_legal variant pointing to the same key.
- Link every transaction by key; leave the name in the transaction untouched.
- Record which system changed first and when, so gaps between legal and system dates are explainable.
Worked example (illustrative): one supplier, four spellings and a customer in two scripts
This example is invented to show structure. A machine shop exports seven years of purchasing and sales records from its ERP and CRM: purchase orders, invoices and quotes. A German supplier renamed itself in 2022, and a Japanese customer appears in both katakana and Latin script.
Before mapping, a name-based count shows 7 distinct counterparties across these rows. After mapping, the count is 2 entities with 7 confirmed variants. The figures are illustrative counts, not observed data.
The steward works in a fixed order. First, every name string is copied as found and an NFC comparison copy is made, which collapses nothing here because the umlaut and the folded ue spelling are different letters, not different encodings. Second, each string is typed: the scanned invoice spelling is an automatic transliteration produced by an old printer driver, so it is typed transliterated_auto rather than legal. Third, the registry extract dated after the rename confirms the new legal name and its effective date, which closes the old name on 2022-06-30. Fourth, the Japanese customer's katakana name is confirmed as legal from the quote's signature block, and the Latin form is confirmed as the one the customer prints on its own invoices, so it is typed transliterated_preferred.
Only then are transaction rows linked by key. Purchase orders issued in the weeks between the legal change and the ERP update still carry the old name, and that is left as it is.
| Source row | Name as recorded | Language and type | Resolves to |
|---|---|---|---|
| PO 2019, ERP | Nordhafen Präzisionstechnik GmbH | de-Latn, previous_legal | ENT-000412 |
| Invoice 2020, scanned PDF | Nordhafen Praezisionstechnik GmbH | de-Latn, transliterated_auto (umlaut folded) | ENT-000412 |
| PO 2023, ERP | Nordhafen Präzision GmbH | de-Latn, legal from 2022-07-01 | ENT-000412 |
| CRM account | Nordhafen Precision | en-Latn, trading | ENT-000412 |
| Quote 2021, CRM | カワセ精機株式会社 | ja-Jpan, legal | ENT-000977 |
| Invoice 2021, ERP | Kawase Seiki Co., Ltd. | ja-Latn, transliterated_preferred | ENT-000977 |
| Email signature import | Kawase Seiki | ja-Latn, internal_alias | ENT-000977 |
What if two different companies share a translated name?
A shared name in translation is not evidence of a shared entity. In the same illustrative export, a second supplier appears as Nordhafen Precision Ltd, registered in another country. Its English trading name matches the German company's CRM alias exactly.
The rule that resolves it: two rows merge only when they share an authority identifier, or when a reviewer records evidence that ties them, such as a contract naming both or a registry entry showing the rename. Jurisdiction disagrees here, the registry numbers differ, and no document links them, so the second supplier gets its own key, ENT-001208. A string-similarity score near 1.0 does not override that.
Which language, date or jurisdiction assumption could reverse a match?
Most wrong merges and wrong splits trace back to an unstated assumption. Write each one down in the mapping notes and test it on a sample before trusting the links.
At a glance
Which language, date or jurisdiction assumption could reverse a match
A wrong merge usually costs more than a wrong split. A split leaves a buyer with two partial histories that a later review can join; a merge mixes two companies' transactions, payment terms and disputes into one history that no model can untangle. When the evidence is thin, keep the entities apart and mark the pair as a candidate for review.
- Script assumption: treating a transliterated name as the legal name. Transliterations vary by scheme and by who produced them, which is why the LEI format records whether the entity or the issuer supplied the Latin form.
- Normalization assumption: comparing raw bytes across systems that store accented letters differently. Normalize a comparison copy to NFC and keep the original string untouched.
- Legal-form assumption: stripping or translating suffixes such as GmbH, S.A. or Co., Ltd. to make names match. Two entities can share a base name and differ only by legal form and country.
- Date assumption: reading 03/04/2022 as March or April depending on locale moves a rename across a transaction. Store ISO 8601 dates and record the source locale.
- Jurisdiction assumption: assuming a registry number is unique worldwide. It is unique only within the registry that issued it.
- Group assumption: merging a parent and a subsidiary because they share a brand. They are separate legal entities and separate counterparties, even when the CRM treats them as one account.
How do you prove the links survived de-identification and packaging?
Run the same counts before and after every transformation and ship the results with the package. If masking or tokenization breaks a link, the counts change and you find out before the buyer does.
Decide early which side of the mapping travels. The variant table itself is identifying, because it lists every name a company has used. When names are masked, keep the variant table with the supplier, ship a consistent token per entity key, and pass along only non-identifying fields such as name_type counts or a rename flag if the buyer's task needs them.
- Distinct entity count by key equals the count before masking.
- Rows per entity and joins between purchase orders, invoices and payments return the same totals.
- No transaction row is linked to a variant whose review_status is not confirmed.
- Every transliterated variant has a language tag and a transliteration_source.
- Every rename has both an effective date and a recorded date, or a note explaining which is missing.
- A spot check of 20 to 50 resolved entities by a reviewer who reads the relevant languages.
Who signs off on a multilingual entity map before it is licensed?
A data steward can build the map, but two other reviews decide whether it goes out. A localization or data expert who reads the scripts involved should confirm the transliterated and alternative-language rows. Counsel should look at any variant that names an individual, such as a sole trader whose business name is a personal name, and at registry extracts whose own terms of use may limit redistribution.
Keep the distinctions clear. Having access to the registry or your own ERP does not mean you may license what is in it; de-identifying names does not by itself create a right to license the records; and contract terms with customers and suppliers can restrict use regardless of who owns the database. None of this is legal advice; counsel applies it to the specific registries, contracts and people involved.
For SourceX, the language mix in these tables is part of the assessment rather than a side issue. On eligibility, U.S., Canadian and EU companies qualify for SourceX if they had 50 or more full-time employees at their peak, not counting contractors, and companies in the UK, Australia, New Zealand, India, Singapore, South Africa, Latin America, the UAE and Saudi Arabia can be considered if the data they offer is at least 90% English, measured on the records actually offered. Every dataset still goes through rights, privacy, quality and buyer-fit review. A company can start with a short assessment of its records, and no files have to be uploaded for that first check.
Frequently asked questions
Should I translate company names into English before licensing records?
No. Keep the name as the source wrote it and add an English or Latin-script form as a separate, typed variant if it helps matching. Replacing the original destroys the evidence a buyer's model needs and can turn a transliteration into an apparent legal name.
What is the difference between a translated name and a transliterated name?
A transliteration writes the same name in another script without changing its meaning, while a translation renders the meaning in another language. Unicode's ICU documentation makes the same point: transliteration converts letters and does not translate words. Type them differently in the variant table, because only an entity-registered translation counts as an alternative-language legal name.
Is an LEI enough to resolve every company in a dataset?
Rarely. LEIs are strong anchors where they exist, and the record keeps former and other-language names, but many small private suppliers and customers do not have one. Use the LEI where available, a national registry number with its jurisdiction next, and a reviewed internal key for the rest.
How should a company rename be dated in a dataset?
Record the legal effective date and the date the change was recorded as separate fields. The LEI format keeps the same distinction for legal entity events. Transactions between the two dates can carry either name, and that is expected rather than an error.
Can I ship the entity name mapping to a data buyer?
Only if the buyer's task needs real names and the agreement allows it. In most de-identified packages the variant table stays with the supplier and the buyer receives a consistent token per entity, plus non-identifying flags such as whether a rename occurred.
Sources
- The LEI-CDF 3.1 format records other entity names, including alternative-language legal names with a language different from the legal name, and legal entity events with a legally effective date separate from the date recorded in the Global LEI System. Source
- GLEIF validation rules require ASCII-transliterated names when the legal name is not romanized, typed as auto-transliterated by the LEI issuer or preferred (supplied by the entity), and keep previous legal names distinct from the current legal name. Source
- The SEC's CIK list is historically cumulative for company names, so a given CIK may be associated with multiple names after company or fund name changes. Source
- The SEC describes the CIK as a unique, permanent filer identifier that never expires. Source
- schema.org defines legalName as the official registered name of an organization, alternateName as an alias, and leiCode as an identifier for a legal entity as defined in ISO 17442. Source
- ICU documentation states that script transliteration converts letters from one script to another without translating the words, and that an inverse transform should not be assumed to give a complete or exact reversal. Source
- Unicode Standard Annex 15 defines normalization forms (NFC, NFD, NFKC, NFKD) so that canonically equivalent text has a single representation. Source
- BCP 47 (RFC 5646) language tags use four-letter script subtags from ISO 15924; sr-Latn denotes Serbian written in Latin script. Source
See if your company qualifies
A short company assessment. No data uploads are needed.