Leadership and readiness
How to check an AI company's data security before sharing anything
By SourceX Editorial · Updated
Short answer
To evaluate an AI company's data security before sharing anything, get written answers on five points: where your records will be stored, who can access them, which subprocessors touch them, how and when they are deleted, and how fast you will hear about an incident. Share nothing beyond descriptive metadata until those answers and confidentiality terms are in place.
Key takeaways
- Share nothing beyond descriptive metadata until security answers are in writing and confidentiality terms are signed.
- Ask for a description of the environment where licensed data will live, not a marketing security page.
- Annotation and labeling vendors are subprocessors, so require a named list and notice before changes.
- Deleting raw files does not undo what a trained model has learned, so the contract must say what deletion covers.
- A SOC 2 report or ISO 27001 certificate covers a defined scope; check that the scope includes the team handling your data.
What should a CTO verify before any data leaves the company?#
A CTO should verify five things before any records leave the company: storage, access, subprocessors, deletion and breach notice. Together they answer one question: once a licensed dataset arrives at the buyer, who can see it, where can it go, and what happens when the license ends.
The order matters. The security review comes before samples, not after a pilot is already running. Descriptive metadata such as system names, record families and years of history can be discussed safely; ticket text, job notes and source code cannot.
- Storage: the environment, region and accounts where your data will live.
- Access: the named team and roles that can open the files, and how access is logged.
- Subprocessors: every vendor that will touch the data, including annotation and cloud providers.
- Deletion: what is deleted at the end of the term, how, and with what confirmation.
- Breach notice: who tells you about an incident, how quickly under the contract, and what cooperation follows.
Storage and access: where the data lives and who can open it#
Storage questions should produce a plain description of the environment. Ask whether your dataset will sit in a segregated bucket or project, whether it is encrypted at rest and in transit, which cloud region holds it, and whether copies can land on researcher laptops or shared notebooks.
Access questions narrow the circle. A buyer should be able to say which team works with supplier data, how access is granted and revoked, whether multi-factor authentication is enforced and whether access is logged in a form it can show you. Contractors deserve the same scrutiny as employees.
Watch for one common gap: data that is well controlled in storage but copied freely once it enters a training pipeline. Ask how licensed data is tracked from storage into training runs, and whether it is mixed with other suppliers' records.
Subprocessors and onward transfers#
Subprocessors are the most overlooked part of a buyer's security posture. Many AI developers use outside vendors for labeling, annotation, evaluation and storage, and each vendor is one more place your records can leak from.
Require a current subprocessor list, notice before a new one is added and written flow-down of the same confidentiality and use restrictions the buyer accepted. If a buyer will not name the vendors whose people would read your records, treat that as a reason to narrow the scope or stop.
| Subprocessor type | Why it matters | What to require |
|---|---|---|
| Cloud and storage providers | Hold the raw files and backups | Named provider and region, encryption, no change without notice |
| Annotation and labeling vendors | People outside the buyer read your records line by line | Named vendors, confidentiality terms, your consent for sensitive sets |
| Evaluation and red-teaming contractors | May receive samples to test models | Same restrictions as the buyer, flowed down in writing |
| Research collaborators | Academic or partner teams may request access | No access without your written approval |
Deletion: what ends when the license ends#
Deletion terms must say exactly what is deleted when the license ends: raw files, working copies, derived datasets, backups and any samples shared during evaluation. Ask for a certificate of deletion signed by an officer, and ask how backups are handled when they cannot be purged immediately.
Be precise about trained models. Deleting raw records does not remove what a model has already learned from them. Whether the buyer may keep using models trained during the term is a commercial term to settle in the license, not something a deletion clause can quietly cover.
Also require a subprocessor list with every deletion certificate, so you know the copies held by annotation and storage vendors were removed too.
Breach notice and incident response#
Breach notice terms decide whether you hear about an incident in time to act. The contract should name an incident contact on each side, set a notice period you and counsel are comfortable with, and require the buyer to explain what happened, which records were involved and what it is doing about it.
Settle responsibility for notifying affected people before anything is shared. If de-identified records are re-identified in a breach, notification duties under state breach and privacy laws may still apply, and cooperation and cost terms are far easier to agree in advance than during an incident.
Which evidence should you ask for?#
The strongest evidence is independent and clearly scoped. A SOC 2 Type II report or an ISO 27001 certificate shows that controls were examined by an outside party, but only within the scope the report defines, so check that the scope covers the systems and teams that will handle your records.
Read a SOC 2 report for what it actually tests. It is an attestation report by a CPA firm, not a certification, against the AICPA Trust Services Criteria, which cover five categories: security, availability, processing integrity, confidentiality and privacy. Security applies in every report; the others are included only if the company chose them, so check whether confidentiality and privacy are in scope. A Type 2 report also tests whether controls operated effectively over a specified period, which a Type 1 report does not.
| Evidence | What it shows | Limit to check |
|---|---|---|
| SOC 2 Type II report | Controls tested by an auditor over a period | Scope may exclude research or training environments |
| ISO 27001 certificate | A certified information security management system | Certificate scope and statement of applicability |
| Completed security questionnaire | The buyer's own answers to your questions | Self-reported; ask for supporting documents on key points |
| Penetration test summary | Recent outside testing and remediation | Summaries omit detail; ask what was in scope |
| Data processing terms | Contractual commitments on use, deletion and notice | Must match what the security team actually does |
Illustrative: a SaaS CTO reviews a buyer before a sample#
Illustrative: the CTO of a fictional vertical SaaS company for property inspections is asked for a sample of support tickets linked to Jira issues. Before agreeing, the CTO sends a short questionnaire covering the five areas and asks for the buyer's SOC 2 report.
The report covers the buyer's production platform but not its research environment, and the questionnaire reveals that an outside annotation vendor would read the tickets. The CTO asks for the vendor's name, a commitment that the sample stays in a segregated project in one region, deletion of all copies after evaluation and an officer-signed certificate.
The buyer agrees to everything except naming a second, occasional vendor. The CTO approves the sample on condition that only the named vendor may access it, and the general counsel writes that restriction into the evaluation agreement before the de-identified tickets are released.
How SourceX handles buyer security#
SourceX treats buyer security as part of the Approval step in the SourceX five-step transaction. Before approving any release, the supplier sees who will receive the data, under what permitted use and with which deletion terms, and nothing is shared during the initial assessment.
Large datasets stay in the supplier's own storage or ship on encrypted drives rather than passing through SourceX. The SourceX Evidence Packet records the permitted use, the privacy record and the release authorization, so the security terms a supplier approved sit alongside the record of what was delivered.
Frequently asked questions
Is a SOC 2 report enough on its own?
No. A SOC 2 report shows that defined controls were tested within a defined scope. It may not cover the research or training environment where your records would live, and it says nothing about your permitted use. Pair it with a questionnaire, a subprocessor list and contract terms.
Should we send a sample before the security review is done?
No. A sample of real tickets, job notes or code is real data. Finish the security review and sign confidentiality terms first, and keep samples small and de-identified so a mistake exposes as little as possible.
Can we audit the buyer?
You can ask for audit rights. Some buyers agree to a limited form, such as a call with the security lead, a documentation review or a written certification, while full on-site audits are less common. Decide which level matches the sensitivity of the records before negotiation starts.
Does de-identification reduce what we need to check?
It reduces the harm if something goes wrong, but it does not replace security. De-identified operational records still contain business information, internal processes and sometimes indirect identifiers. Treat security and de-identification as two separate layers.
What if a smaller AI developer has no formal security report?
That does not rule it out, but it shifts the weight onto other evidence. Ask for a completed questionnaire, a call with whoever owns security, a named subprocessor list and tighter contract terms, and consider limiting the first release to a smaller, lower-sensitivity package.
Sources
- SOC 2 examinations use the AICPA Trust Services Criteria covering security, availability, processing integrity, confidentiality and privacy; the security common criteria apply in every SOC 2 report, and a Type 2 report also tests whether controls operated effectively over a specified period. Source
Related resources
See if your company qualifies
A short company assessment. No data uploads are needed.