Software companies
How to answer an insurance agency's AI vendor evaluation checklist
By SourceX Editorial · Updated
Short answer
To answer an insurance agency's AI vendor evaluation, reply question by question with a plain yes or no, the contract clause behind it and one piece of evidence. Agencies mainly want to know whether client data trains models, who owns outputs, where data goes and how it is deleted. Never answer beyond what your contracts and systems actually do.
Key takeaways
- Answer every question with yes, no or partial, then the reason; agencies tend to read hedged answers as a no.
- Whether agency client data trains any model is the question that decides most evaluations, so answer it first and precisely.
- Attach evidence: the DPA or AI addendum, the sub-processor list, an AI data flow and the retention schedule.
- Keep agency client data and your own vendor records, such as code and support history, in separate answers.
- One approved answer library keeps sales, support and security from contradicting each other.
What agencies are really asking in an AI vendor checklist#
An insurance agency's AI vendor checklist is really asking whether your product puts the agency's client relationships at risk. Agencies hold policyholder details, coverage histories and expirations they treat as their most valuable asset, and an E&O claim or a regulator's question about a vendor's AI use reaches them first.
The questions usually fall into five groups: training and model use, ownership of data and outputs, data flow and sub-processors, retention and deletion, and human review of AI outputs. Whether the form comes from an agency association template, an agency network or the agency's own counsel, the groups repeat, so build your answers once and reuse them.
Facts to pin down before anyone answers#
Facts about your product's AI features should be confirmed with engineering before sales answers a single question. Most wrong answers come from someone describing the roadmap, or a model provider's general policy, instead of the configuration the agency would actually run.
- Which features use AI, which model providers they call, and under which plan or setting.
- Whether any agency data is used to train or fine-tune models, yours or a provider's.
- Which prompts, outputs and embeddings are stored, where, and for how long.
- Which sub-processors receive agency data through AI features.
- Whether customers can switch AI features off, and at what level.
- What your contracts say about aggregated data and product improvement.
Question-by-question answer guide#
A strong answer to each checklist question pairs a direct statement with the document that proves it. The table covers the questions agencies ask most often in some form.
| Checklist question | Strong answer | Evidence to attach |
|---|---|---|
| Do you train AI models on our data? | Yes or no per feature; if no, say it is a contractual commitment, not only a practice | DPA or AI addendum clause, AI data flow diagram |
| Do your AI providers train on our data? | State the provider commitment and the setting that enforces it | Provider terms reference, configuration statement |
| Who owns AI outputs? | The agency owns outputs generated from its data, as between the parties | Output ownership clause |
| Where is our data processed and stored? | List regions and sub-processors for each AI feature | Sub-processor list with AI entries marked |
| How long do you keep prompts and outputs? | Give the retention rule and what triggers deletion | Retention schedule, deletion procedure |
| Can we turn AI features off? | Explain the control and who can use it | Admin guide excerpt |
| Will you tell us before changing data-use terms? | Commit to advance notice and a choice before any expanded use | Notice clause in the master agreement |
| How do people review AI outputs? | Describe where staff approve outputs before they reach clients or carriers | Workflow description |
How to answer the training question without overpromising#
The training question deserves the most care because it is the one agencies remember. A precise answer names what is used and what is not: for example, that agency client data trains no model, that AI providers process it under no-training terms, and that the vendor's own engineering and support records are a separate matter.
Published vendor terms show the level of detail agencies now expect. Notion, for example, states that by default it and its AI subprocessors do not use customer data to train any models, and that embeddings kept in vector databases are removed no later than 60 days after the source page or workspace is deleted. An answer that names the default, the exception and the deletion trigger reads as credible.
If you do use customer data to improve features, say so plainly and describe the control. A vague no that later proves wrong costs more than an honest yes with an opt-out.
Keep agency data and vendor records apart#
Agency client data and your own vendor records belong in separate answers. Agency data is the policyholder details, quotes, coverage notes and documents the agency enters or uploads; your records are source code, support tickets about product defects, release notes and internal product discussions.
Agencies increasingly ask whether a vendor licenses data to AI developers. A truthful answer can say that agency client data is never licensed, while vendor-owned records may be licensed after agency names and client details are removed. Drawing that line in writing now prevents a surprise later.
SourceX does not answer agency questionnaires, but vendors that consider licensing their own records meet the same questions from the other side. In the SourceX five-step transaction, the Rights step confirms that agency client data is excluded, and the SourceX Evidence Packet records provenance, permitted use and the privacy record for any vendor-owned package, which gives you a document to point to when an agency asks.
The evidence pack to keep ready#
An evidence pack is the set of documents you attach to every AI questionnaire, kept current so no answer waits on a document hunt. Each document should carry a version and a date, because agencies notice when a sub-processor list is older than the feature it is supposed to describe.
| Document | What it proves | Update when |
|---|---|---|
| AI addendum or the AI section of the DPA | Training, output ownership and notice commitments | Contract templates change |
| Sub-processor list with AI providers marked | Where agency data goes for AI features | A provider is added or replaced |
| AI feature data flow diagram | Which data each feature sends, stores and returns | A feature ships or changes |
| Retention and deletion schedule | How long prompts, outputs and embeddings persist | Retention settings change |
| Admin control documentation | How an agency switches AI features off | Controls change |
| Security assurance report | General security controls, shared under NDA | A new report period closes |
Mistakes that sink an evaluation#
Mistakes in an AI vendor evaluation are usually about consistency rather than technology. An agency's IT consultant compares the questionnaire, the contract and the sub-processor list, and any gap between them becomes the conversation.
Common errors include answering from a model provider's marketing page instead of your configuration, pointing to a website privacy policy instead of the customer DPA, leaving questions blank instead of writing not applicable and why, and promising deletion from trained models when you have no way to deliver it.
Illustrative: a comparative rater vendor fixes its answers#
Illustrative: a fictional comparative rating software vendor loses two agency evaluations in a row. Its questionnaire said the product did not use AI, but a quote summary feature called an outside model, and an agency's IT consultant found the provider on the sub-processor list.
The CEO has engineering document each AI feature with its provider, retention period and controls. Legal adds a short AI addendum committing to no training on agency data and advance notice before any change. Sales receives one answer library with evidence attached, and the next evaluation closes without follow-up questions on AI.
Frequently asked questions
Should we send our SOC 2 report with the checklist?
Send it under NDA if the agency asks for security assurance, but it rarely answers AI questions directly. A SOC 2 report covers controls, not whether you train on customer data. Pair it with a short AI statement covering training, retention, sub-processors and output ownership.
What if an agency wants contract language, not just answers?
Offer an AI addendum that turns your answers into commitments: no training on agency data, output ownership, retention limits, sub-processor notice and advance notice of changes. Keep it consistent with the questionnaire so the agency's counsel sees the same promises in both places.
How do we handle a question we cannot answer yet?
Say what you know, what you are confirming and when the agency will hear back. Guessing creates a written statement you may have to retract. A partial answer with a clear follow-up date usually lands better than a confident error.
Do agencies ask about carrier data too?
Often, because agencies send carrier quotes, appetite details and policy downloads through vendor systems. Treat carrier information as confidential to the agency and the carrier, and answer training and retention questions for it the same way you answer for policyholder data.
Who should own the answer library?
One accountable owner, usually in security or legal, with engineering sign-off on every AI answer. Sales pulls from the library rather than writing new answers. Review it whenever an AI feature, model provider or contract template changes.
Sources
- Notion's AI security page states that by default Notion and its AI Subprocessors do not use Customer Data to train any models, and that embeddings stored in vector databases are deleted within 60 days after a page or workspace is deleted. Source
Related resources
- QuestionShould companies sell or license their data?
- QuestionDo AI labs buy financial data?
- InsightCan you license spreadsheets and financial models to AI companies?
- InsightOpt-in vs opt-out for AI training in B2B SaaS contracts
- InsightCan a distributor license its pricing and quote history?
- SolutionWhat is AI evaluation data?
See if your company qualifies
A short company assessment. No data uploads are needed.