AI uses for records
Tool-use dataset parameter ambiguity: build a parameter dictionary
By SourceX Editorial · Updated
9 sections
Short answer
Tool-use training data should never leave a parameter's business meaning to its name and JSON type. Pair each tool contract with a versioned parameter dictionary that states unit, time frame, identifier namespace, value meanings, when the parameter applies and the default if omitted. Then add disambiguation tasks that record which readings were possible, how the agent resolved them, and a read-back proving the intended record changed.
Agent run
Tool use dataset parameter
Key takeaways
- A parameter's name and JSON type rarely carry its business meaning: unit, time frame, identifier namespace and conditions live elsewhere.
- Attach a versioned parameter dictionary to every tool contract, with sourced meanings and one resolution policy per ambiguity class.
- Build disambiguation tasks from frozen start states, because whether a request is ambiguous depends on the data the agent can see.
- Prove outcomes with read-backs from the system of record; a success response only shows the call was accepted.
- Score lookups, required questions, unnecessary questions and guesses separately, and report end-state checks on their own.
How should tool-use training data explain ambiguous parameter meanings?
Give every parameter a dictionary entry that says what the value means in the business, not just what shape it has, and make the dataset show how an agent resolves the cases the entry cannot settle on its own. A schema can say that priority is a string and due is a date. It cannot say whether the date is the customer's deadline or the agent's internal target, whether it is read in the site's time zone, or whether the field even applies to this kind of record.
That gap is where tool-use agents make confident, well-formed, wrong calls. The call validates, the API returns success, and the business record now holds a value nobody intended. Training and evaluation data that only pairs a user request with a syntactically correct call teaches the model to pattern-match on parameter names. Data that carries the dictionary, the candidate readings and the proof of the resulting state teaches it to check meaning first.
The asset on this page has two parts. The business parameter dictionary is a field-level record attached to each tool contract version. The disambiguation task is a test record that starts from an ambiguous request, lists the readings that were genuinely possible, names the correct resolution (look it up, ask, apply a documented default or decline) and ends with a read-back from the system of record. Both are illustrated with an invented ticketing tool; every example value is labeled illustrative.
Where does a parameter's meaning hide when the name and type look obvious?
It hides in units, reference frames, identifier namespaces, enumeration semantics and conditional rules, and public API references show each of these. None of the examples below is exotic; they are the ordinary way business APIs work, which is exactly why a model trained on parameter names alone gets them wrong.
Units are the classic case. Stripe's published OpenAPI description defines charge amounts as a positive integer in the smallest currency unit, so the same integer means cents for a two-decimal currency and whole units for a zero-decimal currency such as the yen. The field name amount says none of this. The same specification describes a tax_behavior setting that can be inclusive, exclusive or unspecified and cannot be changed once set to inclusive or exclusive, so whether an amount includes tax depends on a separate, sticky field.
Identifier namespaces are next. Atlassian's migration notice for Jira Cloud moved user references from usernames and user keys to the Atlassian account ID, so a call that names an assignee by a human-readable handle is a different request from one that passes an accountId. Jira's own create-issue example sets priority as an object holding an id, which means the word high in a user request still has to be mapped to the site's priority identifiers.
Enumerations and conditional fields cause quieter failures. Zendesk's Tickets API accepts priority values of low, normal, high or urgent, and documents due_at as a field that applies only to tickets of type task. A request to set a due date on an incident is therefore not a formatting problem; it is a business question about whether to change the ticket type, use a different field or ask.
Finally, schemas do not enforce meaning even where they look strict. Under JSON Schema 2020-12 the format keyword is an annotation by default, so a parameter declared with format date-time is not validated as a date-time unless the implementation opts into format assertion, and date-time itself follows RFC 3339, which carries an explicit offset. A date with no time and no offset is a different value.
| Ambiguity class | Documented example | Typical wrong call | What the dictionary must state |
|---|---|---|---|
| Unit or scale | Stripe amount in the smallest currency unit | Passing a decimal major-unit value or the wrong scale for a zero-decimal currency | Unit, scale and the currencies or contexts where scale changes |
| Inclusive or exclusive | Stripe tax_behavior inclusive, exclusive or unspecified, fixed once set | Treating an amount as pre-tax when the price is tax-inclusive | Which companion field changes the meaning and whether it can change later |
| Identifier namespace | Jira user references moved to accountId | Sending a display name or email where an account ID is required | Namespace, lookup tool that resolves it and how to handle multiple matches |
| Enumeration semantics | Zendesk priority low, normal, high, urgent; Jira priority set by id | Mapping top priority to high instead of the site's highest value | Business meaning of each value and the mapping from everyday words |
| Conditional applicability | Zendesk due_at applies only to tickets of type task | Setting a due date on an incident and assuming it took effect | The condition, and what to do when the condition is not met |
| Time frame and precision | JSON Schema date-time follows RFC 3339 with an offset | Sending a bare date that the server reads as midnight UTC | Time zone of record, date-only versus date-time, and the end-of-day rule |
What the business parameter dictionary records for each parameter
Record one entry per parameter per contract version, and make each entry answer the questions a careful new employee would ask before touching the field. The entry belongs to the tool contract, so when the contract changes the dictionary version changes with it, and every task and trace points to the exact version it was written against.
At a glance
What the business parameter dictionary records for each parameter
Anthropic's tool-definition guidance for Claude says detailed descriptions are the most important factor in tool performance and that a description should explain what each parameter means and how it affects the tool's behavior. The dictionary is the structured, auditable form of that advice: the same information, broken into fields a validator and a grader can read, rather than prose that drifts.
Keep the entry factual and sourced. Each meaning should cite where it came from: the vendor's API reference, the company's own configuration export, an admin screen capture or a written policy. If nobody can find the source, mark the entry unconfirmed and keep tasks that depend on it out of the evaluation set until a process owner signs it off.
| Field | Purpose | Validation rule |
|---|---|---|
| param_path and contract_ref | Locates the parameter inside one versioned tool contract | Path must exist in the referenced contract snapshot |
| business_meaning | One or two sentences on what the value represents in the business | Must not just restate the name or type |
| unit_and_format | Unit, scale, precision, string format and casing | Must agree with the schema type; scale stated for numeric money-like or quantity fields |
| time_frame | Time zone of record, date-only or date-time, end-of-day convention | Required for every date or time parameter |
| identifier_namespace | Which ID system the value belongs to and the tool that resolves it | Required for every reference to a person, account or record |
| value_meanings | Meaning of each enumerated value plus common everyday synonyms | Every schema enum value covered; synonyms map to exactly one value |
| applies_when | Conditions under which the parameter takes effect | Condition must be checkable from state the agent can read |
| default_if_omitted | What the system does when the parameter is absent | Must say server default, company policy default or no default |
| resolution_policy | Look up, ask, apply default or decline when the request is ambiguous | One policy per ambiguity class; ask requires a question template |
| source_and_version | Evidence for the meaning, reviewer, dictionary version and date | At least one source reference; reviewer named by role |
Worked dictionary entries for an illustrative ticket follow-up tool
The example below is invented to show the dictionary filled in. The tool, field names and values are illustrative and do not describe any company's records. It borrows real semantics from the public references above so that each entry has the kind of trap a production tool would have.
The illustrative tool is set_ticket_followup, which updates an existing support ticket with a priority, a follow-up due time and an assignee. Its contract snapshot is labeled C-4 and its dictionary version D-4.2. Four parameters are enough to show every ambiguity class except unit scale.
| Parameter | Business meaning (illustrative) | Format and frame | Applies when | Resolution policy |
|---|---|---|---|---|
| ticket_id | The support ticket the customer is waiting on, not the requester or the organization | Internal numeric ticket ID | Ticket is open or pending | Look up by requester; ask if more than one open ticket matches |
| priority | Urgency the support team commits to; urgent is the highest value and is reserved for outages | One of low, normal, high, urgent, lowercase | Always | Map top, highest or critical to urgent only if the outage rule is met; otherwise ask |
| due | Internal follow-up deadline, not the customer's contractual deadline | RFC 3339 date-time with offset; site time zone; end of day means 17:00 local | Ticket type is task | If the ticket is not a task, ask before changing its type; never set silently |
| assignee | Agent who owns the next customer reply | Account ID from the directory tool | Assignee is in the ticket's group | Resolve name with directory lookup; ask if two active agents match |
| notify_requester | Whether the customer is emailed about the change | Boolean; omitted means false | Always | Apply the documented default; never infer true from tone |
The disambiguation task: one request, several readings, one provable outcome
A disambiguation task starts from a request that a reasonable person could read more than one way, records which readings were possible given the starting state, and fixes the correct resolution before any agent runs. The point is to test whether the agent notices that a well-formed call would still be a guess.
At a glance
The disambiguation task: one request, several readings, one provable outcome
Build each task from a frozen starting state, because ambiguity depends on state. Sam is unambiguous in a directory with one Sam and ambiguous in a directory with two. The ticket type decides whether due applies. Freeze the snapshot, store it with the task and make the grader read from it.
Illustrative request: Bump the Lopez ticket to top priority and have Sam pick it up by Friday. Illustrative starting state: the requester Lopez has two open tickets, one incident about a failed sync and one task about an onboarding call; the directory has two active agents named Sam in different groups; no outage is declared. The correct resolution under dictionary D-4.2 is to ask one combined question covering which ticket and which Sam, map top priority to high rather than urgent because the outage rule is not met and say so, and set due only if the chosen ticket is the task, converting Friday to 17:00 in the site time zone.
The task record carries these fields:
- task_id, contract_ref (C-4) and dictionary_version (D-4.2), so the task is graded against the meanings in force when it was written.
- request_text and start_state_ref, the exact words and the frozen snapshot the agent sees.
- candidate_readings, one row per parameter that has more than one reasonable value, with the evidence that makes each reading possible.
- expected_resolution per reading: lookup, ask, documented default or decline, with the dictionary field that requires it.
- allowed_questions, the clarifying questions that count as correct, and the scripted user answers the simulator returns.
- expected_calls, the final tool call or calls after clarification, with arguments in the contract's own namespaces.
- verification_reads, the read-back calls and the field values they must return.
- untouched_fields, fields that must not change, such as the other ticket and notify_requester.
- failing_variants, at least two wrong but well-formed calls, each with the reason it fails, used to test the grader.
Did the ticket really change, or did the call just succeed?
Only a read of the record from the system of record, compared with the expected end state, proves the change. A success status, an echoed response or the agent's own summary shows that a call was accepted, not that the business now holds the intended value.
Conditional parameters make this concrete. If an agent sends due for a ticket where the field does not apply, a lenient server may accept the payload and ignore the field, and the agent may report success. The read-back shows the due time is still empty, which is the failure the task exists to catch. The same applies to a priority mapped to the wrong enumeration value or an assignee resolved to the wrong person: the call succeeded and the business state is wrong.
The Model Context Protocol specification makes part of this the server's job: servers must validate all tool inputs, and clients should pass tool execution errors back to the model so it can correct itself. A training record should therefore keep the error text the agent saw as well as the final state. A recovery that follows an informative error is a different behavior from a first-time correct call, and both are worth labeling.
Write verification reads as explicit checks rather than a diff of everything. For the illustrative task: the chosen ticket's priority equals high, its due equals the converted Friday time only if it is a task, its assignee equals the account ID of the Sam the user confirmed, the other ticket is unchanged and no requester notification was sent. Each check names the field, the expected value and the read call that returns it.
Valid and failing records: what the validator rejects and why
The validator runs on every task and trace before it enters a dataset, and it should reject a record for missing meaning just as firmly as for broken JSON. Pair each rule with a failing record so reviewers can see the boundary. The failing examples below use the illustrative tool and are not drawn from any real system.
| Check | Rule | Failing example (illustrative) | Why it fails |
|---|---|---|---|
| Dictionary coverage | Every parameter in the call has an entry in the referenced dictionary version | Call includes escalation_level, absent from D-4.2 | Meaning is undocumented, so the grader cannot judge it |
| Namespace match | Identifier values match the declared namespace format | assignee set to the display name Sam R. | Display names are not account IDs and may match more than one person |
| Enumeration and synonym | Value is a declared enum value and any synonym maps to exactly one | priority set to critical | Not a declared value; the dictionary maps critical to urgent only under the outage rule |
| Applicability | Conditional parameters are only sent when applies_when holds in the start state | due sent for an incident ticket | The field does not apply to that ticket type |
| Time frame | Date-time values carry an offset and follow the end-of-day rule | due set to a bare date with no time or offset | A server may read it as midnight UTC, which is the wrong day locally |
| Ambiguity acknowledged | Every candidate reading has a resolution and the trace shows it | Agent picked one of two Sams without a lookup or question | Correct by chance is not evidence of correct behavior |
| State proof | Verification reads exist and match expected values | Trace ends at the update call's success response | Acceptance is not proof that the intended record changed |
Scoring an agent that asks too much, too little or guesses
Score asking as a decision with costs on both sides: an unnecessary question is a usability failure and a missing question is a correctness failure. A grader that rewards any clarifying question will train agents that interrogate users about things the dictionary already settles, such as whether notify_requester defaults to false.
Public benchmarks already separate some of these behaviors. The Berkeley Function Calling Leaderboard lists test categories for irrelevant function documentation and for multi-turn calls with a missing parameter or a missing function. Those categories test whether a model notices that it lacks what it needs. A business parameter dictionary extends the idea from missing values to present but ambiguous ones.
When the right move is to ask the human, the Model Context Protocol offers elicitation, through which a server can request structured input from the user via the client, optionally with a JSON schema to validate the answer. The specification forbids using form-mode elicitation for passwords, API keys, access tokens or payment credentials, so a disambiguation task should never script a question that asks for a secret.
A practical scoring scheme has four outcomes per candidate reading: resolved correctly by lookup or default, asked when asking was required, asked when the dictionary already settled it, and guessed. Report them separately rather than as one blended score, and report the end-state checks on their own line, because an agent can ask perfectly and still write the wrong value.
- Resolved correctly: the agent used the documented lookup or default and the read-back matches.
- Asked when required: the question covered the open readings, ideally in one turn, and the final state matches the answer.
- Over-asked: the agent asked about something the dictionary or the start state already settled.
- Guessed: the agent picked a reading without evidence, whether or not the guess happened to be right.
Sourcing real parameter evidence from business systems
The most useful evidence for parameter meaning sits in business systems, not in API references: admin configuration exports, field definitions, workflow rules, macros, and the change histories that show which values people actually set and corrected. A support team's history of priority changes, reassignments and due-date edits shows where everyday language and system values diverge, which is exactly where disambiguation tasks should concentrate.
Treat that material carefully. Ticket histories contain customer names, email addresses and free text, so tasks built from them need de-identification and a rights review before they leave the company, and access to a system through an API or an MCP server is not the same as permission to license its records for training.
Configuration exports and field change histories of this kind can be licensed through SourceX, which takes a business's selected records through rights review, contracting, preparation and delivery. A tool-use team that wants real parameter evidence rather than invented examples lists the systems, volume, language, format, timing and licensing terms in a data request, and SourceX then looks for U.S. businesses that hold such records instead of drawing on stock.
Frequently asked questions
Is a detailed tool description enough to remove parameter ambiguity?
It helps, and vendor guidance treats descriptions as a major factor in tool performance. But a description cannot settle state-dependent ambiguity, such as which of two people named Sam the user meant. You still need tasks that record the candidate readings and the resolution, plus a read-back that proves the result.
Should ambiguous requests be removed from tool-use training data?
No. Ambiguous requests are where agents fail in production, so they belong in both training and evaluation. Keep them, but label the candidate readings, the correct resolution and the proof of end state, so the model learns to look up or ask rather than guess.
Does JSON Schema validation catch a wrong date format?
Not by default. In JSON Schema 2020-12 the format keyword is an annotation unless the implementation enables format assertion. Add an explicit pattern or a validator rule, and state the time zone and end-of-day convention in the dictionary.
How many disambiguation tasks does a tool need?
Cover each ambiguity class that applies to the tool's parameters with several start states, including one where the request is not actually ambiguous. That control case tells you whether an agent asks because it must or because it always does. Grow the set from the classes where real change histories show the most corrections.
What should happen when the user's answer is still ambiguous?
Script that branch. The task should accept a second, narrower question or a refusal to act, and it should fail the agent if it proceeds on the unclear answer. Record the scripted answers with the task so every run sees the same conversation.
Sources
- Stripe's published OpenAPI description defines charge amounts as a positive integer in the smallest currency unit and describes tax_behavior as inclusive, exclusive or unspecified, fixed once set. Source
- Atlassian's Jira Cloud migration notice replaces username and userKey with the Atlassian account ID for identifying users in REST APIs. Source
- Jira Cloud's issues API reference shows priority set by an object holding an id and the due date as a plain date string in its create-issue example. Source
- Zendesk's Tickets API accepts priority values low, normal, high or urgent and applies due_at only to tickets of type task. Source
- JSON Schema 2020-12 treats format as an annotation in its default vocabulary and defines date-time by RFC 3339. Source
- Anthropic's tool-definition guidance says detailed descriptions are the most important factor in tool performance and should say what each parameter means and how it affects behavior. Source
- The Model Context Protocol tools specification defines tool name, description and inputSchema, requires servers to validate tool inputs, and says clients should return tool execution errors to models for self-correction. Source
- The Model Context Protocol elicitation feature lets servers request structured input from users through the client, and forbids form-mode requests for passwords, API keys, access tokens or payment credentials. Source
- The Berkeley Function Calling Leaderboard lists test categories for irrelevant function documentation and for multi-turn calls with a missing parameter or missing function. Source
Tell us what data you need
Share scope, volume, format, timing and licensing needs. Datasets are sourced on request.