Systems and records
Collections promise outcome dataset: did the customer pay as promised?
By SourceX Editorial · Updated
10 sections
Short answer
To connect a promise to pay with its outcome, record the promise as structured fields (invoices, amount, date, method), freeze the open balance, then count only cash applied to those invoices inside the promised date plus grace days, minus later reversals and excluding credit memos. Label each promise kept, partially kept, late, broken, superseded or not evaluable, and attach the ledger entries a reviewer needs to reconcile it.
Grader rubric
Collections promise outcome
Key takeaways
- A promise can only be linked when it names invoices, an amount, a date and the source text it came from.
- Only cash applied to the promised invoices inside the window, and never reversed, counts toward a promise.
- Credit memos, write-offs and deductions lower balances without payment and must not create kept labels.
- Judge a promise only after grace days, settlement time and the return window have passed.
- Superseded and not evaluable labels belong in the dataset instead of being dropped.
- Every label should carry ledger and bank identifiers so a second reviewer can rebuild it.
How do you connect a promise-to-pay conversation with the actual payment outcome?
Turn each promise into a structured record, then test it against payments that were actually applied to the promised invoices inside a defined window, and only after the window for returned payments has closed. The result is one outcome label per promise, such as kept, partially kept, broken, superseded or not evaluable, with the payment and ledger entries that justify it attached.
Most collections teams already capture half of this. The promise lives in a collector's note, a call log or an email, and the payment lives in the cash application and bank records. A collections promise outcome dataset is valuable only when those two halves are joined by rules that a second person could rerun and get the same answer.
The protocol below is written for business-to-business receivables, where the customer is a company and the payer is its accounts payable department. It is a method for building and checking labels; it does not report any rates or results.
What must a promise record hold before it can be linked?
It needs to say who promised what, for which invoices, by when, and how the promise was recorded. A note that says customer will pay next week cannot be linked; a record with an invoice list, an amount, a date and a promisor can.
At a glance
What must a promise record hold before it can be linked
Enterprise systems that model promises formally show what the minimum looks like. SAP's promise to pay object tracks whether the customer kept the promise by paying the outstanding amount by the agreed date, uses states such as Open, Partially Kept and Broken, and keeps a level that counts how many promises on an invoice were not kept. SAP's valuation step expresses fulfillment as a percentage between 0 and 100 and derives the status from it. Oracle's Advanced Collections uses statuses including Broken, Closed and Fulfilled. If your system has such an object, export it; if promises live only in notes, extract them into the fields below before linking anything.
- Promise ID, created timestamp and the user or channel that recorded it.
- Customer account and the invoice IDs the promise covers, or an explicit flag that it covers the oldest open items.
- Promised amount, or the share of each invoice promised, in the invoice currency.
- Promised date, and whether it is a single payment or an installment schedule.
- Promised payment method, when stated, because a check and a wire settle on different clocks.
- Source text span: the note, email or call transcript segment the promise was taken from.
- Superseding links: the ID of any later promise or dispute that replaced this one.
In what order should each promise be tested against the ledger?
Run the steps in order for every promise, and keep the intermediate outputs. A promise that fails a step is not dropped; it receives a not evaluable label with the step that stopped it, which matters as much for training as the clean cases.
| Step | Rule | Data needed | Flag when |
|---|---|---|---|
| 1. Normalize the promise | Fill invoice list, amount, date and method from the source text | Promise record, collector note or transcript | Amount or date cannot be read from the source |
| 2. Freeze the open balance | Record each promised invoice's open amount on the promise date | AR subledger as of the promise date | Invoice already closed or credited before the promise |
| 3. Set the evaluation window | Promised date plus the stated grace days, plus method lag | Grace day setting, payment method | No grace rule documented for the period |
| 4. Collect applied cash | Sum cash applications to the promised invoices dated inside the window | Cash application and settlement records | Cash left unapplied on the account inside the window |
| 5. Remove reversals | Subtract payments later reversed as returned or NSF | Payment reversals, bank returns | Reversal reason missing |
| 6. Exclude non-cash credits | Credit memos, write-offs and deductions do not count as paid | Credit memo and adjustment records | Credit issued inside the window without a reason code |
| 7. Compute fulfillment | Applied net cash divided by the promised amount, capped at 100 percent | Steps 2 to 6 | Promise amount is zero or missing |
| 8. Assign the label | Apply the label rules in the next section | Steps 1 to 7, superseding links | Two rules apply at once |
| 9. Attach evidence | Store payment, settlement and bank line IDs with the label | Ledger and bank identifiers | Any label without a linked ledger entry |
Which cash movements count toward a promise, and which only look like they do?
Only cash that was received, applied to the promised invoices and not later reversed counts. Everything else that lowers an open balance needs its own treatment, because a model trained to predict promise keeping will learn the wrong lesson if a credit memo is counted as a kept promise.
Returned payments are the most common trap. Microsoft's Dynamics 365 Finance collections page, for example, lets a user process a not sufficient funds payment in one step, and selecting that action cancels the payment. A promise that looked kept on the payment date becomes broken once the cancellation posts, so step 5 has to run after the reversal window, not on the day cash arrives.
Matching ambiguity is the second trap. Bank statement lines reach finance systems through messages such as the ISO 20022 camt.054 debit and credit notification, which reports single or multiple entries to an account. When one line could match several open items, Dynamics 365 lets finance teams require manual matching rather than accept the first match. Labels built from automatically matched cash should record which rule made the match.
- Counts: cash applied to a promised invoice inside the window and not reversed.
- Counts with a flag: cash applied to a promised invoice after the window; it supports a late label, not kept.
- Does not count: unapplied cash on the account, until it is applied to a promised invoice.
- Does not count: credit memos, write-offs, rebates or deductions, which reduce the balance without payment.
- Does not count: payment applied to a different invoice than the one promised, unless the promise allowed oldest-first application.
- Reverses a count: returned, NSF or recalled payments, whenever they post.
How should each promise outcome be labeled?
Use a small, closed label set with one rule each and a required evidence type. Labels that mix outcome and reason, such as broken because of dispute, should be split into an outcome label and a separate reason field.
| Label | Rule | Evidence required | Common mislabel |
|---|---|---|---|
| Kept | Fulfillment reaches 100 percent inside the window | Applied payment and settlement IDs | Credit memo counted as payment |
| Partially kept | Fulfillment above zero and below 100 percent inside the window | Applied payment IDs and remaining balance | Short pay labeled kept because the invoice later closed |
| Late | Full payment arrives after the window closes | Payment date and window end date | Labeled kept because the invoice is now paid |
| Broken | No qualifying cash inside the window, or all of it reversed | Window dates and an empty or reversed cash set | Returned payment left as kept |
| Superseded | A new promise or dispute replaced it before the promised date | ID of the replacing promise or dispute | Labeled broken when the collector agreed a new date |
| Not evaluable | A protocol step could not run | The step number that stopped it | Silently dropped from the dataset |
When is it safe to judge a promise?
Only after the grace period, the payment method's settlement time and the return window have all passed. Judging on the promised date mislabels every promise paid by check and every payment that is later returned.
The packaged systems build this delay in. Oracle lets administrators set the maximum grace days before a promise is broken, and its guide says to schedule the promise reconciliation program only after cash has been applied to open transactions. Whatever system you use, write the grace rule that applied in each period into the dataset, because a change from 3 to 5 grace days changes labels without any change in customer behavior.
For the evaluation date, a practical rule is the promised date plus grace days plus the longest settlement and return time for the payment methods in scope. Record that date on every promise so a reviewer can see when the label became final.
How do the rules label three illustrative promises from one contractor?
The company and figures are invented for illustration. A distributor's collector records three promises from the same contractor customer in one month, each in a call note that names invoices and a date.
Promise A covers two invoices in full by Friday. Cash for both arrives Thursday by ACH and is applied the same day. Two banking days later one payment is returned for insufficient funds and the cancellation posts. After step 5, fulfillment is about 55 percent, and the label is partially kept rather than kept.
Promise B covers one invoice by the 15th. On the 12th the customer opens a dispute over a damaged delivery, and the collector logs a new promise for the undisputed lines. Promise B is labeled superseded with the dispute ID attached, and the new promise is evaluated on its own.
Promise C covers one invoice by the 20th. A credit memo for a pricing error is issued on the 18th and a payment for the remaining balance arrives on the 26th, after the window. The credit memo does not count as payment, so the label is late, with the credit memo stored as context.
The exception: a fourth note says the customer will catch up on everything soon. No invoices, amount or date can be read from it, so it becomes not evaluable at step 1. Keeping that row tells a model that vague commitments exist and are not the same as promises.
Can a reviewer reconcile every label to documents and the final ledger?
Yes, if every label carries ledger identifiers and a reviewer can rebuild it from the subledger without the original analyst. Test that before the dataset leaves finance.
Draw a random sample from each label, with extra weight on partially kept, superseded and not evaluable, which are the labels most likely to be wrong. For each sampled promise, a second reviewer reads the source note, reruns steps 2 to 8 from the AR subledger and bank records, and records agreement or the step where the answers differ. Fix the rule, not just the row, when the same disagreement repeats.
- Every promised invoice exists in the subledger with the same customer account.
- The open balance frozen in step 2 matches an aging snapshot for that date.
- Every counted payment has a settlement or application record and a bank line.
- No counted payment was later reversed.
- Superseded promises link to a record that exists and was created before the promised date.
- The grace rule used matches the configuration that was in force for that period.
When should you narrow the history instead of labeling all of it?
The method assumes promises are about specific invoices and that cash application is reasonably complete. When either assumption fails for a large share of the history, narrow the scope to the periods, business units or customer segments where it holds, rather than labeling everything with low confidence.
- Large unapplied cash balances for long periods: labels will skew toward broken. Limit the dataset to periods where cash application was current.
- Promises made at account level with oldest-first application: label against the account balance and say so in the dataset notes.
- Consumer debts or individual sole proprietors: these raise separate debt collection and privacy questions; get counsel's review before including them.
- Notes containing personal details of the customer's staff, bank account numbers or guarantor information: remove or mask them before any review outside finance.
- System migrations that changed promise statuses or grace settings mid-history: split the dataset at the migration date.
What makes promise outcome history worth licensing to AI developers?
AI developers value collections history most when each promise already carries an outcome a reviewer can trace to the ledger, and when it covers several years of consistent processes. As a rough yardstick, SourceX says buyers tend to want two or more years of consistent history and tens of thousands of records in a single system, though rare kinds of work can qualify with less and each deal is judged on its own.
For receivables teams that go ahead, SourceX carries the work from qualification and rights review through the contract, data preparation, delivery and payment. Checking whether an AR history fits starts with a short company assessment, and no ledger extract has to be uploaded for it.
Frequently asked questions
What is a broken promise to pay?
A broken promise is one where no qualifying payment was applied to the promised invoices by the promised date plus any grace period, or where every payment received was later reversed. Systems such as Oracle Advanced Collections mark a promise broken when the promised date and grace days pass without payment.
Should a partial payment count as a kept promise?
No. Label it partially kept and store the fulfillment percentage. Treating partial payments as kept hides the cases a collections model most needs to learn, and SAP's own valuation step measures fulfillment as a percentage rather than a yes or no.
How many grace days should we allow before judging a promise?
Use the grace setting your team actually applied in each period and record it with the label. If none was configured, set one rule for the dataset that covers the slowest payment method in scope, and document it so reviewers can apply it the same way.
Can promises recorded only in free-text notes be used?
Yes, if the invoice, amount and date can be extracted into structured fields and the original text span is kept for review. Notes that name no invoice, amount or date should be labeled not evaluable rather than guessed.
Do credit memos and write-offs count toward a promise?
No. They lower the open balance without any payment from the customer. Keep them as context on the promise, because a promise followed by a credit memo often signals a dispute the collector did not log.
Sources
- SAP's promise to pay records whether the customer kept the promise by paying the outstanding amount by the agreed date, with states such as Open, Partially Kept and Broken and a level counting promises not kept on an invoice. Source
- SAP's valuation of promises to pay determines the level of fulfillment as a percentage between 0.00 and 100.00 and derives a status from it. Source
- Oracle Advanced Collections promise statuses include Broken, Closed and Fulfilled, and the reconciliation program should be scheduled only after cash has been applied to open transactions. Source
- Oracle Advanced Collections preferences include the maximum grace days before a promise is broken. Source
- Dynamics 365 Finance collections lets users process a not sufficient funds payment in one step, and selecting NSF payment cancels the payment. Source
- Dynamics 365 Finance can require manual matching when advanced bank reconciliation matching rules find multiple documents that match. Source
- The ISO 20022 camt.054 bank-to-customer debit/credit notification informs the account owner of single or multiple debit and credit entries reported to the account. Source
See if your company qualifies
A short company assessment. No data uploads are needed.