AI uses for records
Creative brief constraint dataset: grading every constraint a brief sets
By SourceX Editorial · Reviewed by Noah Loul ·
9 sections
Short answer
Take one real brief version, list every constraint it sets, and turn each into a separately checkable requirement: scripts for length, required and banned words and format, rubrics for tone and claims. Link the rounds, the approved deliverable and its approval, and clear every asset's rights. A creative brief constraint dataset then grades outputs constraint by constraint against what the client actually approved.
Grader rubric
Creative brief constraint
Key takeaways
- Build each task from one brief version and list every constraint as its own checkable item.
- Use scripts for length, words and format, and anchored rubrics for tone, claims and audience fit.
- Record waivers and re-issue the brief so an approved output never fails its own task.
- Mark legal and claims constraints as must-pass and fail the task outright on any miss.
- Clear every asset's ownership, license and AI training status before it enters the pack.
What turns a creative brief into a multi-constraint training task?
Build each task from one real brief version: list every constraint the brief sets, turn each into a requirement that can be checked on its own, attach the approved deliverable and the revision trail, and record who owns each asset and what use is permitted. The task is then to produce a deliverable that satisfies all constraints at once, graded constraint by constraint.
Creative work is a natural source because briefs already bundle several simultaneous demands. A single social post or email may carry a word limit, a required product name, banned claims, a tone set by the brand guide, a legal line, a territory and a deadline. The approved version shows one way all of them were met together, and the rejected rounds show what a reviewer considered a miss.
The value lies in the links. A deliverable without its brief version is just copy, and a brief without the approval record does not say which reading of an ambiguous constraint the client accepted. A creative brief constraint dataset keeps those links intact, which is what lets a model learn to satisfy entangled requirements instead of the easiest one.
Which constraints can a script check, and which need a human rubric?
Format, length, required and forbidden words, and structure can be checked by a script; tone, brand fit, claim accuracy and audience fit need a rubric scored by a trained reviewer. Split the brief along that line before writing a single grading rule.
Public benchmarks show how far scripts can go. Google Research's IFEval code registers 25 verifiable instruction types, from word and sentence counts to forbidden words, keyword frequency, JSON format, title markers and all-lowercase output. Its published input file holds 541 prompts, each carrying one to three of those instructions, and the evaluation code scores every instruction separately in a strict mode and in a loose mode that also accepts the response with markdown asterisks or its first or last line removed.
Rubric checks scale differently. The InFoBench authors decompose each instruction into simpler yes or no criteria and score the share satisfied, a metric they call the Decomposed Requirements Following Ratio, across 500 instructions and 2,250 decomposed questions. Real briefs often carry five or more constraints, and several of them are judgment calls, so a creative task pack usually needs both kinds of check.
| Constraint in the brief | Example wording | Check method | Who or what checks |
|---|---|---|---|
| Length | No more than 120 words in the body | Word count | Script |
| Required element | Name the product exactly as trademarked, once in the first line | String match with position | Script |
| Banned language | No superlatives such as best or fastest; no health claims | Word list plus claim review | Script, then reviewer |
| Tone | Warm, plain, no exclamation marks, matches brand guide v4 | Rubric with anchored examples | Trained reviewer |
| Legal line | Include the approved offer terms line verbatim | Exact match against the approved text | Script, confirmed by legal approver |
| Audience and territory | U.S. small business owners; American English | Rubric plus spelling check | Reviewer and script |
What should the rights cleared constraint and rubric task pack include?
It should carry the brief version, every constraint as a separate checkable item, the deliverables from each round with reviewer notes, the approval, and an asset rights register. The fields below are written so a creative operations team can use them as a template; map them to the project management, proofing and digital asset management tools the agency already runs.
At a glance
What should the rights cleared constraint and rubric task pack include
| Field | Contents | Typical source | Check before use |
|---|---|---|---|
| task_id and brief_version | Stable task key, brief version and date, real or illustrative label | Project management system | Deliverables point to the brief version they answered |
| constraints | Numbered list, each with type, exact wording, check method and pass rule | Brief, brand guide, legal notes | Every constraint has one check method |
| context assets | Product facts, brand guide extract, approved legal lines | Brief attachments and brand portal | Each asset appears in the rights register |
| rounds | Draft per round, reviewer comments, constraint-level misses noted | Proofing or review tool | Comments map to constraint numbers |
| waivers | Any constraint the client relaxed, who agreed and when | Email or approval thread | Waiver dated before the approved round |
| approval | Approved version, approver role, date, scope of approval | Proofing sign-off | Approver had authority for the territory |
| rights register | Owner, license or assignment, territory, term, AI training permission per asset | Contracts, licenses, embedded metadata | No asset with unknown or prohibited status |
| grading | Script checks, rubric items with anchors, scorer notes | Built for the dataset | Rubric tested on approved and rejected rounds |
| outcome | What ran, where, and whether any post-launch correction followed | Traffic or publishing log | Outcome matches the approved version |
Whose rights, which license and which approved round decide what enters the pack?
Three kinds of fact change what may go into the pack: who owns each piece of work, what the license or release allows, and which version the client actually approved. Each belongs in the rights register and the approval fields, not in a reviewer's memory.
Ownership is not automatic for commissioned work. The U.S. Copyright Office's Circular 30 explains that a specially ordered or commissioned work is a work made for hire only if it falls within one of nine statutory categories and the parties expressly agree in a written instrument signed by them; otherwise it is not. Copy, illustrations or photos from a freelancer may therefore belong to the freelancer unless rights were assigned, and the client may own deliverables the agency produced, depending on the agreement.
Embedded metadata can also restrict use. The IPTC Photo Metadata Standard added a Data Mining property in version 2023.1, with values adopted from the PLUS Coalition that include prohibited for AI or machine learning training and prohibited for generative AI or machine learning training, plus a separate field for other constraints. An image carrying one of those values should not enter a training pack, whatever the rest of the paperwork says, until the rights holder agrees otherwise.
| Condition found | Effect on the pack | Record to keep | Who decides |
|---|---|---|---|
| Client owns deliverables under the agreement | Client permission needed to include them | Agreement clause and client sign-off | Agency counsel and client |
| Freelancer kept copyright | Exclude, or obtain an assignment or license | Freelance contract and any assignment | Agency counsel |
| Stock asset licensed for the campaign only | Exclude the asset; keep a placeholder note | Stock license terms | Rights reviewer |
| Image marked prohibited for AI training | Exclude until the rights holder agrees | Embedded metadata snapshot | Rights reviewer |
| Person identifiable in an image or voice | Exclude unless a release covers this use | Model or talent release | Rights reviewer and counsel |
| Later round superseded the approved version | Re-label the task to the new approved round | Approval history | Creative operations lead |
Illustrative case: a launch email with five constraints and three rounds
This case is illustrative: the brief, wording and counts are invented to show the pack in use. A software client briefs a launch email to existing small business customers. Brief version 2 sets five constraints: a body of no more than 120 words, the product name exactly as trademarked once in the first line, no superlatives, the approved terms line verbatim at the end, and a warm, plain tone under brand guide version 4. Assets are a product fact sheet the client owns and a header image licensed from a stock library for this campaign only.
Round 1 runs to 141 words and calls the feature the fastest on the market; the reviewer marks constraints 1 and 3 as misses. Round 2 meets the count at 118 words but paraphrases the terms line; constraint 4 misses. Round 3 passes all five, and the client approves it in the proofing tool with a dated sign-off by the marketing director.
Decision for the pack: include the brief, all three rounds with constraint-level notes, and the approved round 3 as the reference output. Exclude the stock header image because its license covers the campaign only, and record the exclusion. Verified outcome: the publishing log shows round 3 sent unchanged, and no correction followed. The approved text is the target; the rejected rounds become graded negative examples.
Counterexample: in a sister task the approved email runs to 134 words, because the client agreed by email to relax the length limit after round 2. If the pack keeps the original 120-word constraint, the approved output fails its own task and the grader punishes the right answer. The fix is to record the waiver with its date and approver and issue brief version 3, so the task matches what was actually approved.
What evidence shows the approved round really met the brief?
The approved round, matched constraint by constraint to the brief version in force and backed by the approval and rights records, is the proof. A project status of complete or a delivered file in a shared drive proves neither that every constraint was met nor that the work may be reused.
| What is graded | Pass condition | Evidence that settles it | False pass to watch for |
|---|---|---|---|
| Script-checkable constraints | Every one passes under the stated rule | Automated check log | Passing loose formatting while failing strict |
| Rubric constraints | Each rubric item meets its anchor | Scorer notes against anchors | One holistic score hiding a tone miss |
| Brief version | Output is graded against the version the approval covers | Brief history and waivers | Grading an approved output against a superseded brief |
| Approval | Approver had authority and approved this exact version | Proofing sign-off | Status of complete treated as approval |
| Rights | Every included asset cleared for this use | Rights register entries | Campaign license read as a training license |
When constraints collide: conflicts, waivers and weighting
Decide in advance how conflicts and waivers are handled, and weight constraints by what the client actually enforced. Without those rules, a pack of entangled constraints rewards outputs that game the easy checks.
Some constraints cannot be satisfied together. IFEval's code keeps an explicit table of instruction conflicts so incompatible instructions are not combined in one prompt; a creative pack needs the same, written at the brief level, so a task never demands both a 50-word limit and five required sections of detail.
Weighting is the harder call. A decomposed ratio treats every criterion equally, and the TOWER authors argue that some aspects of complex instructions matter more than others and propose weighting them. In creative work the legal line and banned claims usually outrank a stylistic preference, so mark each constraint as must-pass or scored, and fail the task outright on any must-pass miss.
- Must-pass: legal and claims constraints, required trademarks, territory and language.
- Scored: tone, rhythm, call-to-action strength and other judgment items.
- Waived: constraints the client relaxed in writing, recorded with date and approver.
- Excluded: constraints that conflict with another in the same brief and were never resolved.
What an agency should hold back from the pack
Leave out anything the agency cannot show it may reuse, and anything that would expose a client's unreleased plans. Briefs often contain launch dates, pricing strategy and product details that are confidential under the client agreement, and rejected rounds may contain claims the client's legal team struck for a reason.
Remove or replace client names, product names and people's names with stable placeholders when the task does not depend on them, but keep constraint wording intact, because the exact phrasing is the training signal. Images and voice recordings of identifiable people need a release that covers this use. Whether the agency, the client or a freelancer holds the right to license the work is a contract question for counsel, and removing names does not create that right.
Bringing approved creative work to a data license
Agencies and in-house studios hold years of briefs, rounds and approvals that show how real constraints were met, which is scarce material for instruction-following work. SourceX helps such businesses license selected briefs, rounds and approvals as AI datasets, taking on sourcing, rights review, contracting, preparation, delivery and payment.
A creative business can start with a short company assessment that requires no data upload. The company approves the scope and terms and signs the agreement before any work is delivered.
Frequently asked questions
What is a creative brief constraint dataset?
It is a set of tasks built from real or illustrative creative briefs, each listing every constraint as a separately checkable item and linking the rounds, the approved deliverable, the approval and an asset rights register. It is used to train or test models on meeting several constraints at once.
How many constraints should one task carry?
As many as the real brief set, usually more than public benchmarks use. IFEval's published prompts carry one to three verifiable instructions each, while a typical marketing brief adds tone, legal and audience constraints on top of format and length.
Can rejected drafts be used?
Yes, when they are linked to the reviewer's constraint-level notes. They become graded negative examples, provided the agency has the right to reuse them and they contain nothing the client's legal team struck for confidentiality reasons.
Who owns copy written by a freelancer for an agency?
It depends on the contract. The Copyright Office explains that a commissioned work is a work made for hire only in nine statutory categories with a signed written agreement, so otherwise the freelancer may keep copyright unless it was assigned. Counsel should review the specific agreements.
What if an image's metadata says no AI training?
Exclude it. The IPTC Data Mining property can mark an image as prohibited for AI or generative AI training, and that signal should be honored unless the rights holder agrees otherwise in writing.
Sources
- IFEval is Google Research's open code and data for the paper Instruction-Following Evaluation for Large Language Models. Source
- IFEval registers 25 verifiable instruction types and keeps a table of conflicting instructions. Source
- IFEval scores each instruction separately in a strict mode and a loose mode that tolerates markdown asterisks and a removed first or last line. Source
- IFEval's published input file holds 541 prompts, each with one to three verifiable instructions. Source
- InFoBench decomposes 500 instructions into 2,250 yes or no questions and scores them with the Decomposed Requirements Following Ratio. Source
- The TOWER paper proposes weighting the aspects of complex instructions instead of treating them equally. Source
- Copyright Office Circular 30 explains when a commissioned work is a work made for hire: nine categories and a signed written agreement. Source
- IPTC Photo Metadata Standard 2023.1 added a Data Mining property with values that prohibit AI or generative AI training. Source
See if your company qualifies
A short company assessment. No data uploads are needed.