Systems and records
IT configuration drift dataset: changes the ticket never recorded
By SourceX Editorial · Updated
10 sections
Short answer
Reconstruct an unrecorded IT configuration change from the configuration evidence, not the ticket. Compare the approved baseline with the platform's configuration history, list changes with no approved change record, then attribute each from audit logs and the incident timeline. A reconciliation task records the baseline, observed change, actor evidence, every candidate ticket, a match decision and an independent outcome, with secrets replaced by typed placeholders.
Record timeline
IT configuration drift
Key takeaways
- Start from observed configuration history and work back to tickets; a closed ticket may not describe what was actually changed.
- Incident hotfixes and imperative commands that were never written back to the declared configuration are typical drift sources.
- Each detector answers a narrow question: configuration history shows state, audit trails show the actor, plan tools show the difference.
- Match a change to a record only when time, item, actor and content agree, and record partial matches as such.
- Expected automated changes belong in the dataset as negative examples, labeled as expected rather than as failures.
- Redaction happens while building the task: typed placeholders replace secrets, addresses and live resource names.
How do you reconstruct a configuration change the ticket never recorded?
Work from the configuration evidence back to the ticket, not the other way round. Take the approved baseline, compare it with the configuration history the platform recorded, list every change that has no approved change record, and then attribute each one using audit logs and the incident timeline. The ticket is one candidate explanation, not the starting point.
Three record streams have to meet. The declared state is what the team said the system should look like: a baseline document, infrastructure-as-code files or a configuration management playbook. The observed state is what the platform says it actually looked like over time. The decision records are change requests, incident tickets and approvals. A configuration drift case exists wherever the observed state moved and no decision record explains the move.
For a dataset, each reconstructed change becomes one reconciliation task: the configuration item, the baseline it departed from, the observed change with its time, the evidence of who or what made it, the candidate tickets that might explain it, the match decision and the outcome. A closed ticket with a cheerful resolution note is often the easiest record to find and the least informative one.
Why the configuration moves while the ticket stays still
Drift is what happens when changes bypass the process that keeps the baseline current. NIST SP 800-128 describes a baseline configuration as a reviewed and agreed set of specifications that changes only through change control, and treats monitoring as the way to find undocumented and unauthorized changes against it. NIST SP 800-171 Revision 3 frames change control as tracking, reviewing, approving or disapproving, and logging changes.
Many unrecorded changes are not malicious. They are fixes made under pressure and never written back. The Terraform documentation describes refresh-only planning for exactly this situation: someone changed remote objects outside the usual workflow, for example while responding to an incident, and the records now need reconciling. Kubernetes documentation shows another route: scaling a deployment directly changes the live object, and the last-applied configuration annotation does not record the new replica count.
| Drift source | What the evidence looks like | Where it shows up | Usual starting classification |
|---|---|---|---|
| Incident hotfix | Change minutes after an alert, by an on-call engineer, no change ticket | Platform audit log, incident timeline | Emergency change needing retroactive approval |
| Console or CLI edit | Change by a named person outside the deployment pipeline | Cloud audit trail, configuration history | Unrecorded manual change |
| Imperative command over declarative config | Live value differs from the config file and last-applied record | kubectl diff, live object versus files | Unrecorded manual change |
| Automation acting as designed | Repeated changes by a service identity within set bounds | Autoscaler or controller events | Expected change, no ticket required |
| Vendor-managed update | Change by a provider identity or a maintenance window | Provider notices, service events | External change to record, not approve |
| Stale baseline | Declared state never updated after an approved change | Change ticket exists, files unchanged | Documentation gap, not drift in the system |
Which tools show the observed change, and what each cannot show
Every drift detector answers a narrower question than it appears to, so the task should record which tool produced the evidence and what it could not see. AWS Config, for example, records configuration items with a capture time, a state identifier that orders the items for a resource, and a hash that lets you compare two states of the same resource. Its API reference also says the relatedEvents field, which once linked a configuration item to CloudTrail events, is empty as of version 1.3; the documented route is the CloudTrail LookupEvents API.
Plan-style tools compare declared with live state but say nothing about who changed it. Ansible's check mode is a simulation; modules that support it report the changes they would make, modules that do not report nothing, and diff mode adds before-and-after detail where modules support it. A clean check run therefore proves less than it seems when some modules lack support.
| Evidence source | What it establishes | What it cannot establish | Record in the task |
|---|---|---|---|
| AWS Config configuration items | State of a resource at each capture, ordering, state hash | Who made the change; relatedEvents is empty as of version 1.3 | Item ids, capture times, state ids, hashes |
| Cloud audit trail lookup | API caller identity and time for the change | Why the change was made | Event ids, principal type, event time |
| Terraform plan in refresh-only mode | Remote objects changed outside Terraform | Actor, intent or approval | Plan output summary, Terraform version |
| kubectl diff against config files | Fields where live objects differ from files | Changes already overwritten or written back | Object kind, field paths, diff time |
| Ansible check and diff mode | Changes a playbook would make, with before and after where supported | Anything from modules without check mode support | Playbook version, modules lacking support |
What the configuration and change record reconciliation task includes
The task is one row per observed change, with linked evidence and a decision that a second reviewer could reproduce from the same records. Required fields are grouped so that a missing group excludes the task instead of leaving a gap the model will fill with a guess.
At a glance
What the configuration and change record reconciliation task includes
| Field group | Required fields | Rule | Exclude the task if |
|---|---|---|---|
| Configuration item | Item id, type, environment class, owner team | Ids are pseudonymous and stable across the dataset | The item cannot be identified consistently |
| Baseline reference | Baseline or code version, approval date, attribute expected value | Baseline dated before the observed change | Only a current baseline exists |
| Observed change | Attribute path, before value class, after value class, detection time, evidence source | Values are typed placeholders, not raw secrets or addresses | Before and after cannot both be shown |
| Actor evidence | Principal type, role, audit event reference, time | Human, automation or vendor identity stated | No audit evidence and no admission in a record |
| Candidate decision records | Change ids, incident ids, approval times, linked items | All candidates within the time window listed, not only the winner | Ticket systems were not searched |
| Match decision | Decision value, rule applied, reviewer role, decision time | One of five decision values below | Reviewer cannot cite a rule |
| Incident context | Alert ids, timeline entries, runbook version | Order of events preserved | Timeline was rewritten after the fact without a note |
| Redaction log | Fields removed or generalized, method, reviewer | Secrets and exploitable details removed before export | Any credential or live endpoint remains |
| Outcome | Baseline updated, change reverted or retro-approved, verification time | Outcome from a record, not a status field | Only the ticket's closed status is available |
How to decide which ticket, if any, explains the change
A change is matched to a decision record only when time, configuration item, actor and content all agree; a partial match is recorded as such. The five decision values below keep the labels comparable across teams.
Write the time window down. A narrow window misses changes made just before a ticket was opened; a wide one attaches unrelated tickets. A window anchored on the incident's first alert and its resolution time, extended by a stated margin, is easier to defend than one picked per case.
Checklist · 5 items
- Matched approved change: an approved change record names the item and the change, was approved before the change time, and the actor is the implementer or the pipeline.
- Matched emergency change: the change was made during an incident by a responder, and a change record approved it afterwards; record the approval delay.
- Incident action without a change record: the incident timeline or ticket notes describe the action, but no change record exists; this is the core reconstruction case.
- Expected automated change: a service identity made a change within bounds the baseline allows, such as replica counts under an autoscaler.
- Unexplained: no candidate record agrees on item, time and actor; escalate to the system owner and security before the task is used anywhere.
0 of 5 ticked. Tick each item as you go.
What evidence explains the IT decision beyond a closed ticket or a final configuration?
The decision is explained by the sequence, not by either end state. A closed ticket tells you the incident ended; a final configuration tells you where the system landed. Neither says what the responder believed, which option they rejected, or whether the change they made was the fix or a side effect of the fix.
Four kinds of evidence fill that gap. The alert history shows what the responder saw and when. Timeline entries and chat notes written during the incident show what they suspected, in their own words and with their own timestamps. The runbook version in force shows what the documented procedure told them to do, so a departure from it can be seen. The audit trail shows exactly which change was made and by which identity, which may differ from what the notes claim.
When those sources disagree, keep the disagreement in the task instead of resolving it quietly. A note that says restarted while the audit trail shows a network rule edit is one of the most useful patterns a model can learn from operational records: the written explanation and the actual action drift apart under pressure. Record which source the reviewer trusted for each fact and why.
Export order matters for this reason. Pull the configuration history and audit trail for the window first, then the incident record and its timeline, then change records, and only then the ticket text. Reading the ticket last stops its summary from steering the reconstruction toward the story the ticket already tells.
Worked example (illustrative): a network rule widened during an outage
This example is illustrative and uses invented systems, times and identifiers. No real organization or incident is described.
The case: a payment service throws timeouts at 02:10. At 02:24 an on-call engineer widens an outbound network rule on one environment. The service recovers at 02:31. The incident ticket is closed at 03:05 with the note: restarted service, monitoring. No change request exists. Two days later a refresh-only plan reports the rule as changed outside the declared configuration.
- Observed change: the rule's allowed destination range changed from a narrow class to a broad class; the task records value classes, not the actual ranges.
- Actor evidence: the audit trail shows a human principal in the on-call role making the call at 02:24.
- Candidates: one incident ticket inside the window, no change records; the ticket text mentions a restart but not the rule.
- Decision: incident action without a change record. The reviewer cites the time window, item and actor match, and notes that the ticket text contradicts the evidence.
- Outcome: the rule is narrowed again and the baseline updated by an approved change three days later; the task links that change record as the outcome.
- Exception: in the same week the replica count of the service moved several times. The actor was the autoscaler's service identity and every value stayed inside configured bounds. That drift is classified as an expected automated change and does not need a ticket, but it stays in the dataset as a negative example so a model learns that not every unticketed change is a process failure.
Keeping credentials and attack paths out of the task
Configuration records are full of secrets by design, so redaction is part of building the task, not a final pass. The OWASP Logging Cheat Sheet lists data that should not be recorded directly in logs and instead removed, masked, hashed or encrypted, including access tokens, authentication passwords, database connection strings, and encryption keys and other primary secrets. Configuration diffs need the same treatment, plus a few additions specific to infrastructure.
Replace values with typed placeholders that keep the meaning: a key becomes secret of type API key, rotated yes or no; an address range becomes internal narrow, internal broad or public. Remove hostnames, account ids and resource names that point to live systems. Leave out any case where the change exposed a weakness that is still open, and any case from a customer environment unless the customer's agreement permits it. The task should teach the reasoning about records, not provide a map of someone's network.
Which drift cases belong in internal review only
Some reconstructions are valuable to your own operations team and still wrong for an external dataset. Keep them internal when the security team is still investigating, when the change touched a regulated system whose records carry their own retention and disclosure rules, when a managed service provider's records describe client environments the provider does not control, or when the only evidence of the actor is a personal account that cannot be pseudonymized without breaking the case.
Unexplained changes deserve particular care. Until the system owner and security have closed them, they may be evidence of unauthorized access, and they should not leave the company in any form. Once closed, they can become some of the most instructive cases, with the investigation's conclusion recorded as the outcome.
When change and incident history could be licensed
IT teams that have kept configuration history, audit trails and change records for years hold the evidence this task needs. Licensing that history through SourceX means SourceX runs the rights review, contracting, preparation and delivery while the team decides which records are offered.
The company decides what is in scope, and its signature on the scope and terms comes before any data is delivered. Large deliveries can remain in the company's own storage or ship on encrypted drives, so multi-terabyte log archives do not need to be uploaded to a hosted service. Before any of that, a short assessment confirms fit without a single log file being shared.
Frequently asked questions
What is IT configuration drift?
Configuration drift is any difference between a system's approved baseline or declared configuration and its actual state. It can come from manual fixes, imperative commands, automation or vendor updates, and it matters when no approved record explains it.
Is every unticketed configuration change unauthorized?
No. Automation such as an autoscaler can change values within bounds the baseline allows, and some providers apply changes under their own maintenance terms. Classify those as expected or external changes, and keep the unauthorized label for changes no record explains.
Can AWS Config show who made a configuration change?
Not by itself. A configuration item records the resource state, capture time and a state hash, and its API reference says the relatedEvents field is empty as of version 1.3. Use the CloudTrail LookupEvents API to find the caller for a change.
How do you find changes made outside Terraform?
Run a plan in refresh-only mode, which updates state and outputs to match remote objects changed outside Terraform. It shows what changed but not who changed it or why, so pair it with the audit trail and the incident timeline.
Should configuration values be kept in a drift dataset?
Keep the meaning, not the raw value. Replace secrets, addresses, hostnames and account ids with typed placeholders such as secret of type API key or public address range, and record each redaction in a log reviewed before export.
Sources
- NIST SP 800-128 describes a baseline configuration as reviewed and agreed specifications that change only through change control, with monitoring used to find undocumented and unauthorized changes. Source
- NIST SP 800-171 Revision 3 describes configuration change control as tracking, reviewing, approving or disapproving, and logging changes to the system. Source
- The AWS Config ConfigurationItem reference defines capture time, a state id that orders items, a state hash for comparison, and says relatedEvents is empty as of version 1.3 with CloudTrail LookupEvents as the route to events. Source
- Terraform's plan documentation describes refresh-only mode for reconciling state with remote objects changed outside Terraform, for example while responding to an incident. Source
- Kubernetes documentation shows that kubectl apply retains writes made to live objects without merging them back into configuration files, and that a kubectl scale change is absent from the last-applied configuration. Source
- Ansible documentation says check mode is a simulation in which modules without check mode support report nothing, and diff mode gives before-and-after comparisons where supported. Source
- The OWASP Logging Cheat Sheet lists access tokens, passwords, database connection strings and encryption keys among data to remove, mask, hash or encrypt rather than log directly. Source
See if your company qualifies
A short company assessment. No data uploads are needed.