The pipeline for security awareness training evidence automation joins two lists that usually live in different exports. One list is the people who should have finished the course: active employees and contractors from the identity provider, with service accounts removed. The other list is the completion export from the training platform the company already runs. The join writes who is inside the window, who is missing, and who is on a dated exception. That file is the evidence. A slide that says the company "does annual training" is the policy. The file is the operation of the control.
Rutagon builds the join and the refresh for SOC 2 teams who otherwise paste the LMS spreadsheet into a folder the week the auditor asks. The GRC platform can store the summary percentage. The percentage is not the population. NIST SP 800-53 AT-2 is the awareness-training control these requests map back to. The auditor's question is closer to: for the people who were in scope during the period, show completion, and explain the gaps. We generate that explanation from the roster and the export, on a schedule, so the folder is not a scavenger hunt.
How security awareness training evidence automation builds the gap list
The roster is the IdP. An active user with a person mailbox is in the population. A user whose type is service, a break-glass account with no human, and a shared mailbox login are out. The rule is written down in the job, not re-decided each quarter in a spreadsheet filter. Start date comes from the HR source the IdP already syncs, or from the HRIS file if the IdP does not have it. Manager comes from the same record we use to route access reviews. If manager is empty, the person still appears on the gap list, and the row is marked no_manager so the missing attribute is visible.
The LMS export needs a stable person key and a completion time. Email is the usual key. We normalize case and we do not fuzzy-match on display name. Two people and one shared mailbox alias will otherwise collapse into a single completion. The course id is specific. "Completed something" is not the annual security course. A phishing simulation click-rate is a different control and a different file. Mixing them lets a person who clicked a simulation and skipped the course show up as trained.
The window is two tests. For anyone whose start date is older than the new-hire threshold, completion has to fall inside the last 365 days, or inside whatever period the policy names. For a new hire, completion has to fall within the number of days the policy gives them after the start date, commonly 30. A new hire inside that window is in_progress, not missing. A new hire past the window with no completion is missing. Treating every incomplete new hire as a finding on day one produces a list managers ignore. Treating them as fine forever produces a finding in the sample.
as_of: 2026-10-14
population: active IdP users, service accounts excluded
course_id: annual-security
annual_window_days: 365
new_hire_grace_days: 30
match_key: normalized email
missing: person, manager, start_date, days_overdue
exception: leave, owner, end_date
in_progress: hired inside the grace window, no completion yetExceptions are a register, not a note in the margin. Leave, a long disability absence, a contractor whose statement of work has not started: each row has a person, a reason code, an owner, and an end date. The join suppresses that person from missing only while as_of is before the end date. The day after, they return to the gap list if they still have no completion. An exception with no end date is rejected at load. That is how "she's on leave" becomes a permanent hole in the population.
The output is three lists and a cover count: completed in window, missing, in progress, exceptions. The cover count is completed / (population - active exceptions), and the denominator is printed. A percentage without a denominator is how 100% gets reported on the people who happened to be in the LMS, ignoring the people who were never enrolled. We enroll by the gap list. The LMS campaign is fed from missing and from in_progress when the grace window is half elapsed. The evidence file records the campaign id so the next run can show that the gap was sent somewhere, not only that it existed.
What the auditor and the GRC platform each get
The auditor's request is usually a population and a completion record for the period. We write one object per run to the evidence bucket: the as_of timestamp, the roster source, the course id, the four lists, and the count of IdP users excluded as non-people. PBC request automation can attach that object to the training request without a manual export. The GRC platform gets the cover counts and the link. It does not get a second, hand-edited percentage.
Freshness is the same clock as the other evidence jobs. Evidence freshness tracking marks this file stale when as_of is older than the window we set, often 30 days, shorter during the audit. A stale file is not attached to a customer questionnaire as if it were current. The questionnaire map in the evidence register points at this generator. When the file is stale, the questionnaire row goes to the gap sheet.
People who left during the period stay in the historical file for that period if they were in scope while employed and the course was due. They drop out of the live gap list once the IdP shows them disabled. The live list is who needs a nudge now. The period file is who was in scope then. Overwriting the period file with today's roster is how leavers disappear from the sample and new hires appear in a quarter they were not in. Each run writes a new key. It does not replace the key for a closed quarter.
Managers see only their missing people. The full list is the evidence object. The Monday notice is a smaller extract: person, course, days overdue. That extract is enough for a manager to mark leave, which becomes an exception with an end date, or to tell the person to finish the course.
Contractors are in or out based on the roster rule, not based on who the LMS administrator remembered to invite. If the policy says contractors with production access take the course, the population is the join of active contractors and the entitlement "production access," not the full contractor list and not the employee list alone. That join uses the same entitlement extract as the access review. A contractor with a laptop and no production access stays out if the policy says so. The policy sentence is copied onto the evidence cover so the population definition is sitting next to the counts.
Where the automation stops
The job does not pick the course content, and it does not grade a quiz. The LMS does that. We do not scrape a screenshot of the admin dashboard. We do not mark a person complete because their manager said so in chat. Completion is a row in the export with a timestamp. If the LMS is wrong, the fix is in the LMS, and the next run picks it up.
A person with two emails is a known miss. The roster email and the LMS email differ when someone trained under a personal address or an old domain. We keep an alias table, owned by HR operations, mapping one canonical person id to the emails we have seen. The join uses the person id after the alias lookup. Guessing by last name is not in the job. An unmatched LMS row is reported as orphan_completion so a finished course is not dropped on the floor and is not credited to the wrong human.
Multi-framework requests often ask for the same completion file. Multi-framework evidence automation is how one object is referenced from more than one control. We still generate it once. A second course required by a different framework is a second course id and a second join, not a copy of the annual file with the title edited.
Security automation runs this on a schedule next to the access-review export and the log-retention collect. The training proof is the same kind of artifact: a population, a result, a date, and a list of exceptions with owners. Rutagon's deliverable is that join, the exception register with mandatory end dates, the period snapshot that is not overwritten, and the manager extract that contains only the gaps.
Frequently Asked Questions
Who is in the training population?
Active people from the IdP, after service accounts, break-glass accounts, and shared mailbox logins are removed. Contractors are included only when the policy says so, using the entitlement the policy names. The cover sheet prints the exclusion counts so the denominator is explainable.
What counts as complete?
A row in the LMS export for the named course id, with a completion time inside the annual window, matched on normalized email or on a person id from the alias table. A phishing drill, a different course, or a manager's confirmation in chat does not count. Unmatched LMS rows are listed as orphans.
How are new hires handled?
Inside the grace period after the start date they are in_progress. After the grace period with no completion they are missing. The grace period is the number in the policy, and the evidence file records it. They are not hidden from the population, and they are not failed on day one.
How do leave and other exceptions work?
An exception names the person, a reason code, an owner, and an end date. The gap list suppresses them only until that date. An exception without an end date is rejected. The day the exception ends, a person with no completion returns to the missing list.
What should be stored for the period?
One immutable object per run, and a final object for the closed period that is not replaced by next month's roster. It includes the four lists, the denominator, the course id, and the roster source. The GRC platform links to it. The live manager notice is only the current gaps.
Ask for an evidence diagnostic that compares your IdP roster to the latest training export and lists who is missing, in progress, or on an open-ended exception. Contact Rutagon at rutagon.com/contact, 907-841-8407, or contact@rutagon.com.