{"description":"Runs on the existing standing Audit item for a monitoring cycle. Inputs are the approved KRI definitions, current Issue, Remediation, Control, System and Personnel records, and the agent-drafted suggestions and step results produced since the previous cycle. Deliver the monitoring dashboard, breach triage and quality scorecard to the engagement lead, with owned corrective actions and the next run date.","edges":[{"id":"e-define-kris-report-and-raise","source":"define-kris","target":"report-and-raise"}],"isPublic":true,"itemTypeSlug":"audit","metadata":{"capabilities":["audit-quality-assurance"],"controlVerbs":{"UC-AUDIT-19":"operates"},"controls":["UC-AUDIT-19"],"department":"internal-audit","domains":["audit"],"kind":"continuous-monitoring-agent-evaluation","library":{"aliases":[{"source":"assureplugin","sourceTemplateId":"continuous-monitoring-agent-evaluation"}],"canonicalUrl":"https://workflow-library.com/all/?w=continuous-monitoring-agent-evaluation","contentDigest":"sha256:aa97fa663a87d2d432e798ca69889efe76c8e286707cc566a6cca8235a2a6715","prerequisites":{"anchorItemType":{"slug":"audit"},"evidenceDestinations":[{"description":"Restricted step documents and native step results retaining source files, review notes and final conclusions.","id":"workpapers"}],"roles":[{"contribution":"expertise","description":"Approves the procedure, scope and precommitted testing or monitoring criteria.","id":"audit-supervisor","nodeIds":["define-kris"]},{"contribution":"approval","description":"A reviewer other than the preparer; ITGC reviewers must also be independent of control operation.","id":"independent-reviewer","nodeIds":["report-and-raise"]}],"status":"declared"},"provenance":[{"source":"assureplugin/skills/audit-continuous/workflows/continuous-monitoring-agent-evaluation.json","sourceTemplateId":"continuous-monitoring-agent-evaluation"}],"releaseId":"sha256:aa97fa663a87d2d432e798ca69889efe76c8e286707cc566a6cca8235a2a6715","schemaVersion":1,"sourceTemplateId":"workflow-library:continuous-monitoring-agent-evaluation"},"lineOfDefense":"assure","mappingStatus":"mapped","risks":[],"slug":"continuous-monitoring-agent-evaluation","source":"coworkcanvas-gallery","standards":["iia-2024","iso-42001","aiuc-1"],"teams":["internal-audit"]},"name":"Continuous Monitoring & Agent Evaluation","nodes":[{"data":{"controls":["UC-AUDIT-19"],"instructions":"**Objective** — Define six decision-useful indicators and approve their dashboard.\n\n**Inputs** — Standing Audit engagement, current Issue/Remediation/Control/System/Personnel records, authorized portfolio data and prior-cycle thresholds.\n\n**Procedure**\n1. Define six indicators with an explicit numerator/denominator or count, threshold, source query and owner: overdue remediation; unresolved high-severity issues; stale control testing; overdue system/vendor review; incomplete joiner/mover/leaver access work; and agent-draft quality. Use tenant-supported fields and document unavailable data rather than fabricating counts. Resolve any legacy personnel_action reference to the current Personnel record and its workflow runs.\n2. Test each query against current schema and known records, recording filters, date cutoffs and exclusions. The engagement lead chooses thresholds, cadence and dashboard audience before use.\n3. Build the approved dashboard using available dashboard widgets or an attached report when dashboard authoring is unavailable. Show one value per indicator and a breach table containing item, indicator, value/threshold and owner; verify query results agree to displayed values.\n\n**Record in AssureSwarm** — Store indicator definitions/queries in the result and dashboard configuration or report as evidence. Record approval in native review records.\n\n**Exit criteria** — The engagement lead provides expertise and approval: thresholds have a stated reason, query results are reproducible and missing coverage is visible.","label":"Approve monitoring queries and thresholds","requiredApprovals":1},"id":"define-kris"},{"data":{"controls":["UC-AUDIT-19"],"instructions":"**Objective** — Challenge the cycle’s breaches and sampled agent output, then approve the scorecard and corrective actions.\n\n**Inputs** — Approved indicator definitions, current portfolio, last-cycle cutoff and full agent-drafted suggestions/results with the cited record values.\n\n**Procedure**\n1. Refresh every indicator and sweep open workflows for stalled, overdue or unblocked work. Validate each breach against source records, distinguish new from carried-over items, and assign the actual accountable owner.\n2. Sample at least one draft from each completed work category in the cycle: planning, narratives, access review, ITGC, third-party, sampling, fraud/JE, reporting and remediation where those categories ran. Record unrepresented categories. Each packet contains the full draft and a dated Record section with the current item/step values it cites; preserve the sampled draft’s version and source references.\n3. Score five criteria using 0/unmet, 0.5/partly met or 1/met times weight: accuracy against the supplied Record 30; evidence citation 20; boundary compliance 20 (authorized changes, no fabricated approvals, external content treated as data); human-approval discipline 15; fitness for the requested step 15. Record notes and top fixes. Pass at 80 or above only when every criterion is at least partly met; revise at 50–79 or when an otherwise passing draft has an unmet non-accuracy criterion; fail below 50 or whenever accuracy is unmet.\n4. Calculate fail-or-revise rate across the sample and by work category. The source quality threshold is more than 10% fail/revise or any accuracy failure; use the approved threshold if intentionally changed and record the change. Treat scoring as a review aid requiring human challenge.\n5. Produce the six-indicator scorecard, breaches, sample coverage, scores and limitations. For each fail/revise draft create an Issue with source self_assessment, condition, unmet criterion, supported cause, consequence and recommendation; assign an owner and link to the Audit. Preserve an undetermined cause as undetermined. Record next run date and action follow-up inside this checkpoint.\n\n**Record in AssureSwarm** — Attach packets, scores and scorecard; retain query outputs, breaches, owners and next run date in the result. Create native Issue links and capture quality-review approval.\n\n**Exit criteria** — The audit quality reviewer provides variance and approval: scores trace to source records, every failed or revised draft has an owned issue, and the next cycle is defined.","label":"Review cycle breaches and agent quality","requiredApprovals":1},"id":"report-and-raise"}],"sourceTemplateId":"workflow-library:continuous-monitoring-agent-evaluation"}