Having controls is not the same as having effective controls. CommunityPay measures whether its automated enforcement mechanisms actually produce appropriate human responses. This page describes the control effectiveness measurement system — what it measures, how, and why it matters.
The Measurement Gap
Most accounting software (and most SOC reports) answer the question: "Do controls exist?"
CommunityPay answers a harder question: "When controls fire, do people respond appropriately?"
This distinction matters because: - A control that fires but is never acknowledged is effectively absent - A control that fires and is always overridden may indicate the control is miscalibrated - A control that fires and is resolved within SLA indicates operational governance
What Gets Measured
Response Time Metrics
When an exclusion is created or an escalation is triggered, the clock starts. The control effectiveness service measures:
| Metric | Definition |
|---|---|
| Mean response time | Average time from alert creation to first acknowledgment |
| Median response time | 50th percentile response time (more robust to outliers) |
| p95 response time | 95th percentile — captures the tail (slow responses) |
These metrics are computed from immutable escalation event records. Each event records a timestamp; response time is the difference between the alert event and the acknowledgment event.
Acknowledgment Rate
The percentage of alerts that receive an explicit acknowledgment from the responsible party.
Acknowledgment Rate = Acknowledged Alerts / Total Alerts Sent
A declining acknowledgment rate is itself a risk signal — it suggests that responsible parties may be ignoring governance notifications.
SLA Compliance
Each exclusion and escalation has an expected resolution timeframe (the SLA). The SLA compliance metric tracks:
SLA Compliance = Exclusions Resolved Within SLA / Total Resolved Exclusions
SLA timeframes vary by severity: - CRITICAL: Resolution expected within 24 hours - HIGH: Resolution expected within 3 days - MEDIUM: Resolution expected within 7 days - LOW: Resolution expected within 30 days
Override Pattern Analysis
Overrides are legitimate — they exist for situations where guards correctly identify a condition but authorized personnel determine the operation should proceed anyway. However, override patterns reveal governance health:
By scope:
| Scope | Meaning |
|---|---|
| SINGLE | One-time override for a specific transaction |
| BATCH | Override covering a batch of related transactions |
| TEMPORAL | Time-bounded override (e.g., "allow for 48 hours") |
| CATEGORICAL | Override for a category of transactions |
By authorization type:
| Type | Meaning |
|---|---|
| BOARD_APPROVAL | Board resolution authorizes the override |
| ADMIN_EMERGENCY | Administrative emergency (time-critical) |
| SYSTEM_MIGRATION | Override during system migration or data correction |
| AUDIT_CORRECTION | Override to correct an audit finding |
A high proportion of ADMIN_EMERGENCY overrides relative to BOARD_APPROVAL overrides may indicate governance gaps. The override analysis surfaces these patterns.
Escalation Chaining
When an alert is not acknowledged within SLA, it escalates up the chain:
HOA Admin (T+0) → Property Manager (T+3d) → Platform Admin (T+7d)
The chaining rate tracks how often escalations propagate beyond the initial recipient:
Chaining Rate = Escalations with Chain > 1 / Total Escalations
A high chaining rate suggests that first-responders are not engaging with governance alerts, which is itself a control effectiveness concern.
Measurement Sources
All control effectiveness metrics are derived from immutable source data:
| Source | Lineage | What It Provides |
|---|---|---|
| EscalationEvent | LINK_EVENT (append-only) | Alert creation, acknowledgment, resolution timestamps |
| ExclusionNotificationEvent | LINK_EVENT (append-only) | Notification delivery, success/failure |
| ExclusionStatusHistory | LINK_EVENT (append-only) | Exclusion status transitions with timestamps |
| AuditOverride | ROOT_EVENT | Override scope, authorization type, usage count |
| EnforcementDecision | ARTIFACT (immutable) | Guard results, decision outcomes |
Because all source data is immutable or append-only, the metrics cannot be manipulated by modifying historical records. A CPA reviewing control effectiveness metrics can verify them against the raw event data.
Governance Controls Attestation Integration
Control effectiveness metrics feed directly into the Governance Controls Attestation (GCA). The GCA's overall assessment is derived from these metrics:
| Assessment | Criteria |
|---|---|
| STRONG | Control pass rate > 95%, SLA compliance > 95%, low override rate |
| ADEQUATE | Control pass rate > 85%, SLA compliance > 85% |
| NEEDS_IMPROVEMENT | Control pass rate > 70%, or SLA compliance 70-85% |
| DEFICIENT | Control pass rate < 70%, or SLA compliance < 70% |
This creates a direct, measurable link between operational governance behavior and the board-facing attestation. Boards do not receive subjective assessments — they receive quantified metrics derived from verifiable source data.
Control Plane Health
In addition to governance-level metrics, CommunityPay tracks technical control plane health:
| Metric | What It Measures |
|---|---|
| Total guard evaluations | How many times guards have been invoked |
| Guard pass rate | Percentage of evaluations that passed |
| Average evaluation time | Mean execution time per enforcement evaluation |
| Decision distribution | Breakdown of ALLOW / BLOCK / OVERRIDE / ERROR |
| Guard-specific performance | Per-guard execution time and pass rate |
These technical metrics confirm that the enforcement system is operating correctly — guards are being invoked, evaluations are completing in expected time, and the error rate is minimal.
What This System Proves
The control effectiveness measurement system answers questions that exist above the technical control layer:
- "Are our controls monitored?" — Response time and acknowledgment metrics prove that alerts reach people and people respond
- "Are issues resolved promptly?" — SLA compliance metrics prove that governance issues are addressed within defined timeframes
- "Are overrides appropriate?" — Override pattern analysis shows whether bypasses are authorized through proper channels
- "Is governance improving or degrading?" — Trend analysis over periods shows the direction of governance health
- "Can we prove all of this?" — All metrics are derived from immutable source data that an auditor can independently verify
CARI Integration
Control effectiveness metrics measured here feed directly into the CARI Enforcement Integrity sub-score, which carries a 15% weight in the composite CARI score. Override rates, SLA compliance percentages, and enforcement block ratios are key signals: an HOA with high override rates or missed SLA targets will see a reduced Enforcement Integrity component. These metrics are computed from the same immutable enforcement decision records used for audit verification.
For published methodology and component weights, see CARI Methodology and Scoring Framework.
How CommunityPay Enforces This
- Response time metrics captured: mean, median, and p95 for escalation acknowledgment
- Acknowledgment rate tracked: percentage of alerts acknowledged by responsible parties
- SLA compliance measured: percentage of exclusions resolved within their expected timeframe
- Override patterns analyzed: by scope (SINGLE, BATCH, TEMPORAL, CATEGORICAL) and by authorization type
- Escalation chaining tracked: how often escalations propagate up the chain
- All metrics derived from immutable source data — escalation events, notification events, override records