Policy Testing

Use the Policy Test tool to verify detection coverage before deploying rule changes, and to tune false positive rates over time.

The Test Interface

Dashboard → Policies → Test

Paste any text into the input box and click Evaluate. The response shows:

• Action — block, warn, or log
• Risk score — 0.0 to 1.0
• Matched rules — each rule that fired, with category and confidence
• Response time — confirms you are within the < 100ms target

Test Coverage Checklist

Run at least one test per data category after changing compliance profiles or custom rules:

Category	Should action
Credit card number	block (PCI, SOC 2)
Email address	block or warn (GDPR, CCPA)
Social Security Number	block (GDPR, HIPAA)
API key / secret	block (SOC 2, ISO 27001)
Medical record content	block (HIPAA)
Benign technical content	allow

Test Cases by Data Category

PII

# Full name + contact — should trigger PII rule
My name is Sarah Johnson and you can reach me at sarah.johnson@acme.com or +1 (555) 867-5309.

# Social Security Number — should block
Please store SSN 078-05-1120 for the employee record.

# Passport number — should block
Passport number: US1234567, expires 2028-03-15.

# Home address — should warn or block
Ship to: 123 Main Street, Springfield, IL 62701.

PHI (HIPAA)

# Medical record + condition — should block
Patient MRN 00234891 presented with Type 2 diabetes and hypertension.

# Health insurance ID — should block
Insurance member ID: BC123456789, group: ACME-HEALTH-2024.

# Prescription details — should block
Rx: Metformin 500mg twice daily for patient DOB 1978-04-22.

Financial / PCI

# Credit card — should block
Please charge Visa 4532015112830366, exp 09/27, CVV 123.

# IBAN — should block
Wire to IBAN: DE89370400440532013000, BIC: COBADEFFXXX.

# Bank account — should block
Account: 123456789, routing: 021000021.

Credentials

# API key pattern — should block
OPENAI_API_KEY=sk-proj-abc123XYZ789abc123XYZ789abc123XYZ789abc123XYZ789

# AWS access key — should block
AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

# Private key header — should block
-----BEGIN RSA PRIVATE KEY-----
MIIEowIBAAKCAQEA...

# Generic secret — should block
DATABASE_URL=postgresql://admin:SuperSecretPassword123@db.internal:5432/prod

Benign content (should NOT trigger)

These inputs verify you have not over-tuned detection:

# Technical discussion — should allow
Can you explain how SHA-256 hashing works in blockchain applications?

# Internal jargon without sensitive data — should allow
The Q2 roadmap includes three milestones: M1 platform migration, M2 API versioning, M3 dashboard redesign.

# Partial / clearly fictional data — should allow
In our example, we'll use the test email user@example.com and the placeholder card 4111111111111111.

TIP

The test card 4111111111111111 passes Luhn validation and is used in PCI DSS testing environments. If your PCI profile blocks it, that is correct behavior — adjust expectations accordingly.

Regression Testing After Rule Changes

After modifying a custom rule or changing a compliance profile, run the full suite of test cases above to confirm:

1. All true positives still fire (no missed detections)
2. No new false positives appeared in the benign test set

Run tests as an admin with the same compliance profile active as your production users.

Tuning False Positives

If a rule fires on legitimate content:

1. Identify the matched rule — the test response shows matchedRules[].ruleId
2. Add an exception — Dashboard → Policies → Custom Rules → Edit the rule
3. Narrow the pattern — make the regex more specific, or add a negative lookahead
4. Lower confidence threshold — set confidence to low to log rather than block

Example: A credential detection rule is triggering on log output that contains the word secret in a non-sensitive context:

{
  "name": "Credential detection - narrowed",
  "pattern": "(secret|password|token)\\s*[=:>]\\s*['\"][^'\"]{8,}['\"]",
  "category": "credentials",
  "action": "block",
  "confidence": "high"
}

The narrowed pattern requires an assignment operator and a quoted value, reducing false positives from log lines that mention the word "secret" in plain text.

Bulk Testing via API

For CI integration or large test suites, use the evaluation API directly:

# Evaluate a single string
curl -s -X POST \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"content": "My SSN is 078-05-1120", "organizationId": "org_abc123"}' \
  https://your-api-gateway/api/v1/policies/evaluate

# Expected response
{
  "action": "block",
  "riskScore": 0.95,
  "matchedRules": [
    { "ruleId": "pii-ssn", "category": "pii", "confidence": "high" }
  ],
  "responseTimeMs": 12
}

Wrap this in a test script and run it in CI to catch regression before deploying rule changes to production.

Performance Baseline

The Policy Engine targets < 100ms for policy evaluation. If you observe evaluation times above 250ms:

1. Check Redis connectivity — cache misses force rule recompilation on every request
2. Reduce the number of active custom rules — very large rule sets (> 200 rules) increase evaluation time
3. Check database query times — audit log writes should be < 50ms

The test interface shows response time for each evaluation. Use it to benchmark before and after rule changes.