DMARC-Sentinel/README.md

DMARC Sentinel

DMARC Sentinel is a self-hosted Docker application for monitoring DMARC aggregate reports. It reads report emails from configured IMAP folders, extracts XML reports, stores normalized telemetry in SQLite, runs deterministic security and deliverability analysis, generates alerts, and uses an LLM to turn already-derived facts into human-readable summaries and recommendations.

Detection is deterministic. The LLM is mandatory for explanations, daily summaries, weekly summaries, and dashboard text, but it is never the source of truth for whether an alert exists or how severe it is.

What It Does

• Connects to one or more IMAP inboxes and reads only the configured DMARC folder by default.
• Supports .xml, .xml.gz, .gz, and .zip report attachments.
• Parses DMARC aggregate XML into reports, records, and authentication results.
• Deduplicates reports by raw XML SHA256.
• Classifies known senders using CIDR allowlists, DKIM auth domains, SPF auth domains, and aligned DKIM evidence.
• Creates deterministic alerts for unknown failures, known sender failures, quarantine/reject dispositions, new sources, high failure rates, missing reporters, and first-seen policies/reporters.
• Uses OpenAI to produce alert explanations and daily/weekly operational summaries from sanitized JSON facts.
• Exposes a server-rendered FastAPI dashboard and gethomepage Custom API widget endpoints.
• Sends email alerts and digests when SMTP settings are configured.

First Run

cp .env.example .env
cp config/config.example.yml config/config.yml
nano .env
nano config/config.yml
docker compose up -d --build

Then process the existing mailbox backlog:

docker compose exec dmarc-sentinel python -m app.cli backlog --inbox tukutoi --folder DMARC

Visit the app through your NPM reverse proxy:

https://sentinel.tukutoi.com

mailcow / SOGo Expectation

No installation is required on the mailcow server. The app logs into the existing mailbox over IMAP.

Expected mailbox setup:

mail_account: hello@tukutoi.com
catch_all: true
reports_recipient: dmarcreports@tukutoi.com
imap_folder: DMARC
primary_domain: tukutoi.com

Move or filter DMARC aggregate report messages into the DMARC folder in SOGo/mailcow. DMARC Sentinel reads that folder unless a CLI/API backlog option overrides it.

Docker Compose

The compose file binds the dashboard to 127.0.0.1:8000 for local HITL testing and also attaches to the external NPM proxy network with a static container IP:

networks:
  npm_proxy:
    external: true

NPM should proxy to:

http://192.168.99.18:8000

The app listens inside the container on 0.0.0.0:8000; on a local Docker host it is also available at:

http://127.0.0.1:8000

Configuration

Runtime config is read from config/config.yml; use config/config.example.yml as the template.

Important sections:

• app: base URL, timezone, poll interval, database URL, log level, attachment limits.
• security: Basic Auth and homepage bearer-token settings.
• llm: OpenAI model, retry, timeout, and data-sending controls. Raw XML and raw email are disabled by default.
• inboxes: IMAP host, folder, recipient, processed/failed folder behavior, and env var names for credentials.
• known_senders: deterministic sender classification rules per domain.
• alerts: SMTP env var names and deterministic thresholds.

Secrets live in .env, not in config.yml.

OPENAI_API_KEY is required when llm.provider is openai. The app only bypasses this when DMARC_SENTINEL_ALLOW_NO_LLM_FOR_TESTS=true, which is intended for tests.

Adding another inbox or monitored domain should only require adding entries to inboxes and known_senders.

Backlog Command

docker compose exec dmarc-sentinel python -m app.cli backlog --inbox tukutoi

Options:

--folder DMARC
--since YYYY-MM-DD
--before YYYY-MM-DD
--limit 500
--dry-run
--reprocess
--mark-seen

Backlog mode scans all matching messages, skips already imported XML hashes unless --reprocess is passed, and does not modify messages unless configured or explicitly flagged.

gethomepage Widget

Endpoint:

GET /api/homepage
Authorization: Bearer YOUR_HOMEPAGE_API_TOKEN

Example gethomepage config:

- Monitoring:
    - DMARC Sentinel:
        href: https://sentinel.tukutoi.com
        description: DMARC monitoring
        widget:
          type: customapi
          url: https://sentinel.tukutoi.com/api/homepage
          headers:
            Authorization: Bearer YOUR_HOMEPAGE_API_TOKEN
          mappings:
            - field: status
              label: Status
            - field: dmarc_pass_rate
              label: Pass
            - field: critical_alerts
              label: Critical
            - field: warnings
              label: Warnings
            - field: summary
              label: Summary

Domain-specific endpoint:

GET /api/homepage/tukutoi.com

Dashboard

The dashboard is protected with Basic Auth using DASHBOARD_USERNAME and DASHBOARD_PASSWORD. It includes:

• / overview with monitored domains, daily volume, pass rate, alerts, unknown sources, last check, and latest LLM daily summary.
• /domains/{domain} with trends, top sources, known vs unknown sender data, reports, alerts, and LLM summary.
• /reports/{report_id} with normalized report evidence and record table.
• /alerts with acknowledge, resolve, and reopen actions.
• /inboxes with status and manual process/backlog buttons.

HTTP API

Implemented endpoints:

GET /health
GET /api/homepage
GET /api/homepage/{domain}
GET /api/domains
GET /api/domains/{domain}/summary
GET /api/domains/{domain}/reports
GET /api/domains/{domain}/sources
GET /api/reports/{id}
GET /api/alerts
POST /api/alerts/{id}/ack
POST /api/alerts/{id}/resolve
POST /api/alerts/{id}/reopen
POST /api/admin/import-jobs/process-now
POST /api/admin/import-jobs/backlog
GET /api/admin/import-jobs
GET /api/admin/import-jobs/{id}

/health is public. Homepage API routes use bearer auth when enabled. Dashboard and admin/API management routes use Basic Auth.

Troubleshooting

• OPENAI_API_KEY is required: set it in .env. Do not use the test bypass in production.
• IMAP folder errors: confirm the folder is exactly named DMARC and exists for hello@tukutoi.com.
• No reports imported: check that messages contain valid DMARC aggregate XML attachments and that they are in the configured folder.
• Duplicate reports skipped: the raw XML SHA256 has already been imported.
• NPM cannot connect: confirm the npm_proxy Docker network exists and the container has 192.168.99.18.
• Dashboard login fails: confirm DASHBOARD_USERNAME and DASHBOARD_PASSWORD in .env.
• Homepage widget returns 401: confirm HOMEPAGE_API_TOKEN and the Authorization: Bearer ... header.
• Email alerts do not send: verify SMTP host, port, username, password, sender, and recipient env vars.

Development

Run tests:

DMARC_SENTINEL_ALLOW_NO_LLM_FOR_TESTS=true pytest

The XML parser uses defusedxml, ZIP extraction rejects path traversal and nested archives, and decompressed attachment sizes are capped by config.