DMARC-Sentinel/dmarc-watcher/doc/runtime.md

Runtime and Development Workflow

This file summarizes how the implemented repository runs and how developers can validate changes.

Configuration Loading

app/config.py loads settings in this order:

1. DMARC_SENTINEL_CONFIG, when set;
2. config/config.yml, when present;
3. config/config.example.yml.

Secrets are read from environment variables named by the loaded settings. The settings page renders loaded values and environment-variable presence as read-only status; the application does not implement settings writes through the dashboard.

When llm.provider is openai, validate_llm_environment() requires the configured API key environment variable unless DMARC_SENTINEL_ALLOW_NO_LLM_FOR_TESTS=true.

Local Test Workflow

Install dependencies in a Python environment, then run:

DMARC_SENTINEL_ALLOW_NO_LLM_FOR_TESTS=true python3 -m pytest

tests/conftest.py also sets test defaults for the LLM bypass, auth credentials, and homepage token.

There is no repository-local Ruff configuration or requirement in requirements.txt; if linting is added, document the exact command with the added tooling.

Docker Runtime

The Docker image starts the FastAPI app with:

uvicorn app.main:app --host 0.0.0.0 --port 8000

The compose service:

• loads environment variables from .env;
• mounts ./config at /app/config as read-only;
• mounts ./data at /app/data;
• mounts ./logs at /app/logs;
• publishes 8000:8000;
• attaches to the external npm_proxy network with static address 192.168.99.18.

The app initializes database tables on import through app.main calling init_db().

CLI Backlog Processing

Backlog processing is implemented by app/cli.py:

python -m app.cli backlog --inbox <inbox-id>

Available arguments are:

• --folder;
• --since YYYY-MM-DD;
• --before YYYY-MM-DD;
• --limit;
• --dry-run;
• --reprocess;
• --mark-seen.

The CLI calls process_inbox() in backlog mode and prints the resulting ProcessingSummary counters.

HTTP Processing Paths

app/main.py exposes admin endpoints that call the same processing pipeline:

• POST /api/admin/process-now: processes the configured inbox using request fields from ProcessNowRequest.
• POST /api/admin/backlog: runs backlog processing using request fields from BacklogRequest.

Both endpoints use dashboard Basic Auth dependencies.

Scheduled Work

On FastAPI startup, start_scheduler(settings) registers:

• polling for all enabled inboxes every settings.app.poll_interval_minutes;
• daily summaries at 07:00 in settings.app.timezone;
• weekly summaries on Monday at 07:30 in settings.app.timezone.

Polling uses process_inbox() with mode="new". Summary jobs aggregate stored records, call the LLM wrapper, store LLMReport rows, and send digest email when SMTP settings allow it.

Lightweight Validation Commands

For documentation-only changes, useful checks are:

find doc -type f -maxdepth 1 -print
python3 -m pytest

The filename rule for this docs folder is: every Markdown file under doc/ must be lowercase, except README.md.