107 lines
3.6 KiB
Markdown
107 lines
3.6 KiB
Markdown
# Runtime and Development Workflow
|
|
|
|
This file summarizes how the implemented repository runs and how developers can validate changes.
|
|
|
|
## Configuration Loading
|
|
|
|
`app/config.py` loads settings in this order:
|
|
|
|
1. `DMARC_SENTINEL_CONFIG`, when set;
|
|
2. `config/config.yml`.
|
|
|
|
If neither path exists, startup fails with an instruction to create `config/config.yml` from `config/config.example.yml`.
|
|
|
|
Secrets are read from environment variables named by the loaded settings. The settings page renders loaded values and environment-variable presence as read-only status; the application does not implement settings writes through the dashboard.
|
|
|
|
When `llm.provider` is `openai`, `validate_llm_environment()` requires the configured API key environment variable unless `DMARC_SENTINEL_ALLOW_NO_LLM_FOR_TESTS=true`.
|
|
|
|
## Local Test Workflow
|
|
|
|
Install dependencies in a Python environment, then run:
|
|
|
|
```bash
|
|
DMARC_SENTINEL_ALLOW_NO_LLM_FOR_TESTS=true python3 -m pytest
|
|
```
|
|
|
|
`tests/conftest.py` also sets test defaults for the LLM bypass, auth credentials, and homepage token.
|
|
|
|
There is no repository-local Ruff configuration or requirement in `requirements.txt`; if linting is added, document the exact command with the added tooling.
|
|
|
|
## Docker Runtime
|
|
|
|
The Docker image starts the FastAPI app with:
|
|
|
|
```bash
|
|
alembic upgrade head && uvicorn app.main:app --host 0.0.0.0 --port 8000
|
|
```
|
|
|
|
The compose service:
|
|
|
|
- loads environment variables from `.env`;
|
|
- mounts `./config` at `/app/config` as read-only;
|
|
- mounts `./data` at `/app/data`;
|
|
- mounts `./logs` at `/app/logs`;
|
|
- publishes `8000:8000`;
|
|
- attaches to the external `npm_proxy` network with static address `192.168.99.18`.
|
|
|
|
The app initializes database tables on import through `app.main` calling `init_db()`.
|
|
Alembic migrations are run first in Docker and are the preservation path for existing SQLite databases. `init_db()` remains as a lightweight fallback for missing tables in local/test workflows.
|
|
|
|
Manual migration command inside the container:
|
|
|
|
```bash
|
|
docker compose exec dmarc-sentinel alembic upgrade head
|
|
```
|
|
|
|
## CLI Backlog Processing
|
|
|
|
Backlog processing is implemented by `app/cli.py`:
|
|
|
|
```bash
|
|
python -m app.cli backlog --inbox <inbox-id>
|
|
```
|
|
|
|
Available arguments are:
|
|
|
|
- `--folder`;
|
|
- `--since YYYY-MM-DD`;
|
|
- `--before YYYY-MM-DD`;
|
|
- `--limit`;
|
|
- `--dry-run`;
|
|
- `--reprocess`;
|
|
- `--mark-seen`.
|
|
|
|
The CLI calls `process_inbox()` in backlog mode and prints the resulting `ProcessingSummary` counters.
|
|
|
|
## HTTP Processing Paths
|
|
|
|
`app/main.py` exposes admin endpoints that call the same processing pipeline:
|
|
|
|
- `POST /api/admin/import-jobs/process-now`: starts a queued process-now job using request fields from `ProcessNowRequest`.
|
|
- `POST /api/admin/import-jobs/backlog`: starts a queued backlog job using request fields from `BacklogRequest`.
|
|
- `GET /api/admin/import-jobs`: lists import jobs.
|
|
- `GET /api/admin/import-jobs/{id}`: returns import job progress.
|
|
|
|
These endpoints use dashboard Basic Auth dependencies.
|
|
|
|
## Scheduled Work
|
|
|
|
On FastAPI startup, `start_scheduler(settings)` registers:
|
|
|
|
- polling for all enabled inboxes every `settings.app.poll_interval_minutes`;
|
|
- daily summaries at 07:00 in `settings.app.timezone`;
|
|
- weekly summaries on Monday at 07:30 in `settings.app.timezone`.
|
|
|
|
Polling uses `process_inbox()` with `mode="new"`. Summary jobs aggregate stored records, call the LLM wrapper, store `LLMReport` rows, and send digest email when SMTP settings allow it.
|
|
|
|
## Lightweight Validation Commands
|
|
|
|
For documentation-only changes, useful checks are:
|
|
|
|
```bash
|
|
find doc -type f -maxdepth 1 -print
|
|
python3 -m pytest
|
|
```
|
|
|
|
The filename rule for this docs folder is: every Markdown file under `doc/` must be lowercase, except `README.md`.
|