215 lines
6.2 KiB
Markdown
215 lines
6.2 KiB
Markdown
# LedgerDock
|
|
|
|
LedgerDock is a self-hosted document management system (DMS) for ingesting, processing, organizing, and searching files.
|
|
|
|
## Core Capabilities
|
|
|
|
- Drag and drop upload from anywhere in the UI
|
|
- File and folder upload with path preservation
|
|
- Asynchronous extraction and OCR for PDF, images, DOCX, XLSX, TXT, and ZIP
|
|
- Metadata and full-text search
|
|
- Routing suggestions based on previous decisions
|
|
- Original file download and extracted markdown export
|
|
|
|
## Technology Stack
|
|
|
|
- Backend: FastAPI, SQLAlchemy, RQ worker (`backend/`)
|
|
- Frontend: React, Vite, TypeScript (`frontend/`)
|
|
- Infrastructure: PostgreSQL, Redis, Typesense (`docker-compose.yml`)
|
|
|
|
## Runtime Services
|
|
|
|
The default `docker compose` stack includes:
|
|
|
|
- `frontend` - React UI (`http://localhost:5173`)
|
|
- `api` - FastAPI backend (`http://localhost:8000`, docs at `/docs`)
|
|
- `worker` - background processing jobs
|
|
- `db` - PostgreSQL (internal service network)
|
|
- `redis` - queue backend (internal service network)
|
|
- `typesense` - search index (internal service network)
|
|
|
|
## Requirements
|
|
|
|
- Docker Engine
|
|
- Docker Compose plugin
|
|
- Internet access for first-time image build
|
|
|
|
## Quick Start
|
|
|
|
From repository root:
|
|
|
|
```bash
|
|
docker compose up --build -d
|
|
```
|
|
|
|
Before first run, set required secrets and connection values in `.env` (or your shell):
|
|
|
|
- `POSTGRES_USER`
|
|
- `POSTGRES_PASSWORD`
|
|
- `POSTGRES_DB`
|
|
- `DATABASE_URL`
|
|
- `REDIS_PASSWORD`
|
|
- `REDIS_URL`
|
|
- `AUTH_BOOTSTRAP_ADMIN_USERNAME`
|
|
- `AUTH_BOOTSTRAP_ADMIN_PASSWORD`
|
|
- optional `AUTH_BOOTSTRAP_USER_USERNAME`
|
|
- optional `AUTH_BOOTSTRAP_USER_PASSWORD`
|
|
- `APP_SETTINGS_ENCRYPTION_KEY`
|
|
- `TYPESENSE_API_KEY`
|
|
|
|
Start from `.env.example` to avoid missing required variables.
|
|
|
|
Open:
|
|
|
|
- Frontend: `http://localhost:5173`
|
|
- API docs: `http://localhost:8000/docs`
|
|
- Health: `http://localhost:8000/api/v1/health`
|
|
|
|
Use bootstrap credentials (`AUTH_BOOTSTRAP_ADMIN_USERNAME` and `AUTH_BOOTSTRAP_ADMIN_PASSWORD`) to sign in from the frontend login screen.
|
|
|
|
Stop the stack:
|
|
|
|
```bash
|
|
docker compose down
|
|
```
|
|
|
|
## Security Must-Know Before Real User Deployment
|
|
|
|
This repository starts in a development-friendly mode. Before exposing it to real users or untrusted networks, verify these controls:
|
|
|
|
1. Environment mode and host binding:
|
|
- Set `APP_ENV=production`.
|
|
- Keep `HOST_BIND_IP=127.0.0.1` and publish through an HTTPS reverse proxy instead of broad host bind.
|
|
|
|
2. Bootstrap credentials:
|
|
- Replace all `AUTH_BOOTSTRAP_*` values with strong unique passwords before first public deployment.
|
|
- Disable optional bootstrap user credentials unless they are needed.
|
|
|
|
3. Processing log text persistence:
|
|
- Keep `PROCESSING_LOG_STORE_MODEL_IO_TEXT=false` and `PROCESSING_LOG_STORE_PAYLOAD_TEXT=false` unless temporary debugging is required.
|
|
- Enabling these values can store sensitive prompt, response, and payload text.
|
|
|
|
4. Provider outbound restrictions:
|
|
- Keep `PROVIDER_BASE_URL_ALLOW_HTTP=false` and `PROVIDER_BASE_URL_ALLOW_PRIVATE_NETWORK=false`.
|
|
- Set a strict `PROVIDER_BASE_URL_ALLOWLIST` containing only approved provider hosts.
|
|
|
|
5. Public URL and CORS posture:
|
|
- Use HTTPS in `PUBLIC_BASE_URL`.
|
|
- Restrict `CORS_ORIGINS` to exact production frontend origins only.
|
|
|
|
6. Redis transport security:
|
|
- For live deployments, use `REDIS_URL` with `rediss://`, set `REDIS_SECURITY_MODE=strict`, and set `REDIS_TLS_MODE=required`.
|
|
|
|
7. Development compose defaults:
|
|
- Review `.env.example` and `docker-compose.yml` security-related defaults before deployment.
|
|
- Do not promote development defaults unchanged into production.
|
|
|
|
## Common Operations
|
|
|
|
Start or rebuild:
|
|
|
|
```bash
|
|
docker compose up --build -d
|
|
```
|
|
|
|
Stop:
|
|
|
|
```bash
|
|
docker compose down
|
|
```
|
|
|
|
Tail logs:
|
|
|
|
```bash
|
|
docker compose logs -f
|
|
```
|
|
|
|
Tail API and worker logs only:
|
|
|
|
```bash
|
|
docker compose logs -f api worker
|
|
```
|
|
|
|
Reset all runtime data (destructive):
|
|
|
|
```bash
|
|
docker compose down -v
|
|
```
|
|
|
|
## Frontend-Only Local Workflow
|
|
|
|
If backend services are already running, you can run frontend tooling locally:
|
|
|
|
```bash
|
|
cd frontend && npm run dev
|
|
cd frontend && npm run build
|
|
cd frontend && npm run preview
|
|
```
|
|
|
|
`npm run preview` serves the built app on port `4173`.
|
|
|
|
## Configuration
|
|
|
|
Main runtime variables are defined in `docker-compose.yml`:
|
|
|
|
- API and worker: `DATABASE_URL`, `REDIS_URL`, `REDIS_SECURITY_MODE`, `REDIS_TLS_MODE`, `STORAGE_ROOT`, `PUBLIC_BASE_URL`, `CORS_ORIGINS`, `AUTH_BOOTSTRAP_*`, `PROCESSING_LOG_STORE_*`, `CONTENT_EXPORT_*`, `TYPESENSE_*`, `APP_SETTINGS_ENCRYPTION_KEY`
|
|
- Frontend: optional `VITE_API_BASE`
|
|
|
|
When `VITE_API_BASE` is unset, the frontend uses `http://<current-hostname>:8000/api/v1`.
|
|
|
|
Application settings saved from the UI persist at:
|
|
|
|
- `<STORAGE_ROOT>/settings.json` (inside the storage volume)
|
|
|
|
Provider API keys are persisted encrypted at rest (`api_key_encrypted`) and are no longer written as plaintext values.
|
|
|
|
Settings endpoints:
|
|
|
|
- `GET/PATCH /api/v1/settings`
|
|
- `POST /api/v1/settings/reset`
|
|
- `PATCH /api/v1/settings/handwriting`
|
|
- `POST /api/v1/processing/logs/trim` (admin only)
|
|
|
|
Auth endpoints:
|
|
|
|
- `POST /api/v1/auth/login`
|
|
- `GET /api/v1/auth/me`
|
|
- `POST /api/v1/auth/logout`
|
|
|
|
Detailed DEV and LIVE environment guidance, including HTTPS reverse-proxy deployment values, is documented in `doc/operations-and-configuration.md` and `.env.example`.
|
|
|
|
## Data Persistence
|
|
|
|
Docker named volumes used by the stack:
|
|
|
|
- `db-data`
|
|
- `redis-data`
|
|
- `dcm-storage`
|
|
- `typesense-data`
|
|
|
|
## Validation Checklist
|
|
|
|
After setup or config changes, verify:
|
|
|
|
- `GET /api/v1/health` returns `{"status":"ok"}`
|
|
- Upload and processing complete successfully
|
|
- Search returns expected results
|
|
- Preview and download work for uploaded documents
|
|
- `docker compose logs -f api worker` has no failures
|
|
|
|
## Repository Layout
|
|
|
|
- `backend/` - FastAPI API, services, models, worker
|
|
- `frontend/` - React application
|
|
- `doc/` - technical documentation for architecture, API, data model, and operations
|
|
- `docker-compose.yml` - local runtime topology
|
|
|
|
## Documentation Index
|
|
|
|
- `doc/README.md` - technical documentation entrypoint
|
|
- `doc/architecture-overview.md` - service and runtime architecture
|
|
- `doc/api-contract.md` - endpoint and payload contract
|
|
- `doc/data-model-reference.md` - persistence model reference
|
|
- `doc/operations-and-configuration.md` - runtime operations and configuration
|
|
- `doc/frontend-design-foundation.md` - frontend design rules
|