diff --git a/README.md b/README.md index ab7be30..73d2675 100644 --- a/README.md +++ b/README.md @@ -39,6 +39,7 @@ Files are stored under sharded date/hash paths: ```text data/ index/memes.jsonl + index/tags.json memes/YYYY/MM/DD/aa/bb/. meta/YYYY/MM/DD/aa/bb/.json ``` @@ -60,7 +61,8 @@ Set `SITE_URL` in production so canonical and API discovery URLs use the public ## Public API -- `GET /api/memes?page=1&pageSize=12`: lists approved memes only. `pageSize` is capped at `48`. +- `GET /api/memes?page=1&pageSize=12`: lists approved memes with `caption` and `tags`. `pageSize` is capped at `48`. Add `q=angry%20cat` for ranked caption/topic search and `tag=cats` for a topic filter. Repeat `tag` to require all selected topics. +- `GET /api/tags`: lists the available managed tag names. - `GET /api/memes/`: returns public metadata for one approved meme. - `GET /meme/`: opens an approved meme in the viewer and provides its shareable permalink. - `GET /media/`: returns the normalized WebP image for one approved meme. @@ -70,6 +72,25 @@ API uploads use the exact same path as the browser form: persistent IP upload qu Public API reads are rate-limited per IP and return `429` with `Retry-After` when exceeded. List and metadata responses include `RateLimit-*` headers and a `rateLimit` object. +## Captions, tags, and administration + +The LLM moderation call also writes a factual searchable caption and chooses relevant tags from the admin-managed catalog. The server filters out invented tags. Uploaders cannot supply captions or tags; reads and search are public, while every metadata mutation requires the existing `X-Admin-Token` authentication. + +Search is lexical: case/accent-insensitive words, prefixes, and small spelling corrections, ranked by relevance then newest first. All query words and all selected tags must match. It does not use embeddings or infer semantic synonyms. Limits: 200 query characters, 20 tag filters, 200 managed tags, 60 characters per tag, and 2000 caption characters. + +Open `/admin/` to manage pending, published, or all images; edit captions and assigned tags; and add or remove topics. Tag names are lowercase and accept letters, numbers, spaces, underscores, and hyphens. Removing a tag clears its assignments from every image. + +Existing records load with empty captions/tags; no offline migration is needed. Define your topics, then use **GENERATE ALL MISSING CAPTIONS** to backfill all existing images. The browser processes images sequentially, saves each result immediately, and can stop after the current image. Re-running resumes missing captions. **GENERATE AI FOR SELECTED** can regenerate metadata for existing captions (for example after adding topics). Generation uses `OPENAI_API_KEY` and `OPENAI_MODERATION_MODEL`, incurs normal model usage, and leaves existing metadata and approval status intact on failure. Manual edits made while an AI request is running are preserved via a version check. Without an API key, uploads remain pending with empty metadata, and admins can still edit manually. + +Authenticated admin endpoints (not part of the public OpenAPI): + +- `GET /api/admin/memes?page=1&pageSize=12&status=all`: paginated image management; `status` accepts `pending`, `approved`, or `all`. Supports `q`, repeated `tag`, and `missingCaption=true`. +- `PATCH /api/admin/memes/` with `{ "caption": "...", "tags": ["cats"] }`: edit either or both metadata fields; unknown tags are rejected. +- `POST /api/admin/memes//generate`: generate caption and tags without re-moderating; optional `{ "onlyMissing": true }` skips already-captioned images. +- `GET /api/admin/tags`, `POST /api/admin/tags` with `{ "name": "cats" }`, and `DELETE /api/admin/tags/`: manage the catalog. + +Admin responses use `Cache-Control: no-store`. Unauthenticated admin API requests return `404`. + ## Docker ```sh diff --git a/public/admin.html b/public/admin.html index e7d3e82..1b9a630 100644 --- a/public/admin.html +++ b/public/admin.html @@ -3,16 +3,16 @@ - The Meme Protocol Review + The Meme Protocol Admin - - + + - +
-
THE_MEME_PROTOCOL_REVIEW
+
THE_MEME_PROTOCOL_ADMIN
@@ -25,7 +25,41 @@ PENDING_REVIEW 0 ITEMS
-
+
+ Manage tags +

AI chooses from these tags. Removing a tag also removes it from every image.

+
+ + + +
+
+
+
+ + + + + + + +
+
+ + + +

AI generation uses the configured model and can be edited below.

+
+
+ diff --git a/public/assets/admin.js b/public/assets/admin.js index cd64543..3344135 100644 --- a/public/assets/admin.js +++ b/public/assets/admin.js @@ -4,11 +4,70 @@ history.replaceState(null, '', '/admin'); const grid = document.querySelector('#review-grid'); const reviewStatus = document.querySelector('#review-status'); const reviewCount = document.querySelector('#review-count'); +const statusFilter = document.querySelector('#review-filter'); +const tagFilter = document.querySelector('#review-tag'); +const queryInput = document.querySelector('#review-query'); const selected = new Set(); +const dirty = new Set(); +let tags = []; +let page = 1; +let totalPages = 1; +let stopGeneration = false; +let busy = false; -document.querySelector('#refresh-review').addEventListener('click', loadPending); -document.querySelector('#approve-selected').addEventListener('click', () => moderateSelected('approve')); -document.querySelector('#delete-selected').addEventListener('click', () => moderateSelected('delete')); +function handle(action) { + return async (event) => { + if (event.type === 'submit') event.preventDefault(); + if (busy) return; + busy = true; + document.body.classList.add('admin-busy'); + try { await action(event); } + catch (error) { reviewStatus.textContent = error.message; } + finally { + busy = false; + document.body.classList.remove('admin-busy'); + freezeEditors(false); + document.querySelector('#stop-generation').hidden = true; + } + }; +} + +document.querySelector('#refresh-review').addEventListener('click', handle(async () => { + if (allowReload()) { await loadTags(); await loadMemes(); } +})); +document.querySelector('#approve-selected').addEventListener('click', handle(() => moderate([...selected], 'approve'))); +document.querySelector('#delete-selected').addEventListener('click', handle(() => moderate([...selected], 'delete'))); +document.querySelector('#generate-selected').addEventListener('click', handle(() => generateMany([...selected]))); +document.querySelector('#generate-missing').addEventListener('click', handle(backfill)); +document.querySelector('#stop-generation').addEventListener('click', () => { stopGeneration = true; }); +document.querySelector('#review-filters').addEventListener('submit', handle(async () => { + if (!allowReload()) return; + page = 1; + await loadMemes(); +})); +for (const [id, delta] of [['previous-page', -1], ['next-page', 1]]) { + document.querySelector(`#${id}`).addEventListener('click', handle(async () => { + if (!allowReload()) return; + page = Math.max(1, Math.min(totalPages, page + delta)); + await loadMemes(); + })); +} + +document.querySelector('#tag-form').addEventListener('submit', handle(async (event) => { + if (!allowReload()) return; + await request('/api/admin/tags', 'POST', { name: document.querySelector('#tag-name').value }); + event.target.reset(); + await loadTags(); + await loadMemes(); +})); +document.querySelector('#tag-catalog').addEventListener('click', handle(async (event) => { + const button = event.target.closest('[data-remove-tag]'); + if (!button || !allowReload()) return; + if (!confirm(`Remove “${button.dataset.removeTag}” from the catalog and every image?`)) return; + await request(`/api/admin/tags/${encodeURIComponent(button.dataset.removeTag)}`, 'DELETE'); + await loadTags(); + await loadMemes(); +})); grid.addEventListener('change', (event) => { const checkbox = event.target.closest('[data-select-id]'); @@ -16,101 +75,188 @@ grid.addEventListener('change', (event) => { if (checkbox.checked) selected.add(checkbox.dataset.selectId); else selected.delete(checkbox.dataset.selectId); }); - -grid.addEventListener('click', (event) => { +grid.addEventListener('input', (event) => { + const form = event.target.closest('[data-edit-id]'); + if (form) dirty.add(form.dataset.editId); +}); +grid.addEventListener('submit', handle(async (event) => { + const form = event.target.closest('[data-edit-id]'); + if (!form) return; + const id = form.dataset.editId; + const data = new FormData(form); + const { meme } = await request(`/api/admin/memes/${id}`, 'PATCH', { caption: data.get('caption'), tags: data.getAll('tags') }); + dirty.delete(id); + replaceCard(meme); + reviewStatus.textContent = 'METADATA_SAVED'; +})); +grid.addEventListener('click', handle(async (event) => { const action = event.target.closest('[data-admin-action]'); if (!action) return; - moderate([action.dataset.id], action.dataset.adminAction); + if (action.dataset.adminAction === 'generate') await generateMany([action.dataset.id]); + else await moderate([action.dataset.id], action.dataset.adminAction); +})); +window.addEventListener('beforeunload', (event) => { + if (dirty.size) { event.preventDefault(); event.returnValue = ''; } }); -await loadPending(); +await handle(async () => { await loadTags(); await loadMemes(); })(new Event('load')); -async function loadPending() { - selected.clear(); - reviewStatus.textContent = 'FETCHING_PENDING_QUEUE'; - const response = await fetch('/api/admin/pending', { headers: adminHeaders() }); - const payload = await response.json(); - if (!response.ok) { - reviewStatus.textContent = 'REVIEW_AUTH_FAILED'; - grid.innerHTML = '
ADMIN_TOKEN_INVALID
'; - return; - } - reviewStatus.textContent = 'PENDING_REVIEW'; - reviewCount.textContent = `${payload.total} ITEMS`; - renderPending(payload.memes); -} - -function renderPending(memes) { - if (memes.length === 0) { - grid.innerHTML = '
NO_PENDING_MEMES
'; - return; - } - - grid.replaceChildren(...memes.map((meme) => { - const article = document.createElement('article'); - article.className = 'meme-card'; - article.innerHTML = ` -
- - SCORE ${meme.moderationScore} -
-
- Pending meme ${shortId(meme.id)} -
-
-

${escapeText(meme.moderationReason || 'Queued for review.')}

-
-
-
- ${formatBytes(meme.byteSize)} -
-
- - -
-
- `; - return article; +async function loadTags() { + ({ tags } = await request('/api/admin/tags')); + const previous = tagFilter.value; + tagFilter.replaceChildren(new Option('All tags', ''), ...tags.map((tag) => new Option(tag, tag))); + tagFilter.value = tags.includes(previous) ? previous : ''; + const catalog = document.querySelector('#tag-catalog'); + catalog.replaceChildren(...tags.map((tag) => { + const item = document.createElement('span'); + item.className = 'admin-tag'; + const name = document.createElement('span'); + name.textContent = tag; + const button = document.createElement('button'); + button.type = 'button'; + button.className = 'text-action danger-action'; + button.textContent = 'REMOVE'; + button.dataset.removeTag = tag; + button.setAttribute('aria-label', `Remove tag ${tag}`); + item.append(name, button); + return item; })); + if (!tags.length) catalog.textContent = 'No tags yet. Add topics before generating AI metadata.'; } -async function moderateSelected(action) { - if (selected.size === 0) return; - await moderate([...selected], action); +async function loadMemes() { + selected.clear(); + dirty.clear(); + reviewStatus.textContent = 'LOADING_IMAGES'; + const params = new URLSearchParams({ page, pageSize: 12, status: statusFilter.value, q: queryInput.value }); + if (tagFilter.value) params.set('tag', tagFilter.value); + const payload = await request(`/api/admin/memes?${params}`); + page = payload.page; + totalPages = payload.totalPages; + reviewStatus.textContent = `${statusFilter.value.toUpperCase()}_IMAGES`; + reviewCount.textContent = `${payload.total} ITEMS`; + document.querySelector('#page-status').textContent = `PAGE ${page} OF ${totalPages}`; + document.querySelector('#previous-page').disabled = page <= 1; + document.querySelector('#next-page').disabled = page >= totalPages; + grid.replaceChildren(...payload.memes.map(renderCard)); + if (!payload.memes.length) grid.innerHTML = '
NO_MATCHING_MEMES
'; +} + +function renderCard(meme) { + const article = document.createElement('article'); + article.className = 'meme-card'; + article.dataset.memeId = meme.id; + article.innerHTML = ` +
+ + ${meme.status.toUpperCase()} · ${meme.moderationScore} +
+
${escapeText(meme.caption || `Meme ${shortId(meme.id)}`)}
+

${escapeText(meme.moderationReason || 'No moderation reason.')}

+
+ + +
Tags
+ +
+
+
${formatBytes(meme.byteSize)}
+
+ + ${meme.status === 'pending' ? `` : ''} + +
+
`; + const options = article.querySelector('.image-tag-options'); + for (const tag of tags) { + const label = document.createElement('label'); + const checkbox = document.createElement('input'); + checkbox.type = 'checkbox'; + checkbox.name = 'tags'; + checkbox.value = tag; + checkbox.checked = meme.tags.includes(tag); + label.append(checkbox, document.createTextNode(tag)); + options.append(label); + } + if (!tags.length) options.textContent = 'Add topics in Manage tags.'; + article.querySelector('[data-select-id]').checked = selected.has(meme.id); + if (busy) article.querySelectorAll('.metadata-editor input, .metadata-editor textarea').forEach((control) => { control.disabled = true; }); + return article; +} + +function replaceCard(meme) { + grid.querySelector(`[data-meme-id="${meme.id}"]`)?.replaceWith(renderCard(meme)); } async function moderate(ids, action) { - const endpoint = action === 'approve' ? '/api/admin/approve' : '/api/admin/delete'; - const response = await fetch(endpoint, { - method: 'POST', - headers: { ...adminHeaders(), 'Content-Type': 'application/json' }, - body: JSON.stringify({ ids }) - }); - if (!response.ok) { - reviewStatus.textContent = 'REVIEW_ACTION_FAILED'; - return; + if (!ids.length || !allowReload()) return; + if (action === 'delete' && !confirm(`Permanently delete ${ids.length} image(s)?`)) return; + await request(`/api/admin/${action}`, 'POST', { ids }); + await loadMemes(); +} + +async function generateMany(ids) { + if (!ids.length) return; + if (ids.some((id) => dirty.has(id)) && !confirm('AI generation will replace unsaved edits on these images. Continue?')) return; + stopGeneration = false; + freezeEditors(true); + document.querySelector('#stop-generation').hidden = false; + let done = 0; + for (const id of ids) { + if (stopGeneration) break; + reviewStatus.textContent = `GENERATING ${done + 1} OF ${ids.length}`; + const { meme } = await request(`/api/admin/memes/${id}/generate`, 'POST'); + dirty.delete(id); + replaceCard(meme); + done += 1; } - await loadPending(); + reviewStatus.textContent = `GENERATED ${done} OF ${ids.length}`; } -function adminHeaders() { - return { 'X-Admin-Token': token }; +async function backfill() { + if (!allowReload()) return; + stopGeneration = false; + freezeEditors(true); + document.querySelector('#stop-generation').hidden = false; + let done = 0; + // Read page one repeatedly as completed images leave the missing-caption set. + while (!stopGeneration) { + const batch = await request('/api/admin/memes?status=all&missingCaption=true&pageSize=48'); + if (!batch.memes.length) break; + for (const image of batch.memes) { + if (stopGeneration) break; + reviewStatus.textContent = `GENERATING MISSING CAPTIONS · ${done} DONE · ${batch.total} IN LAST BATCH`; + const { meme } = await request(`/api/admin/memes/${image.id}/generate`, 'POST', { onlyMissing: true }); + replaceCard(meme); + done += 1; + } + } + await loadMemes(); + reviewStatus.textContent = `GENERATED ${done} MISSING CAPTIONS${stopGeneration ? ' · STOPPED' : ''}`; } -function shortId(id) { - return `0x${id.slice(0, 4).toUpperCase()}...${id.slice(-4).toUpperCase()}`; +function freezeEditors(disabled) { + document.querySelectorAll('.metadata-editor input, .metadata-editor textarea').forEach((control) => { control.disabled = disabled; }); } -function formatBytes(value) { - if (value >= 1024 * 1024) return `${(value / 1024 / 1024).toFixed(1)}MB`; - return `${Math.ceil(value / 1024)}KB`; +function allowReload() { + return !dirty.size || confirm('Discard unsaved image metadata edits?'); } +async function request(url, method = 'GET', body) { + const response = await fetch(url, { + method, headers: { 'X-Admin-Token': token, ...(body ? { 'Content-Type': 'application/json' } : {}) }, + ...(body ? { body: JSON.stringify(body) } : {}) + }); + const payload = await response.json(); + if (!response.ok) throw new Error(payload.error || 'Admin request failed.'); + return payload; +} + +function shortId(id) { return `0x${id.slice(0, 4).toUpperCase()}...${id.slice(-4).toUpperCase()}`; } +function formatBytes(value) { return value >= 1024 * 1024 ? `${(value / 1024 / 1024).toFixed(1)}MB` : `${Math.ceil(value / 1024)}KB`; } function escapeText(value) { const span = document.createElement('span'); span.textContent = value; - return span.innerHTML; + return span.innerHTML.replaceAll('"', '"'); } diff --git a/public/assets/styles.css b/public/assets/styles.css index 275035f..149c898 100644 --- a/public/assets/styles.css +++ b/public/assets/styles.css @@ -1132,3 +1132,127 @@ dialog::backdrop { animation: none; } } + +/* Image metadata administration */ +.admin-toolbar { + display: flex; + flex-wrap: wrap; + align-items: center; + gap: 12px; + margin: 20px 0; + color: var(--outline); + font-size: 12px; +} + +.admin-toolbar input, +.admin-toolbar select, +.metadata-editor textarea { + background: var(--surface, #080808); + border: 1px solid var(--outline-variant); + color: var(--on-surface, #eee); + padding: 10px; + font: inherit; + max-width: 100%; +} + +.admin-toolbar input:focus, +.admin-toolbar select:focus, +.metadata-editor textarea:focus { + outline: 1px solid var(--primary); +} + +.admin-tag-manager { + border: 1px solid var(--outline-variant); + padding: 16px; + color: var(--outline); + font-size: 12px; +} + +.admin-tag-manager summary { + color: var(--primary); + cursor: pointer; +} + +.admin-tag-list, +.image-tag-options { + display: flex; + flex-wrap: wrap; + gap: 10px; +} + +.admin-tag { + display: inline-flex; + align-items: center; + gap: 12px; + border: 1px solid var(--outline-variant); + padding: 8px; +} + +.metadata-editor { + display: grid; + gap: 12px; + padding: 12px; + color: var(--outline); + font-size: 12px; +} + +.metadata-editor textarea { + width: 100%; + resize: vertical; + line-height: 1.5; +} + +.metadata-editor fieldset { + min-width: 0; + border: 1px solid var(--outline-variant); + padding: 10px; +} + +.image-tag-options { + max-height: 160px; + overflow-y: auto; +} + +.image-tag-options label { + display: flex; + align-items: center; + gap: 6px; +} + +.image-tag-options input { + accent-color: var(--primary); +} + +.admin-pagination { + justify-content: center; +} + +.admin-busy button:not(#stop-generation), +.admin-busy input, +.admin-busy textarea, +.admin-busy select { + pointer-events: none; + opacity: 0.65; +} + +.admin-toolbar [hidden] { + display: none; +} + +button:disabled { + opacity: 0.4; + cursor: default; +} + +.admin-page .topbar { + position: sticky; + height: auto; + min-height: 64px; + flex-wrap: wrap; + gap: 12px; + padding-block: 12px; +} + +.admin-page .shell { + padding-top: 24px; +}