Nautilus/bitsforfree
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Tighten meme moderation quality gate

Browse Source
This commit is contained in:
smileBeda
2026-07-03 12:45:59 -03:00
parent d80d596664
commit 2baa814cbd
7 changed files with 102 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/moderation.js
+9 -4
View File
@@ -1,4 +1,5 @@
const DEFAULT_MODEL = process.env.OPENAI_MODERATION_MODEL || 'gpt-4o-mini';
const AUTO_APPROVE_MIN_SCORE = 80;
export async function moderateImage({ buffer, mime }) {
if (!process.env.OPENAI_API_KEY) {
@@ -64,11 +65,15 @@ function moderationPrompt() {
'Do not reject merely because a meme is rude, critical, weird, darkly humorous, or adult in tone.',
'Reject only if the image appears illegal or likely illegal to host or distribute, including child sexual content, sexual content involving minors, explicit non-consensual sexual content, bestiality, credible illegal activity instructions, terrorist or extremist recruitment, doxxing/private identity documents, or explicit threats that appear actionable.',
'If legality or age is ambiguous, choose pending.',
'Assign MEME_CONSENSUS_SCORE from 0-100: higher means it is clearly a meme/reaction/roast/remix and suitable for this site; lower means random photo, spam, ad, QR scam, screenshot dump, or unclear.',
'Quality matters. Do not auto-approve low-quality, generic, or AI-slop memes; send them to pending for admin review.',
'Low-quality memes usually have generic stock meme templates with large caption text, captions that could fit dozens of unrelated images, jokes that rely almost entirely on text, images that add little or nothing, literal descriptions instead of punchlines, excessive text that obscures the image, no clear comedic payoff, or a humorless generated feel.',
'High-quality memes usually have strong visual composition, a joke contained in or strengthened by the image, visual storytelling, an unexpected punchline, original artwork or a meaningful remix, and a caption/image pairing where the joke would lose meaning if the image changed.',
'Before deciding, ask: is there an actual punchline; could the exact same caption work on 50 unrelated images; does the image meaningfully contribute; is the text overwhelming the artwork; does it feel like a specific joke someone wanted to make; would users share it because it is clever rather than merely because it exists?',
'Assign MEME_CONSENSUS_SCORE from 0-100: higher means it is legal, clearly a meme/reaction/roast/remix, visually meaningful, has a real punchline, and is suitable for this site; lower means random photo, spam, ad, QR scam, screenshot dump, unclear, generic template caption, text-only joke, weak payoff, or AI-slop.',
'Return only compact JSON with keys: decision, score, reason.',
'decision must be one of: approved, pending, rejected.',
'Use approved only when it appears legal and score is at least 65.',
'Use pending for ambiguity, uncertainty, low meme relevance, or anything that needs human review.',
`Use approved only when it appears legal and score is at least ${AUTO_APPROVE_MIN_SCORE}.`,
'Use pending for ambiguity, uncertainty, low meme relevance, weak quality, generic/slop memes, or anything that needs human review.',
'Use rejected only for likely illegal content.'
].join('\n');
}
@@ -95,7 +100,7 @@ function normalizeDecision(raw) {
? raw.reason.trim().slice(0, 300)
: 'No moderation reason supplied.';
if (decision === 'approved' && score < 65) {
if (decision === 'approved' && score < AUTO_APPROVE_MIN_SCORE) {
return { status: 'pending', score, reason: `${reason} Low MEME_CONSENSUS_SCORE queued for review.` };
}
return { status: decision, score, reason };