Skip to content

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

6 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

 ███████╗██╗      ██████╗ ██████╗   ██████╗██╗  ██╗███████╗ ██████╗██╗  ██╗
 ██╔════╝██║     ██╔═══██╗██╔══██╗ ██╔════╝██║  ██║██╔════╝██╔════╝██║ ██╔╝
 ███████╗██║     ██║   ██║██████╔╝ ██║     ███████║█████╗  ██║     █████╔╝
 ╚════██║██║     ██║   ██║██╔═══╝  ██║     ██╔══██║██╔══╝  ██║     ██╔═██╗
 ███████║███████╗╚██████╔╝██║      ╚██████╗██║  ██║███████╗╚██████╗██║  ██╗
 ╚══════╝╚══════╝ ╚═════╝ ╚═╝       ╚═════╝╚═╝  ╚═╝╚══════╝ ╚═════╝╚═╝  ╚═╝

slopcheck

Catch AI slop by comparing a draft to your own writing instead of a list of banned words. It has no dependencies and runs on the Python standard library. You supply the corpus.

pip install git+https://github.com/vdruts/slop-check.git
slopcheck draft.md --authentic ./my-writing/

How it works

A list of banned words never holds up. The words change, your own writing gets flagged, and the model reaches for slop the list never had.

slopcheck measures a difference instead. You point it at a folder of your own writing, and it learns how your sentences move and which phrases you actually use. Then it reads a draft and flags two things: where the draft stopped sounding like you, and where it leans on phrases a machine repeats but you never write.

The rule underneath it: you repeating yourself is voice. A machine repeating itself in ways you don't is slop.

Bring your own corpus

slopcheck has no built-in idea of good writing. It only knows yours, learned when you run it.

slopcheck draft.md --authentic ./my-writing/

Point --authentic at real prose you wrote: voice-note transcripts, old posts, essays. A few thousand words is enough to start. Everything the tool treats as correct comes from those files and nothing else.

Keep that folder out of the repo. It is your writing and it is often personal. Pass it when you run the tool and add it to your gitignore. This repo already ignores corpus/, authentic/, and my-corpus/.

For the phrase-fingerprint check, add a second folder of text a machine wrote for you before:

slopcheck draft.md --authentic ./my-writing/ --comparison ./past-ai-drafts/

Leave off --comparison and you still get every style check against your baseline.

Install

There is no PyPI package. That name belongs to an unrelated project, so install from git or run from source.

Install from git:

pip install git+https://github.com/vdruts/slop-check.git

Run from source:

git clone https://github.com/vdruts/slop-check.git
cd slop-check
python -m slopcheck draft.md --authentic ./my-writing/

Needs Python 3.9 or newer, and nothing else.

Or install it as a Claude Code skill

This repo doubles as a Claude Code skill. SKILL.md teaches the agent to run the check, triage the flags, and do the rewriting itself, while slopcheck grades every round.

git clone https://github.com/vdruts/slop-check.git ~/.claude/skills/slopcheck

For one project instead of everywhere, clone into the project's .claude/skills/slopcheck. Then ask Claude to slopcheck a draft and tell it where your writing lives. The skill offers to save your corpus paths to the project's CLAUDE.md so you only explain that once.

Try the demo

The examples/ folder ships a fake corpus: a made-up gardener's writing plus some machine-written blog spam. Run it to see the difference before you plug in your own.

python -m slopcheck examples/candidate-clean.md \
  --authentic examples/authentic/ --comparison examples/comparison/

python -m slopcheck examples/candidate-slop.md \
  --authentic examples/authentic/ --comparison examples/comparison/

The clean draft passes with zero flags. The slopped one gets 24.

What it checks

Check Level Meaning
fingerprint FAIL A phrase that shows up across your AI pool but never in your real writing.
blocklist FAIL A generic cliché like "delve" or "move the needle". You can swap the list.
rhythm (staccato) FAIL Average sentence length dropped far below your baseline.
rhythm (uniform) WARN Sentence lengths barely vary. Real writing does.
fragment-wall FAIL Runs of tiny sentences well above your own rate. One run counts once, deliberate parallelism is exempt, and a punchy or dialogue-heavy corpus raises your allowance.
negation-preamble FAIL "Not X. Not Y." stacked for effect.
antithesis-density WARN / FAIL Negative-contrast frames ("X, not Y", "isn't X, it's Y", "not Y, or Z") used well above your own rate. A direct-response voice that uses them on purpose has a high baseline and passes; a corpus that never does flags the stuffing.
portentous-adverb WARN "quietly transforming", "seamlessly scaling".
punctuation WARN Exclamation marks above your own corpus rate. An exclamatory voice keeps its exclamation marks.

Every threshold has a default, and every one is configurable.

The fingerprint check

This is the part a banned-word list can't do. A list only knows the slop you already thought of. The fingerprint check finds the slop your own pipeline invented.

It takes every 3, 4, and 5 word run in the draft and keeps the ones that appear in several of your past AI drafts and in none of your real writing. What is left is phrases the machine likes and you don't. Nobody maintains that list. It rebuilds itself every time you drop another draft into the comparison folder.

Two guards stop false positives. Runs made only of glue words like "the of and to" are ignored. A comparison file that is more than 30% identical to the draft is treated as an earlier version of the same piece, so a post is not flagged against its own draft history.

Score

Every run returns a single voice score from 0 to 100. It starts at 100 and loses 12 points per FAIL and 4 per WARN, both configurable. A clean draft scores 100 and the demo slop scores 0. Use it when you want one number for a dashboard or a commit gate. It is also the signal the loop below optimizes.

Actor-critic loop

The checker only judges a draft. Pair it with a writer and it fixes one.

You supply the writer. It is any function that takes a draft plus feedback and returns a new draft, so the loop stays model-neutral and the core keeps its zero dependencies. Use whatever model you already have.

from slopcheck import refine

def rewrite(draft, feedback):
    # your model goes here: OpenAI, Anthropic, a local model, whatever you use.
    # feedback is the critic's flags turned into plain instructions.
    return call_your_model(draft, feedback)

out = refine(
    draft,
    authentic_paths=["./my-writing/"],
    rewrite_fn=rewrite,
    comparison_paths=["./past-ai-drafts/"],
    max_rounds=4,
)

print(out["best_score"], out["clean"])
print(out["best_text"])

Each round, slopcheck scores the draft, turns the flags into a plain instruction such as "remove the banned phrase here" or "three short sentences run together near this line", hands that to your writer, and scores the result. It stops when the draft is clean or it runs out of rounds, and it returns the best version it saw plus the history of every round. The writer is the actor, slopcheck is the critic, and you are wiring them into a loop that ends when the draft sounds like you.

Config

Defaults live in slopcheck.Config. Override them with a JSON file:

{
  "staccato_ratio": 0.6,
  "fingerprint_min_files": { "3": 4, "4": 2, "5": 2 },
  "variant_overlap": 0.35,
  "blocklist": ["delve", "circle back", "synergize"],
  "markers": ["you see", "here's the kicker"]
}
slopcheck draft.md --authentic ./my-writing/ --config slopcheck.json

blocklist replaces the built-in cliché list with your own. markers are phrases that belong to your voice; the tool counts how often they appear and never flags them. Add --raw for plain-text files you don't want markdown-stripped.

Python API

from slopcheck import check, Config

result = check(
    "draft.md",
    authentic_paths=["./my-writing/"],
    comparison_paths=["./past-ai-drafts/"],
    config=Config(staccato_ratio=0.6),
)

print(result["fails"], "failures")
for f in result["flags"]:
    print(f["level"], f["check"], f["detail"])

check() returns a plain dict, so it drops into CI or a pre-commit hook. The CLI exits 1 on any FAIL, 0 when clean, and 2 on a setup error.

Scoring many drafts? Build a Critic once and reuse it. The corpora are read and tokenized a single time, and every check after that is in-memory arithmetic. The refine loop does this internally.

from slopcheck import Critic

critic = Critic(["./my-writing/"], ["./past-ai-drafts/"])
for draft in drafts:
    print(critic.check_text(draft)["score"])

Validation

The repo carries its own controlled evaluation (eval/control_experiment.py) and publishes the results in VALIDATION.md, including the negative one: slopcheck catches slop patterns and voice drift, and it does not catch clean generic AI prose in a single draft. Read that file before trusting the tool with anything that matters. Rerunning the experiments takes one command.

Why not an AI humanizer

Most humanizers are an LLM grading its own writing. The tells it can't see are the ones it produces, so it can't catch them. slopcheck does the judging with arithmetic over two folders you control. It does not claim to know good writing in general. It knows what your writing looks like, and what your machine keeps doing that you don't.

License

MIT. See LICENSE.

About

Corpus-differential AI-slop detector: catch machine writing by differencing a draft against your own corpus, not a blocklist. Zero dependencies, pure stdlib.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages