How To Build An AI Filter That Detects AI-generated Content In Your Own Drafts Before Hitting Publish

Writers, editors, educators, and content professionals increasingly face a quiet dilemma: the very tools meant to accelerate their work—LLMs like ChatGPT, Claude, or Gemini—can erode authenticity, nuance, and voice if used uncritically. Worse, many don’t realize how much of their final draft has been smoothed, padded, or rephrased by AI—until it’s too late. Plagiarism checkers won’t catch this. Grammar tools won’t flag it. And commercial AI detectors? They’re often inaccurate, opaque, and ethically fraught—trained on scraped data, prone to bias against non-native English, and incapable of distinguishing *your* voice from *your assistant’s*. The solution isn’t avoidance—it’s agency. This guide shows you how to build a lightweight, transparent, self-contained AI filter you run locally, inside your writing environment, before you ever click “Publish.” No cloud uploads. No subscription. Just code you understand, customize, and trust.

Why off-the-shelf AI detectors fail writers—and why you need your own

Most publicly available AI detection tools operate as black boxes. You paste text, get a percentage, and walk away—none the wiser about *why* it scored that way. Research from Stanford and the University of Maryland confirms consistent over-detection of human-written text by tools like Originality.ai and GPTZero, especially for technical, academic, or non-idiomatic English. A 2023 study found false positive rates exceeding 65% for essays written by ESL students. Worse, these services often retain submitted text for model training—a serious concern for draft confidentiality.

What writers actually need is not a verdict—but a diagnostic lens. One that highlights patterns *you* care about: excessive hedging (“it could be argued that…”), passive voice density, lexical repetition suppression, or syntactic uniformity. These aren’t proof of AI use—but they’re signals worth reviewing. Your filter should act like a thoughtful editor, not a courtroom judge.

“Detection isn’t about labeling text—it’s about surfacing stylistic choices so the writer can decide what serves their voice and intent.” — Dr. Maya Lin, Computational Linguist & Editor of Journal of Digital Rhetoric

Core principles: What makes a responsible, writer-centric AI filter

A trustworthy filter for your own drafts rests on three non-negotiable foundations:

• Local execution: All analysis happens on your machine—no text leaves your device.
• Interpretable logic: Every score or highlight maps to a concrete linguistic feature you can verify (e.g., “This sentence has 4 consecutive clauses with identical subject-verb-object structure”).
• Calibrated thresholds: It doesn’t declare “AI” or “human”—it flags passages where stylistic patterns deviate significantly from *your* baseline (established from your prior published work).

This shifts the goal from classification to reflection. You’re not outsourcing judgment—you’re augmenting self-awareness.

Step-by-step: Building your personal AI filter in under 90 minutes

You don’t need a data science degree. You need Python (3.9+), a text editor, and ~15 minutes to install dependencies. We’ll use textblob, spacy, and scikit-learn—all open-source, well-documented, and lightweight.

1. Install core tools: Run pip install spacy textblob scikit-learn numpy pandas. Then download the English model: python -m spacy download en_core_web_sm.
2. Create your voice baseline: Gather 3–5 recent published pieces you consider authentically “yours” (minimum 500 words each). Save them as my_voice_1.txt, my_voice_2.txt, etc. These become your stylistic anchor.
3. Extract your signature metrics: Run this script once to compute your personal baselines:

import spacy
from textblob import TextBlob
import numpy as np

nlp = spacy.load(\"en_core_web_sm\")

def analyze_text(text):
    doc = nlp(text.lower())
    blob = TextBlob(text)
    
    # Your unique metrics
    avg_sentence_length = np.mean([len(sent) for sent in doc.sents])
    passive_ratio = sum(1 for token in doc if token.dep_ == \"nsubjpass\") / len(doc) if len(doc) > 0 else 0
    hedge_density = len([word for word in blob.words if word.lower() in [\"perhaps\", \"might\", \"could\", \"may\", \"seems\"]]) / len(blob.words) if blob.words else 0
    conjunction_rate = len([token for token in doc if token.pos_ == \"CCONJ\"]) / len(doc) if len(doc) > 0 else 0
    
    return {
        \"avg_sentence_length\": avg_sentence_length,
        \"passive_ratio\": passive_ratio,
        \"hedge_density\": hedge_density,
        \"conjunction_rate\": conjunction_rate
    }

# Analyze your baseline texts
baselines = []
for i in range(1, 6):
    try:
        with open(f\"my_voice_{i}.txt\", \"r\", encoding=\"utf-8\") as f:
            baselines.append(analyze_text(f.read()))
    except FileNotFoundError:
        continue

# Compute averages
if baselines:
    baseline_avg = {
        k: np.mean([d[k] for d in baselines])
        for k in baselines[0].keys()
    }
    print(\"Your voice baseline:\", baseline_avg)

4. Build the real-time checker: Save this as ai_filter.py:

def flag_ai_patterns(text, baseline, threshold=0.35):
    doc = nlp(text.lower())
    blob = TextBlob(text)
    current = analyze_text(text)
    
    alerts = []
    
    # Flag sentences where passive ratio exceeds baseline by >40%
    if current[\"passive_ratio\"] > baseline[\"passive_ratio\"] * 1.4:
        alerts.append(f\"⚠️ High passive voice ({current['passive_ratio']:.2%} vs your norm of {baseline['passive_ratio']:.2%})\")
    
    # Flag hedge density > 2x baseline
    if current[\"hedge_density\"] > baseline[\"hedge_density\"] * 2:
        alerts.append(f\"⚠️ Elevated hedging language ({current['hedge_density']:.2%} vs {baseline['hedge_density']:.2%})\")
    
    # Flag sentence length uniformity (low variance = robotic rhythm)
    sentence_lengths = [len(sent) for sent in doc.sents]
    if len(sentence_lengths) > 3 and np.std(sentence_lengths) < 3.5:
        alerts.append(\"⚠️ Unusually uniform sentence lengths—consider varying rhythm\")
    
    return alerts

# Usage: 
# alerts = flag_ai_patterns(your_draft_text, baseline_avg)
# if alerts: print(\"\
\".join(alerts))

5. Integrate into your workflow: Paste your draft into a Python script, Jupyter notebook, or even a VS Code Python terminal. Call flag_ai_patterns(draft_text, baseline_avg). It returns plain-English observations—not scores.

Your AI filter in action: A real-world case study

Sarah K., a senior technical writer at a cybersecurity firm, noticed her quarterly reports felt “flatter” after adopting LLM-assisted drafting. She’d rewrite complex threat analyses with ChatGPT for clarity—then publish without reviewing the edits. After two reports were flagged internally for “uncharacteristic vagueness,” she built her own filter using this method.

Her baseline came from 4 published white papers (2022–2023). Her filter immediately highlighted three patterns in her latest draft: passive voice was up 72% (vs her 12% baseline), hedge density tripled (“could potentially indicate,” “may suggest,” “might imply”), and 83% of sentences were 18–22 words long—versus her usual 12–38 word range. She revised only the flagged sections: converting passive constructions, replacing hedges with confident assertions (“indicates” instead of “could indicate”), and breaking up monotonous clauses. The final version retained technical precision but regained her authoritative, direct tone. Her manager noted the improvement before Sarah even mentioned the change.

Do’s and Don’ts of AI-aware writing

FAQ

Can this detect all AI-generated text?

No—and it shouldn’t try. Modern LLMs generate highly variable output. Instead of chasing detection, this tool helps you maintain *intentionality*. If your draft consistently triggers multiple flags, it’s time to re-engage with the material—not to “pass a test.”

What if I write in multiple genres (e.g., poetry + technical docs)?

Create separate baselines. Save poetry_baseline.json and technical_baseline.json, then load the appropriate one before checking. Voice is contextual—not universal.

Does this work with non-English drafts?

Yes—with adaptation. Replace en_core_web_sm with a spaCy model for your language (e.g., fr_core_news_sm for French). Adjust hedge words and syntactic patterns accordingly. The architecture remains identical.

Conclusion: Reclaim authorship, one draft at a time

AI isn’t the problem—uncritical dependence is. The most powerful AI filter isn’t a product you buy. It’s the habit of asking, *“Did I say this—or did the model say it for me?”* Building your own lightweight detector does more than flag patterns. It trains your attention. It reinforces that voice isn’t accidental—it’s cultivated, protected, and chosen. Every time you pause to examine a flagged sentence, you’re exercising the very skill AI cannot replicate: authorial intention. You decide when uniformity serves clarity, when hedging serves ethics, and when passive voice serves emphasis. That discernment—the quiet, daily practice of owning your words—is the ultimate filter. Start small: run the baseline script tonight. Review one draft tomorrow. Notice what changes—not in the text, but in your relationship to it.