Honest about AI detectors: what our heuristic measures, and what it doesn't

title: "Honest about AI detectors: what our heuristic measures, and what it doesn't" description: "Inksong shows a before-and-after AI-likelihood score. It's a heuristic — not a third-party guarantee. Here's exactly what's in it." date: "2026-05-09" tag: "Product" author: "Inksong"

Inksong shows a 0–100 score before and after humanization. We label it as a heuristic on every page where it appears. We want to explain what's actually in it — both because it's good product hygiene and because the AI-detection space is full of overclaims that haven't aged well.

What the score is

A weighted sum of these signals:

• Sentence-length variance (burstiness). AI-generated text trends toward uniform sentence lengths. Human writing varies more, especially across paragraph transitions. We measure the standard deviation of sentence-length-in-tokens across the document, normalized by document length.
• Token-level repetition. Certain bigrams ("It is", "Furthermore,", "In conclusion,") appear with characteristic frequency in default-mode LLM output. We don't blacklist them — they're perfectly fine in human writing — but their density in a passage is a signal.
• Hedge-word density. "Generally," "typically," "it can be argued that," "this suggests." LLMs over-hedge by default. Human writers hedge too, but with more variation in how they hedge (active subjects, specific authorities).
• Vocabulary entropy. How predictable is each next word given the previous few? AI text is on average more predictable than human text in the same domain. We approximate this with a small in-house n-gram model trained on a mixed corpus.
• Function-word distribution. The ratio of content-words to function-words ("the", "of", "in") behaves slightly differently in AI vs human writing across domains.

We sum these into a 0–100 number. Higher means more AI-like under our heuristic.

What the score is not

• It is not Turnitin. It's not GPTZero. We didn't run those services in a back-test and we don't claim parity. They use proprietary classifiers and they change all the time.
• It is not a guarantee that an academic-integrity tool won't flag your document. Detection tooling is fast-moving and noisy. If you're submitting work where a third-party detector is the gating mechanism, you should ultimately rewrite by hand and review.
• It is not a measure of writing quality. A well-written human academic paper can score moderately high on burstiness uniformity. A badly-written human email can score low. The score measures resemblance to default-mode AI generation, not whether the prose is good.
• It is not trained on your specific use case. We don't fine-tune the detector per user. Domain-aware detection is on the roadmap; right now the same heuristic applies whether you're writing legal opinions or marketing copy.

Why a heuristic at all?

Because the feedback loop matters. If you humanize a document and have no idea whether it moved the needle, you can't iterate. A small in-house score gives you that signal cheaply and instantly. We'd rather show an imperfect number with full disclosure than send you to a third-party tool that's also imperfect and charges for it.

When we have published comparisons against third-party detectors and a domain-aware version of the score, we'll write about it here.

How to read the number, honestly

• A drop from 85 → 25 is real signal that the document reads more naturally on our axes.
• That doesn't mean third-party detectors will agree.
• If you're optimizing for a specific detector, the right workflow is: humanize with Inksong, run your specific detector externally, iterate on the parts it flags.

If we ever start claiming Inksong scores guarantee third-party bypass, tell us to stop. We'd rather lose the sale than make a promise we can't keep.