Building a content-aware diff for humanized text

title: "Building a content-aware diff for humanized text" description: "A standard diff is line-oriented and useless for prose. Here's how we render before/after for documents users actually want to read." date: "2026-05-13" tag: "Engineering" author: "Inksong"

When the humanizer finishes, the user needs to verify the rewrite. Did the meaning survive? What actually changed, sentence by sentence? Where did the model intervene aggressively and where did it leave the source alone? A standard diff answers none of these questions well. Run a unified diff over a paragraph and you get every word marked as deleted-then-added, because almost every word's position has shifted. The result is a wall of red and green that obscures every change worth seeing.

We built a diff that respects how prose is actually read.

What users actually want to see

Two questions, in order.

The first is "does the rewrite preserve meaning?" That is answered at the sentence level. A reviewer wants to scan down the document and confirm that sentence N in the output is recognizably the same idea as sentence N in the input, even if the words are different.

The second is "what specifically changed?" That is answered at the word level, but only for sentences that survived as the same sentence. Showing word-level changes between sentences that are entirely different additions or deletions adds noise without information.

Both questions are answered by a diff that aligns on sentence boundaries first, groups changes by paragraph for navigation, and highlights word-level changes inline only within sentences that the algorithm has matched across the rewrite.

The algorithm

It's four steps.

Step 1: tokenize into sentences. We split both the input and the output into sentences using a regex that knows about abbreviations (Dr., e.g., i.e., et al.), citation patterns ((Smith, 2019), [1, 2]), and decimal numbers. Naive splitting on .!? breaks every academic document on the first parenthetical citation. The result is two ordered lists of sentences, one for the input and one for the output.

Step 2: align sentences greedily. We walk the input sentence list and try to match each one to a sentence in the output. The match function is Jaccard similarity over token sets — the size of the intersection divided by the size of the union, after lowercasing and stripping punctuation. Above a configurable threshold, the pair is considered "the same sentence, rewritten." The default threshold is 0.6, which catches typical paraphrase while rejecting genuinely different sentences. The walk is greedy and one-directional; we considered Hungarian assignment for global optimality but found greedy matching produced indistinguishable results on real documents at a fraction of the compute.

Step 3: word-level diff within aligned pairs. For each aligned pair, we run a standard word-level diff using the diff library and render the result inline. Words present only in the input show as deletions; words present only in the output show as additions; words shared between the two render as unchanged. The output is a single sentence with inline markup, easy to scan.

Step 4: render unaligned sentences as wholes. A sentence in the input with no match in the output is a full deletion. A sentence in the output with no match in the input is a full addition. These are rare on conservative rewrites and common on aggressive ones. We render them at sentence granularity, not word granularity.

The combined output is a paragraph-grouped, sentence-aligned, word-level-where-it-helps diff that a reviewer can actually read.

Why not just use diff-match-patch?

diff-match-patch is excellent and does word-level diffs better than we ever would. We use it under the hood inside step 3.

What it doesn't do is preserve the sentence-and-paragraph structure that humans care about when reviewing prose. Run it over an entire document and the view becomes a stream of fine-grained insertions and deletions with no respect for sentence boundaries. The reviewer ends up reading the same text three times — the original, the diff, and the rewrite — to figure out what changed. Sentence-aligned diffing surfaces the high-signal changes (a sentence that was rewritten, a sentence that was added or removed) and reserves word-level highlighting for the case where it's actually informative: a sentence that survived as the same sentence with edits.

The difference matters most on long documents. A 4,000-word document with a conservative rewrite has maybe fifteen sentences that materially changed and a hundred that shifted a word or two. The sentence-aligned view collapses the second category to a glance and surfaces the first category as the actual review surface.

Domain awareness

The alignment threshold is configurable per domain. Academic content uses 0.7 because we expect more conservative rewrites — citations, terminology, and structure constrain how much a sentence can shift. Lowering the threshold below 0.7 starts to false-match unrelated sentences that happen to share filler words.

Creative content uses 0.5 because rewrites are intentionally aggressive and the same idea can come out with very few shared tokens. A higher threshold would mark too many sentences as unaligned deletions-and-additions, hiding the fact that the rewrite is doing what the user asked for.

Other domains land between these two. The default of 0.6 is what we use when no domain is set.

Closer

The diff lives in /dashboard/documents/[jobId] next to the download buttons. It's quietly the part of the product we get the most positive feedback on, which surprised us — we expected the humanization itself to do the carrying. If the rendering side of this is interesting to anyone, we'll write a follow-up.