01
Frequency Comparison Is Not a Character Diff
A line or character diff preserves position and shows exact insertions and deletions. It is ideal for code, contracts, and sentence-level editing. Word-frequency comparison discards position, groups matching terms, and measures how their shares changed.
Exact position and sequence
Finding literal edits
Counts and normalized shares
Finding vocabulary and emphasis changes
Vector direction
Summarizing weighted overlap
These methods can be used together. Start with a diff to see what moved, then use frequency changes to understand what became more or less prominent across the complete document.
02
A Six-Step Text Comparison Workflow
Define A and B
Use the original as A and the revision as B. Keep the direction consistent across reports.
Match preprocessing
Apply the same language, stop words, tokenization, and source extraction rules to both inputs.
Compare document metrics
Review token count and vocabulary size before interpreting individual term changes.
Sort normalized changes
Start with the largest absolute percentage-point changes, not only raw count changes.
Inspect context manually
Search the source for changed terms and decide whether the shift was intentional and useful.
Export the evidence
Save the full table when the comparison is part of editorial QA or a repeatable process.
03
Normalize When the Texts Have Different Lengths
Raw counts can mislead when B is longer than A. A term may appear more times while representing a smaller share of the revised text. Calculate each term's share inside its own document:
share(t, d) = count(t in d) / analyzed tokens in dThen subtract A from B and express the result in percentage points:
share change = (share in B − share in A) × 100 percentage pointsA positive result means the term occupies a larger share in B. A negative result means it occupies a smaller share. This does not declare the change good or bad.
04
A Worked Frequency-Change Example
Suppose “conversion” appears 6 times in a 600-word original and 8 times in a 1,200-word revision.
The raw count rose by two, but the normalized share fell by 0.33 percentage points. B contains more occurrences because it is longer, not because the term became more prominent.
05
Compare Individual Words and Consecutive Phrases
Unigram changes show which individual terms gained or lost share. Bigram changes preserve short local sequences and can reveal that a concept moved from one phrasing to another even when its component words stayed common.
Phrase tables need a window-based denominator. For a document with m tokens, there are m−1 bigram positions. Use the same n-gram size and filtering behavior on both sides. The N-gram guide explains this denominator in detail.
06
Interpret the Largest Changes in Source Context
Added vocabulary
A new term may represent useful specificity, a topic expansion, boilerplate, or an accidental repetition.
Removed vocabulary
A missing term may indicate tighter editing, lost detail, a renamed concept, or a changed purpose.
Vocabulary size
A larger vocabulary is not automatically better. Genre, length, names, and technical notation all affect it.
Zipf diagnostics
Distribution changes can flag unusual repetition, but they need a second signal such as relevance or contextual review.
For automated workflows, the public comparison API returns both analyses and the complete normalized change tables.
07
Vocabulary Change Does Not Measure Quality or Meaning
Frequency comparison does not judge factual accuracy, readability, semantic equivalence, tone, or search performance. Template extraction can also add navigation, footer, or cookie text when public URLs are compared.
Use the output to locate changes worth reviewing. Do not automatically restore every removed term, delete every repeated phrase, or select a “winner” based on one numeric direction.
The comparison tool intentionally reports evidence instead of assigning a quality score to A or B.
Related methods
TEXT OR URL · COMPLETE A/B EXPORT
Inspect what changed beyond the line diff
Compare normalized word and bigram frequency with one synchronized preprocessing configuration.
Open the text comparison tool →