Cosine Similarity for Text

01

What Does Cosine Similarity Measure?

Cosine similarity measures the angle between two vectors. When the vectors point in the same direction, the score is 1. When they have no shared positive dimensions, the score is 0. Text-frequency vectors are non-negative, so the calculator reports values in the 0–1 range.

Vector direction matters more than magnitude. A short document and a much longer document can receive a strong score when their terms occur in similar proportions. That makes cosine similarity more suitable than a raw dot product when document lengths differ.

02

The Cosine Similarity Formula

Let vectors A and B contain weights for the same ordered vocabulary. Their dot product is divided by the product of their Euclidean norms:

The dot product adds the products of corresponding term weights. Each norm is the square root of the sum of squared weights:

If either vector has zero length, the calculator returns 0 because there is no usable direction to compare. This can happen when an input contains no analyzable tokens after cleanup and stop-word filtering.

03

A Worked Cosine Similarity Example

Suppose the shared vocabulary is [cat, dog, sleeps]. After counting terms, two documents become:

The dot product is 2×1 + 0×1 + 1×1 = 3. The vector norms are √5 and √3. Therefore:

The documents overlap on “cat” and “sleeps,” but B also contains “dog” and distributes its weight differently. The result is substantial surface overlap without identical vectors.

04

Bag of Words and TF-IDF Can Produce Different Scores

In Bag of Words mode, the calculator uses normalized term frequencies. Every retained term contributes according to its share of the document. Common vocabulary can therefore dominate the overlap.

In TF-IDF mode, the same terms are reweighted using their document frequency across the pair. A term appearing in both documents receives the minimum IDF weight, while a term unique to one side receives more weight in that document. This often reduces the relative influence of shared generic words and changes both vector norms.

Read the TF-IDF formula guide for the exact smoothing rule used by the calculator.

05

Term Contributions Explain Where the Score Comes From

A scalar score hides which dimensions created the overlap. The calculator therefore returns the product Aᵢ × Bᵢ for every shared term. Terms with the largest products contribute most to the dot product.

Contribution rows are useful for QA. They can reveal that a strong score is driven by boilerplate, navigation labels, repeated product names, or genuinely central vocabulary. A high score backed by irrelevant template terms should not be interpreted the same way as one backed by the subject matter you intended to compare.

06

How to Interpret a Cosine Similarity Score

Zero means there is no shared positive-weight vocabulary under the selected settings. One means the vectors have the same direction, which can happen even when one document repeats every term more times than the other. Values between zero and one describe a continuum of vector overlap.

There is no universal threshold for “duplicate,” “related,” or “good.” A useful cutoff depends on document type, length, preprocessing, weighting mode, and the cost of false positives. Build thresholds from labeled examples in your own workflow rather than importing an arbitrary percentage.

07

Cosine Similarity Is Not Semantic Equivalence

BoW and TF-IDF vectors discard word order and do not understand synonyms, negation, argument structure, or factual meaning. “The treatment is safe” and “the treatment is not safe” share most of their words. Two paraphrases can use different vocabulary and appear less similar than they are semantically.

The score is not a plagiarism verdict, authorship detector, quality grade, or search-ranking predictor. Use it as an explainable lexical signal alongside contribution terms and human review.