Vectors & cosine similarity

Turning text into numbers

To find the chunks most similar to a question, we need to compare text mathematically. So we turn each chunk into a vector — a bag of numbers describing it. The honest toy version is bag-of-words: just count how often each word appears, ignoring order.

collections.Counter (a dict subclass you met in Stage A) does the counting for free. "the walker cancelled the walk" becomes {"the": 2, "walker": 1, "cancelled": 1, "walk": 1}. That dict is the vector — each word is a dimension, each count is that dimension's value.