In the last post, we explained how language models understand text by converting words into sets of numbers that represent subtle aspects of context and meaning. Now let's build on this knowledge by looking at a practical application of the technology.
For this post, let's assume we want to improve the search functionality on the website for the City of Zacville. Our site is using a traditional keyword search, which looks for specific terms within the pages of our site when the user enters a query.
Keyword search has many benefits! It's fast and can easily surface content that is an exact match for the search terms. But it doesn't handle ambiguity very well. If the user doesn't know exactly what they're trying to find (which is often the case when a resident visits your site), they can easily get tripped up or receive poor results. Misspellings, variations in terminology, and other slight discrepancies can ruin the experience.
Seeing the difference
The demo below shows how keyword search and semantic search handle the same queries differently. Step through the scenarios to see where keyword search succeeds, where it fails, and how semantic search bridges the gap.
How does it work?
Semantic search works by creating and storing embeddings for your searchable content. When the user submits a search query, you can convert that into an embedding as well, and find the searchable content that is the closest to the search query.
In this way, semantic search lives up to its name. You aren't searching by keyword phrase, but based on the context and meaning of both the search query and the underlying content to be searched!
Understanding the tradeoffs
While semantic search is powerful, there are definitely tradeoffs to consider.
It's slower. Semantic search requires first creating an embedding from the user's search query. This takes additional time. Your user interface must provide cues that it's working since it takes longer to perform a search this way.
You need a plan to keep your embeddings up-to-date. It's not enough to create embeddings of your content up front. You'll need a plan to ensure that any change of content triggers a process to update the embeddings as well.
It'll take some trial and error. While you can create an embedding from a large body of text, including an entire webpage, you'll often find better results by using a technique called chunking. Chunking involves breaking down a large body of text into smaller pieces so that the embedding you create captures more nuanced context and meaning. But choosing the right chunk size can be more of an art than a science.
It costs money to create embeddings. Unless you're using an open-source embedding model (or have trained your own), you'll have to pay a service like OpenAI to create embeddings. That's not to say it's expensive... you can probably create embeddings for your entire website for a couple of dollars. But since every search query has to be embedded, and content changes need to be re-embedded, it does represent an ongoing cost, however slight it might be.
Storing your embeddings. While keyword search (and other types of search like full-text) can be run directly on your content's database, embeddings need to be stored in a slightly different way. You might need to use a third-party Vector Database service, or if you have the in-house expertise, host your own vector database (such as ChromaDB or a PostgreSQL database with the pgvector extension installed).
Takeaways
Semantic search offers a great alternative to traditional search methods, and can significantly improve both the results you provide your users as well as the user experience they have while trying to find information. Using natural language to find information is much more intuitive than trying to guess the right keywords.
This technique can be particularly valuable in at least two other contexts that cities wrestle with frequently: searching through past agendas and minutes to find related information, and identifying documents that are responsive to open records requests.
There are also methods by which you can combine keyword and semantic search together to get the best of both worlds! It takes a little more effort, but it can be a very powerful combination.
If you care about making it easy for your residents, business owners, developers, and even employees to find information, including semantic search in your toolbox is something to consider!