Cosine Similarity
cosine similarity is used in computer science to group data, or to find data as a response to a query.
it is the cosine of the angle between two vectors.
if the angle is 90, cos(90) = 0 - not similar
if the angle is 0, cos(0) = 1 - same
if the angle is 180, cos(180) = -1 - opposite
To compare two sentences, you need to find the cosine similarity between the numerical representation of the two sentences. Embeddings are numerical representations of words or sentences. Embedding models are offered by OpenAI, Cohere, etc. Their apis take a sentence and return the embeddings.
Never miss a post from
Satyajeet Jadhav
Get notified when Satyajeet Jadhav publishes a new post.
Comments
Participate in the conversation.
Read More
Embeddings eli5 version
Computers don’t understand text like humans do. Computers understand numbers.To get computers to understand text, you need to convert the text to numbers.
Retrieval Augmented Generation (RAG) on Postgres
So I recently learnt that Postgres works quite well for RAG Applications. Here is the flow I followed.
Semantic Search, aka Magic
The related notes feature searches all your notes to find the ones that are closest in meaning to your current note.Searching notes to find text similar in meaning to your query is called semantic search. We are trying to build a semantic search engine.
The laws of psychology in marketing
A short story from the book, Alchemy:
Wisdom
Excerpt from“The varieties of scientific experience” by Carl Sagan.
Naming
He was sitting in front of a 3D printing machine that was reading a design and transforming it from bits to atoms. He suddenly got thoughtful.
LangChain for LLM Application Development
Langchain provides convenience wrappers over OpenAI models. It lets you wrap your prompts so that even the complex prompts can be reused. Parsers let you define the format of the output you want to extract from the OpenAI response.