Semantic Search Engine Using Generative AI

With the advent of Large Language Model (LLM) like ChatGPT, semantic search has received much attention in recent time. This project takes advantage of LLM to help build semantic search engine which finds wide spread applications across industries. It starts with chunking the documents and creating vector database using Amazon Titan text embedding model. When the user wants to search a specifc information from the document(s), it fetches the information that are semantically closer to the user’s query and displays top 3 answers based on similarity score. An User Interface (UI) is developed using Gradio which helps user to interact with search query and receive response from the LLM.

Salient Features:
– Simple and intuitive application
– Using Langchain as an orchestration framework
– Leveraging open source tools like LanceDB as a vectorDB
– Titan text embedding model for larger context length
– Using external data as a knowledge base
– Gradio as an UI
– Personalized experience

Tech Stacks:
Deep Learning Frameworks, Text embeddings Model, Chat Model, Vector database, Gradio, Python

Industry:
Education, Legal, Healthcare