[AI] Exploring RAGs. Creating a Chat over custom Data

How to create a Python Apps to chat with your data as context?

How to use RAGs

Previously this year, I got lucky enough to find these open source projects:

They both use LangChain as RAG framework

And they were great to give me an understanding oon what a RAG is and how to build one.

We can build very interesting QnA over knowledge apps: https://github.com/langchain-ai/chat-langchain

General RAG Architecture

This is the general idea of a RAG architecture:

flowchart TD
    A[User Query] -->|Send Query| B[Retrieval Module]

    subgraph RAG_Framework[ ]
        B -->|Retrieve Documents| C[Knowledge Base]
        C -->|Relevant Documents| B
        B -->|Provide Context| D[Generation Module]
        D -->|Generate Response| E[Final Response]
    end
    
    E -->|Send Back| F[User Response]

    style RAG_Framework fill:#f9f9f9,stroke:#333,stroke-width:2px
    style A fill:#FFDDC1
    style E fill:#85E3FF
    style F fill:#85E3FF

    %% Annotations
    classDef query fill:#FFDDC1,stroke:#333,stroke-width:1px;
    classDef retrieval fill:#FFABAB,stroke:#333,stroke-width:1px;
    classDef knowledgeBase fill:#FFC3A0,stroke:#333,stroke-width:1px;
    classDef generation fill:#D5AAFF,stroke:#333,stroke-width:1px;
    classDef response fill:#85E3FF,stroke:#333,stroke-width:1px;

And as you can imagine, there as few frameworks already out there.

RAG Frameworks

As you can imagine, there are few frameworks available to implement a RAG with Python:

Llama-Index

You might Know Llama-Index because of its RAG capabilities.

• https://github.com/run-llama/llama_index
  • https://docs.llamaindex.ai/en/stable/

LlamaIndex is a framework for building context-augmented generative AI applications with LLMs including agents and workflows.

ChatBot for Real Estate - LlamaIndex

LLamaIndex is awsome.

And for a real estate agent bot, LlamaIndex + Mem0 does the trick.

How Exactly?

See this repo folder.

You will need OpenAI & Anthropic APIs

LLamaIndex Use Cases

But LLamaIndex can do much more than that.

For some reason it is the RAG framework used at the PrivateGPT project.

For example, we can use LlamaIndex Pandas Query Engine to chat with Pandas DF’s as well

Exploring LangChain

The LangChain framework is amazing.

• https://pypi.org/project/langchain/
• https://python.langchain.com/v0.2/docs/introduction/

It can helpful to:

1. Chat with PDFs
2. Even with CSV’s…
3. …or a Database!

4. There is another one! - Langchain with Pandas DF

LangChain PandasDF Chat

With LangChain, we can create an agent to chat with a Pandas DF.

It will actually create internal prompts, so that the agent will create the python pandas queries so that you get the data you wanted feed to the LLM reply.

I have explored LangChain + Pandas DF at this post and in this DataChat repo folder

Exploring PandasAI

I was using the PandasAI project previously to talk with dataframes as covered on this Post

• https://pypi.org/project/pandasai/
• https://platform.openai.com/

HayStack as RAG Framework

The Haystack framework is completely new to me.

pip install haystack-ai

EmbedChain - Mem0

It seems that the embedchain project got absorbed into a bigger one.

Im talking about the mem0 framework.

#!pip install embedchain==0.1.0

Try embedchain and see how it gets a website content as reference:

import os
os.environ['OPENAI_API_KEY'] = 'sk-youropenaikey'

from embedchain.store.assistants import OpenAIAssistant

assistant = OpenAIAssistant(name="OpenAI DevDay Assistant", instructions="You are an organizer of the OpenAI DevDay Conference")

# Option 2: You can also configure more things by instantiating like this
# assistant = OpenAIAssistant(
#     name="OpenAI DevDay Assistant",
#     instructions="You are an organizer of the OpenAI DevDay Conference",
#     tools=[],
#     data_sources=[{"source": "https://www.youtube.com/watch?v=U9mJuUkhUzk", "data_type": "youtube_video"}]
# )

# Option 3: You can load an existing assistant by using assistant id
# assistant = OpenAIAssistant(assistant_id="xxxx")

assistant.add("https://openai.com/blog/new-models-and-developer-products-announced-at-devday")

while(True):
  question = input("❓ Enter question: ")
  if question in ["exit", "quit", "q"]:
    print("Exiting....")
    break
  answer = assistant.chat(question)
  print("💡 Assistant: ", answer)
  print()

Summing Up

We have seen some interesting RAG Frameworks working in Python

Whats next from here?

Why not building something cool?

Interesting API keys for LLMs

Other LLMs that I have not covered yet in posts

• Mistral AI
• https://openrouter.ai/modelsOpenRouter

You can always use Ollama!

LLMs that have already appeared:

• Anthropic API - https://console.anthropic.com/workbench/
• GROQ API - https://console.groq.com/keys
• OpenAI API - https://platform.openai.com/api-keys - This one has been handy to try those RAG projects

FAQ

More Github Actions CI/CD

1. Setup GHA CI/CD as per this guide and see use cases
2. https://jalcocert.github.io/JAlcocerT/create-streamlit-chatgpt/#conclusion---and-what-i-learnt

• https://github.com/nektos/act

#https://github.com/nektos/act/releases/tag/v0.2.70

wget https://github.com/nektos/act/releases/download/v0.2.70/act_Linux_x86_64.tar.gz

tar -xzf act_Linux_x86_64.tar.gz
sudo mv act /usr/local/bin/
sudo chmod +x /usr/local/bin/act

act --version

Then go to the repo folder (where ./github/workflows are)

act

Running LLMs Locally

• GPT4ALL Guide
  • https://github.com/nomic-ai/gpt4all

Interesting RAG Resoures

VectorDBs

Vector Admin to manage them with UI.

Qdrant or ChromaDB can be deployed with Containers.