How to do live search/research with AI

TL;DR

Setting up Local Deep Researcher and Perplexica

Intro

Last year I had the Groq YT project as a very useful one.

And the notebookLM as a kind of RAG from google, that could be useful to summarize docs.

OpenLTables

Using notebookLM to understand new syntax

Today, notebookLM can summary not only PDFs, but it also sumarizes youtube videos (given a link) and…it can provide even a audio summary of it / podcast like summary.

Which is crazy.

But how about other ways to interact with real/live data for LLM context?

Plus the possibility to miss-out on Web Dynamic content - like gh discussions

Live Search with LLMs

Recently, xAI added live search capabilities to their models.

Goodbye knowledge cuttof! or so they say

But actually, there has been couple of tools that allow us to interact with Web information.

WindSurf IDE

Agents inside your IDE

Grok Live Search

Few weeks ago I was tinkering with grok API, which can be used also via LiteLLM.

What you need to have handy:

• https://x.ai/api
  • https://console.x.ai/
  • https://docs.x.ai/docs/overview
  • https://docs.x.ai/docs/models
  • https://github.com/JAlcocerT/Streamlit-MultiChat/tree/main/Z_Tests/GrokAPI 🐍

Data Chat Repository

Source Code for DB Chat with Langchain

Local Deep Researcher

Ollama x DDG Search x LangGraph

Local Deep Researcher Repo

Local Deep Researcher with langGraph - Source Code on Github

Local Deep Researcher

Its all about the magic of langgraph.

Forked Local Deep Researcher

Source Code that brings OpenAI/Groq options

git clone https://github.com/JAlcocerT/local-deep-researcher


docker-compose -f ./local-deep-researcher/Z_DeployMe/docker-compose.yml up -d
# docker exec -it ollama sh
# ollama pull deepseek-r1:8b

cd local-deep-researcher
cp .env.example .env #by default ollama x DDG as search engine
uvx \
                 --refresh \
                 --from "langgraph-cli[inmem]" \
                 --with-editable . \
                 --python 3.11 \
                 langgraph dev

#https://langchain-ai.github.io/langgraph/cloud/reference/cli/#dev
uvx \
  --refresh \
  --from "langgraph-cli[inmem]" \
  --with-editable . \
  --python 3.11 \
  langgraph dev --host 0.0.0.0 --port 8011 # <-- Added host and port



uvx \
  --refresh \
  --from "langgraph-cli[inmem]" \
  --with-editable . \
  --python 3.11 \
  langgraph dev --host 192.168.1.8 --port 8011

See how:

1. Your search gets translated into a search query: how can i selfhost raneto with docker-compose?

2. Results are loaded from DDG: duckduckgo search is used

Perplexity

Sometime ago I used Whoogle and SearXNG to make private Web Search.

• https://fossengineer.com/selfhosting-whoogle-docker/
• SearXNG: https://fossengineer.com/selfhosting-whoogle-docker/#searxng-with-docker

But today, GenAI can help us to retrieve relevant Web results: https://www.perplexity.ai

• https://docs.perplexity.ai/getting-started/overview

It’s all about real time information, right?

EDIT Coming from the future:

And…Perplexity offers an API:

curl https://api.perplexity.ai/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "sonar-pro",
    "messages": [
      {
        "role": "user", 
        "content": "What are the major AI developments and announcements from today across the tech industry?"
      }
    ]
  }' | jq

The Perplexica AI Project

Perplexica is an AI-powered search engine.

It is an Open source alternative to Perplexity AI

• Project Source Code at Github
• License MIT ❤️

The AI-powered search engine ✅ An Alternative to Perplexity AI

It uses advanced machine learning algorithms like similarity searching and embeddings to refine results and provides clear answers with sources cited.

Using SearxNG to stay current and fully open source, Perplexica ensures you always get the most up-to-date information without compromising your privacy.

Why Perplexica AI?

• All Mode: Searches the entire web to find the best results.
• Writing Assistant Mode: Helpful for writing tasks that does not require searching the web.
• Academic Search Mode: Finds articles and papers, ideal for academic research.
• YouTube Search Mode: Finds YouTube videos based on the search query.
• Wolfram Alpha Search Mode: Answers queries that need calculations or data analysis using Wolfram Alpha.
• Reddit Search Mode: Searches Reddit for discussions and opinions related to the query.

SelfHosting Perplexica AI

git clone https://github.com/ItzCrazyKns/Perplexica.git
cd Perplexica

cp sample.config.toml config.toml #Rename the sample.config.toml file to config.toml
docker-compose up -d

conda create -n px python=3.11 -y && conda activate px

pip install torch transformers accelerate huggingface_hub sentencepiece 

git clone https://github.com/searxng/searxng && cd searxng

nano ./searx/settings.yml #add search format json
make docker.build   

docker run --rm -d -p 32768:8080 -v "${PWD}/searxng:/etc/searxng" -e "BASE_URL=http://localhost:$PORT/" -e "INSTANCE_NAME=my-instance" searxng/searxng

#ollama
curl -fsSL https://ollama.com/install.sh | sh
ollama pull llama3
ollama pull bgesmall

#perplexica
git clone https://github.com/ItzCrazyKns/Perplexica.git && cd Perplexica
docker-compose up -d

FAQ

Scrap the Web with AI

• https://github.com/VinciGit00/Scrapegraph-ai

Python scraper based on AI

F/OSS Search Engines

• Whoogle
• SearXNG
• LibreX / Y - https://github.com/hnhx/librex
• WebSurfX - https://github.com/neon-mmd/websurfx https://github.com/neon-mmd/websurfx/blob/rolling/docs/installation.md

🚀 An open source alternative to searx which provides a modern-looking ✨, lightning-fast ⚡, privacy respecting 🥸, secure 🔒 meta search engine

• https://github.com/gigablast/open-source-search-engine

• https://github.com/yacy/yacy_search_server

Distributed Peer-to-Peer Web Search Engine and Intranet Search Appliance

More Search Engines

• Brave Search (Independent Search)
• Ecosia (Bing Based)
• DuckDuckGo - This can be used without API keys!
• Yandex, StartPage, SearchEncrypt, Swisscow, Mojeek…

Interesting Search Engines to have a look:

• https://metager.org/
• Qwant
• PrivacyWall

Other F/OSS Alternatives to PerplexityAI

• Morphic - https://github.com/miurla/morphic - Apache v2
  • https://github.com/miurla/morphic?tab=Apache-2.0-1-ov-file#readme
  • https://www.morphic.sh/

An AI-powered search engine with a generative UI

• LLocalSearch - https://github.com/nilsherzig/LLocalSearch
  • Apache v2 Licensed

LLocalSearch is a completely locally running search aggregator using LLM Agents. The user can ask a question and the system will use a chain of LLMs to find the answer. The user can see the progress of the agents and the final answer. No OpenAI or Google API keys are needed.

• OpenSearch-AI - https://github.com/supermemoryai/opensearch-ai

SearchGPT / Perplexity clone, but personalised for you.

• https://github.com/Omniplex-ai/omniplex
  • https://github.com/Omniplex-ai/omniplex?tab=AGPL-3.0-1-ov-file#readme

Gaps to Cover

I got to know about these when playing around with the AIssistant here

• https://github.com/latitude-dev/latitude-llm

LatitudeLLM is an open-source prompt engineering platform designed to streamline AI feature development.

It addresses the following problems:

• Managing prompts can be cumbersome when they’re embedded in code.
• Testing AI outputs can be inconsistent and challenging.
• Analyzing performance metrics of AI models requires a centralized tool.

As per the docs, for now SDKs are Rust or JS/TS

Key Features:

• Collaborative Prompt Management: Enables team collaboration on prompt development.

• Testing and Validation: Uses LLMs to verify AI responses for accuracy and relevance.

• Performance Insights: Provides analytics on response times and cost comparisons.

• Version Control: Tracks changes in prompts for better management.

• API and SDK Support: Facilitates easy integration into existing systems.

• Open-Source Community Driven: Encourages contributions and improvements from users.

Deployment Options:

1. Latitude Cloud: Managed solution for quick setup.

2. Latitude Self-Hosted: Open-source version for greater control and customization.

In conclusion, Latitude simplifies the process of integrating AI into products while facilitating ongoing evaluation and improvement.

There are ofc tools already waiting to be used…

How to use HF?

In HuggingFace you will find really cool and Open AI Projects to try out:

• Image to Code: https://huggingface.co/spaces/HuggingFaceM4/screenshot2html

• Audio to text: https://huggingface.co/spaces/sanchit-gandhi/whisper-jax

• https://www.futuretools.io/?pricing-model=open-source

But, be prepared to use your CC details!