AI Projects on a Raspberry Pi
These year the Pi5 was launched.
What a perfect oportunity that was to get finally a Raspbery Pi 4 with a reasonable price and do some AI projects with it.
This time I got a 4GB Pi4, ARM64!
Let’s do some cool stuff with it.
- Raspberry Pi can do more than IoT. Go AI with a RPi
- Python Checks 👇
- Docker Checks
- Third Party APIs: OpenAI, Anthropic, Groq, also Ollama!
- Use AI Tools with your Raspberry Pi
Project | Code | Use Case |
---|---|---|
Streamlit Multi-Chat ✓ | Source Code 🐍 Container | Chat with Several LLMs (APIs). See Blog |
Youtube Groq Summaries ✓ | Source Code 🐍Container | Summaries of YT Videos with Groq. Blog |
Chat with PDF ✓ | Source Code 🐍 Container | Chat with PDFs (OpenAI). Blog |
These are LLM related AI projects that you can play with.
mindmap
root((AI with RPi))
AI
::icon(fas fa-robot)
APIs
OpenAI
Anthropic
Groq
Docker Compose Configuration
Portainer UI
Hardware
Raspberry Pi 4
::icon(fab fa-raspberry-pi)
DevTools
Python
::icon(fab fa-python)
Docker Containers
::icon(fab fa-docker)
GithubActions
::icon(fas fa-code-branch)
These projects can have working container images for x86/ARM64.
Credits to AlejandroAO for The Diagram and Initial Project which I forked
I made a Youtube Video on the PDF AI Tool
The images are build with Github Actions with QEMU for MultiArch (X86/ARM64).
You can see them asGithub Packages
1
2
3
4
5
6
7
8
9
#https://github.com/JAlcocerT/phidata/tree/main/Z_DeployMe
docker pull ghcr.io/jalcocert/phidata:yt-groq #https://github.com/users/JAlcocerT/packages/container/package/phidata
#https://github.com/JAlcocerT/Streamlit-MultiChat/pkgs/container/streamlit-multichat
docker pull ghcr.io/jalcocert/streamlit-multichat:latest #https://github.com/JAlcocerT/Streamlit-MultiChat/tree/main/Z_DeployMe
#https://github.com/JAlcocerT/ask-multiple-pdfs/tree/main/Z_Deploy_me
sudo docker pull ghcr.io/jalcocert/ask-multiple-pdfs:v1.0 #https://github.com/JAlcocerT/ask-multiple-pdfs/pkgs/container/ask-multiple-pdfs
You can goahead and deploy with a Docker-Compose/Portainer.
Or if you are now with Docker/Containers, let me guide you through the build process:
- Lets clone each repository
- Use each
Dockerfile
to build the container images from source - Tweak the AI Stack with the name of the local image
You will need Docker installed. And Portainer UI will be beneficial.
1
2
3
4
5
6
7
8
9
10
git clone https://github.com/JAlcocerT/phidata
git clone https://github.com/JAlcocerT/Streamlit-MultiChat
git clone https://github.com/JAlcocerT/ask-multiple-pdfs
cd ./phidata && sudo docker build -t phidata_yt_groq . && cd ..
cd ./Streamlit-MultiChat && sudo docker build -t streamlit-multichat . && cd ..
cd ./ask-multiple-pdfs && sudo docker build -t ask-multiple-pdfs . && cd ..
If you want to try just one of them, you can use a quick Docker CLI like so, when the image is build:
1
2
3
4
5
6
7
8
docker run -d \
--name phidata_yt_groq \
-p 8509:8501 \
-e GROQ_API_KEY=your_api_key_here \
phidata_yt_groq \
streamlit run cookbook/llms/groq/video_summary/app.py
#ifconfig #to get to know the device local IP
You will see the Youtube Groq summarizer at: http://devicelocalip:8509
This is a realiable and DIY way of trying cool projects out there!
AI Stack
If you have the API Keys and Docker installed…
- https://platform.openai.com/api-keys
- https://console.groq.com/keys
- https://console.anthropic.com/settings/keys
Just use this configuration to spin the 3 AI services:
1
2
3
4
curl -o docker-compose.yml https://raw.githubusercontent.com/JAlcocerT/Docker/main/AI_Gen/Project_AIs/docker-compose.yml
docker-compose up -d
You can use it as well via Portainer UI as a Stack:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
version: '3'
services:
# streamlit-chat-pdfs:
# image: ghcr.io/jalcocert/ask-multiple-pdfs:v1.0 #BUILD IT FOR ARM64 First
# container_name: chat_multiple_pdf
# volumes:
# - ai_chat_multiple_pdf:/app
# working_dir: /app # Set the working directory to /app
# command: /bin/sh -c "export OPENAI_API_KEY='your_api_key_here' && streamlit run appv3_pass.py"
# #command: tail -f /dev/null
# ports:
# - "8501:8501"
# restart: unless-stopped
streamlit-multichat:
image: ghcr.io/jalcocert/streamlit-multichat #:v1.0
container_name: streamlit_multichat
volumes:
- ai_streamlit_multichat:/app
working_dir: /app
command: /bin/sh -c "\
mkdir -p /app/.streamlit && \
echo 'OPENAI_API_KEY = \"sk-proj-openaiAPIhere\"' > /app/.streamlit/secrets.toml && \
echo 'GROQ_API_KEY = \"gsk_groqAPIhere\"' >> /app/.streamlit/secrets.toml && \
echo 'ANTHROPIC_API_KEY = \"sk-ant-yourANTHROPICapihere\"' >> /app/.streamlit/secrets.toml && \
streamlit run Z_multichat.py"
ports:
- "8501:8501"
restart: always
environment:
MODEL_API_KEY: sk-proj-openaiAPIhere
MODEL: gpt-4o-mini #gpt-4
TEMPERATURE: 0 #dont be creative :)
#restart: always
phidata_yt_groq:
image: ghcr.io/jalcocert/phidata:yt-groq #phidata_yt_groq
container_name: phidata_yt_groq
ports:
- "8502:8501"
environment:
- GROQ_API_KEY=your_api_key_here # your_api_key_here
command: tail -f /dev/null #streamlit run cookbook/llms/groq/video_summary/app.py
restart: unless-stopped
volumes:
ai_streamlit_multichat:
volumes:
ai_chat_multiple_pdf:
See how each AI Project is consuming resources:
1
2
htop
sudo docker stats streamlit_multichat
With 4GB its more than enough! I got ~400mb/3.71GB of RAM
FAQ
Get Docker ready for SelfHosting, like so.
You can also try the projects separately with just Python and the required API’s.
Make sure to setup a proper Python Venv.
More Vector DataBases - Docker Config Files
The diagram has been possible thanks to MermaidJS Jekyll Integration and the Fontawsome Icons
Youtube Video - e9hJZrT7HLw
AI Projects Quick CLI Setup
1
2
3
4
5
6
7
8
sudo docker run -d \
--name chat_multiple_pdf \
-v ai_chat_multiple_pdf:/app \
-w /app \
-e OPENAI_API_KEY=your_api_key_here \
-p 8501:8501 \
ghcr.io/jalcocert/ask-multiple-pdfs:v1.0 \
/bin/sh -c "streamlit run appv3_pass.py"
For the MultiChat Project:
1
2
3
4
export OPENAI_API_KEY="sk-proj-openaiAPIhere"
export GROQ_API_KEY="gsk_groqAPIhere"
export ANTHROPIC_API_KEY="sk-ant-yourANTHROPICapihere"
export MODEL_API_KEY="sk-proj-openaiAPIhere"
Then:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
sudo docker run -d \
--name streamlit_multichat \
-v ai_streamlit_multichat:/app \
-w /app \
-p 8501:8501 \
-e MODEL="gpt-4o-mini" \
-e TEMPERATURE="0" \
ghcr.io/jalcocert/streamlit-multichat \
/bin/sh -c "\
mkdir -p /app/.streamlit && \
echo 'OPENAI_API_KEY = \"$OPENAI_API_KEY\"' > /app/.streamlit/secrets.toml && \
echo 'GROQ_API_KEY = \"$GROQ_API_KEY\"' >> /app/.streamlit/secrets.toml && \
echo 'ANTHROPIC_API_KEY = \"$ANTHROPIC_API_KEY\"' >> /app/.streamlit/secrets.toml && \
streamlit run Z_multichat.py"