A 45‑minute beginner guide to a tiny Open‑WebUI‑style chat app
Welcome to another official UVF IT build guide! In this tutorial we’ll spin up a tiny, self‑hosted chat UI that mimics the core experience of Open‑WebUI – a text box, a send button, and a streaming response from a local LLM endpoint.
You’ll use Python 3, FastAPI, SQLite and the free Ollama Docker image (or any OpenAI‑compatible API) – all on your own machine, no paid services required.
Create a fresh folder for the clone, set up an isolated Python virtual environment, and install the minimal dependencies we need: FastAPI for the API layer, Uvicorn as the ASGI server, and a SQLite driver for persistence. This gives us a clean starting point and avoids polluting your global Python packages.
All commands are cross‑platform; on Windows use python instead of python3 if that matches your installation. After this step you should see a new mychat/ directory with a venv/ folder and a requirements.txt ready for future extensions.
mkdir mychat && cd mychat
python3 -m venv venv
source venv/bin/activate # Windows: venv\Scripts\activate
pip install --upgrade pip
pip install fastapi uvicorn[standard] aiosqlite
pip freeze > requirements.txtWe store each chat turn in a tiny SQLite table called messages. Each row records the role (either user or assistant), the raw text, and a timestamp so we can order the conversation later. The helper functions below create the database file on first run and expose simple CRUD helpers that the FastAPI routes will call.
Keeping the DB logic in its own module (db.py) isolates persistence concerns and makes the later API code easier to read. The functions are synchronous for simplicity because SQLite operations are fast and the overall app is low‑traffic.
db.py
import aiosqlite
import os
from datetime import datetime
DB_PATH = "messages.db"
async def init_db():
"""Create the messages table if it doesn't exist."""
async with aiosqlite.connect(DB_PATH) as db:
await db.execute(
"""
CREATE TABLE IF NOT EXISTS messages (
id INTEGER PRIMARY KEY AUTOINCREMENT,
role TEXT NOT NULL,
content TEXT NOT NULL,
ts TEXT NOT NULL
);
"""
)
await db.commit()
async def add_message(role: str, content: str):
"""Insert a new chat turn and return its row id."""
ts = datetime.utcnow().isoformat()
async with aiosqlite.connect(DB_PATH) as db:
cursor = await db.execute(
"INSERT INTO messages (role, content, ts) VALUES (?, ?, ?)",
(role, content, ts),
)
await db.commit()
return cursor.lastrowid
async def get_conversation():
"""Return all messages ordered by timestamp."""
async with aiosqlite.connect(DB_PATH) as db:
async with db.execute("SELECT role, content FROM messages ORDER BY ts") as cur:
return [{"role": r, "content": c} async for r, c in cur]
# Ensure DB exists when the module is imported
import asyncio
asyncio.run(init_db())We will create a tiny FastAPI service that stores chat messages in a local SQLite file and forwards user prompts to an Ollama model. The GET endpoint returns the full conversation so the UI can render history, while the POST endpoint accepts a new user message, saves it, and streams the LLM response back to the client.
The streaming response uses FastAPI's StreamingResponse so the browser can display the assistant reply token‑by‑token, mimicking the feel of Open‑WebUI without any heavy frontend libraries.
main.py
import sqlite3
from fastapi import FastAPI, Request, HTTPException
from fastapi.responses import StreamingResponse, FileResponse
from fastapi.staticfiles import StaticFiles
import httpx
import json
import os
app = FastAPI()
# Serve static UI files from the 'static' folder
app.mount("/static", StaticFiles(directory="static"), name="static")
# SQLite helper – create table if missing
DB_PATH = "chat.db"
conn = sqlite3.connect(DB_PATH, check_same_thread=False)
conn.execute("CREATE TABLE IF NOT EXISTS messages (id INTEGER PRIMARY KEY AUTOINCREMENT, role TEXT, content TEXT)")
conn.commit()
def insert_message(role: str, content: str):
conn.execute("INSERT INTO messages (role, content) VALUES (?,?)", (role, content))
conn.commit()
def get_history():
cur = conn.execute("SELECT role, content FROM messages ORDER BY id")
return [{"role": r, "content": c} for r, c in cur.fetchall()]
# Endpoint to serve the UI entry point
@app.get("/")
async def root():
return FileResponse(os.path.join("static", "index.html"))
# GET chat history
@app.get("/api/history")
async def history():
return get_history()
# POST new user message and stream LLM reply
@app.post("/api/message")
async def chat(request: Request):
data = await request.json()
user_msg = data.get("message")
if not user_msg:
raise HTTPException(status_code=400, detail="Message required")
insert_message("user", user_msg)
# Ollama endpoint – can be overridden via env var
ollama_url = os.getenv("LLM_URL", "http://localhost:11434/api/chat")
async def generator():
async with httpx.AsyncClient(timeout=None) as client:
async with client.stream("POST", ollama_url, json={"model": "llama3", "messages": get_history(), "stream": True}) as resp:
async for line in resp.aiter_lines():
if line:
chunk = json.loads(line)
if "message" in chunk:
token = chunk["message"].get("content", "")
if token:
insert_message("assistant", token)
yield token
return StreamingResponse(generator(), media_type="text/event-stream")The UI is a single HTML file that loads the existing conversation on start, lets the user type a new prompt, and displays the assistant’s reply as it streams in. All logic lives in plain JavaScript – no build step, no frameworks – keeping the clone lightweight and easy to understand.
index.html
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<title>Mini LLM Chat</title>
<style>
body {font-family: Arial, sans-serif; margin:0; display:flex; flex-direction:column; height:100vh;}
#chat {flex:1; overflow-y:auto; padding:1rem;}
.msg {margin:0.5rem 0;}
.user {color:#0066cc;}
.assistant {color:#333;}
#input {display:flex; padding:0.5rem; border-top:1px solid #ccc;}
#input textarea {flex:1; resize:none;}
#input button {margin-left:0.5rem;}
</style>
</head>
<body>
<div id="chat"></div>
<div id="input">
<textarea id="msg" rows="1" placeholder="Type a message…"></textarea>
<button onclick="sendMessage()">Send</button>
</div>
<script>
const chatDiv = document.getElementById('chat');
async function loadHistory(){
const res = await fetch('/api/history');
const msgs = await res.json();
msgs.forEach(renderMessage);
}
function renderMessage(m){
const div = document.createElement('div');
div.className = 'msg ' + m.role;
div.textContent = m.content;
chatDiv.appendChild(div);
chatDiv.scrollTop = chatDiv.scrollHeight;
}
async function sendMessage(){
const textarea = document.getElementById('msg');
const userText = textarea.value.trim();
if(!userText) return;
renderMessage({role:'user', content:userText});
textarea.value='';
const resp = await fetch('/api/message', {
method:'POST',
headers:{'Content-Type':'application/json'},
body:JSON.stringify({message:userText})
});
const reader = resp.body.getReader();
const decoder = new TextDecoder();
let assistantText='';
while(true){
const {value,done}=await reader.read();
if(done) break;
assistantText+=decoder.decode(value);
// Update or create assistant bubble
let last = chatDiv.lastElementChild;
if(!last || !last.classList.contains('assistant')){
last = document.createElement('div');
last.className='msg assistant';
chatDiv.appendChild(last);
}
last.textContent=assistantText;
chatDiv.scrollTop = chatDiv.scrollHeight;
}
}
loadHistory();
</script>
</body>
</html>First, ensure Ollama is running and the model you want (e.g., llama3) is pulled. Then start the FastAPI server with Uvicorn; it will serve both the API and the static UI. Finally, open your browser to the local address and interact with the chat – you should see messages appear instantly and the assistant reply stream token by token.
# 1. Start Ollama and pull a model (run once)
ollama pull llama3
# 2. Optionally set a custom LLM endpoint
export LLM_URL=http://localhost:11434/api/chat
# 3. Launch the FastAPI app
uvicorn main:app --reload| Check | Command / Why it matters |
|---|---|
| Server starts without errors | uvicorn main:app --reload shows "Uvicorn running on http://127.0.0.1:8000" |
| Open http://127.0.0.1:8000 in a browser and see an empty chat box | Browser loads index.html served by FastAPI |
| Send a message and receive a streaming reply | Chat UI shows user text then assistant response streaming from Ollama |
You now have a functional, offline‑first LLM chat UI. Feel free to swap the Ollama endpoint for any OpenAI‑compatible service, add a login page, or persist user sessions – the core is yours to extend.