UVF IT: Tiny Go Search Engine Clone

Welcome to another official UVF IT build guide. In this tutorial you’ll spin up a tiny Go‑based search engine inspired by Hister, using only the Go standard library and a free SQLite database. No Docker, no paid APIs – just pure code you can run locally.

We’ll scaffold a small Go project, create a SQLite schema for documents, expose a tiny HTTP API to index and query text, run it locally, and verify that you can search a sample page. Feel free to extend it with a UI or browser extension later.

Before you start

You need Go 1.22+ and SQLite3 installed on your machine; the guide assumes a Unix‑like shell.
The clone only supports plain‑text indexing and simple LIKE queries – no ranking, stemming, or security hardening that the real Hister provides.
Running on port 8080 may conflict with other services; adjust the code if needed.

Component Overview

Phase 1: Scaffold the project

We start by creating a fresh Go module so all our code lives under a single import path. Then we fetch a pure‑Go SQLite driver (modernc.org/sqlite) that works without external C bindings, keeping the setup lightweight.

Having the module and driver ready lets us write Go code that talks to a local .db file without any extra configuration.

mkdir histerclone && cd histerclone
go mod init example.com/histerclone
go get modernc.org/sqlite@v1.23.0

Phase 2: Define data model & init DB

Next we create a tiny Go file that opens (or creates) a SQLite database and ensures the required table exists. The table stores each document’s URL and its raw text – enough for simple substring searches.

Running this program once will bootstrap the schema; subsequent runs will simply reuse the existing table.

We keep the schema creation idempotent so the code can be executed on every start without harming existing data.

initdb.go

package main

import (
    "database/sql"
    "log"
    _ "modernc.org/sqlite"
)

func main() {
    db, err := sql.Open("sqlite", "hister.db")
    if err != nil {
        log.Fatalf("open db: %v", err)
    }
    defer db.Close()

    schema := `CREATE TABLE IF NOT EXISTS documents (
        id INTEGER PRIMARY KEY AUTOINCREMENT,
        url TEXT NOT NULL UNIQUE,
        content TEXT NOT NULL
    );`
    if _, err := db.Exec(schema); err != nil {
        log.Fatalf("create schema: %v", err)
    }
    log.Println("Database initialized successfully")
}

Phase 3: Implement API endpoints

In this step we wire up a tiny HTTP server that can receive documents and search them. The POST /index endpoint stores a title and body into a SQLite file, while GET /search?q=term looks up matching titles using a simple LIKE query. Keeping everything in one file makes the clone easy to read and run.

We use Go's standard net/http package and the modern go-sqlite3 driver. The code creates the database on first run, defines a documents table, and returns JSON responses so you can interact with it via curl or any HTTP client.

main.go

package main

import (
    "database/sql"
    "encoding/json"
    "log"
    "net/http"
    "os"
    "strings"
    _ "github.com/mattn/go-sqlite3"
)

type Document struct {
    ID    int64  `json:"id"`
    Title string `json:"title"`
    Body  string `json:"body"`
}

var db *sql.DB

func initDB() {
    var err error
    db, err = sql.Open("sqlite3", "hister.db")
    if err != nil { log.Fatal(err) }
    stmt := `CREATE TABLE IF NOT EXISTS documents (id INTEGER PRIMARY KEY AUTOINCREMENT, title TEXT, body TEXT);`
    if _, err = db.Exec(stmt); err != nil { log.Fatal(err) }
}

func indexHandler(w http.ResponseWriter, r *http.Request) {
    if r.Method != http.MethodPost { http.Error(w, "method not allowed", http.StatusMethodNotAllowed); return }
    var doc Document
    if err := json.NewDecoder(r.Body).Decode(&doc); err != nil { http.Error(w, err.Error(), http.StatusBadRequest); return }
    res, err := db.Exec("INSERT INTO documents (title, body) VALUES (?, ?)", doc.Title, doc.Body)
    if err != nil { http.Error(w, err.Error(), http.StatusInternalServerError); return }
    id, _ := res.LastInsertId()
    doc.ID = id
    w.WriteHeader(http.StatusCreated)
    json.NewEncoder(w).Encode(doc)
}

func searchHandler(w http.ResponseWriter, r *http.Request) {
    q := r.URL.Query().Get("q")
    if strings.TrimSpace(q) == "" { http.Error(w, "query missing", http.StatusBadRequest); return }
    rows, err := db.Query("SELECT id, title FROM documents WHERE title LIKE ? OR body LIKE ?", "%"+q+"%", "%"+q+"%")
    if err != nil { http.Error(w, err.Error(), http.StatusInternalServerError); return }
    defer rows.Close()
    var results []Document
    for rows.Next() {
        var d Document
        if err := rows.Scan(&d.ID, &d.Title); err != nil { continue }
        results = append(results, d)
    }
    json.NewEncoder(w).Encode(results)
}

func main() {
    initDB()
    http.HandleFunc("/index", indexHandler)
    http.HandleFunc("/search", searchHandler)
    log.Println("Listening on :8080")
    if err := http.ListenAndServe(":8080", nil); err != nil { log.Fatal(err) }
    // ensure DB is closed on exit
    defer db.Close()
}

Phase 4: Run locally

Now we compile the Go program and start the server. The binary will automatically create hister.db in the current directory if it doesn't exist, so you don't need any manual setup.

Running the server in the background lets you test the API with curl or any HTTP client while you develop further features.

go mod init hister-clone
go get github.com/mattn/go-sqlite3
go build -o hister-clone main.go
./hister-clone &
# note the PID; you can stop it later with kill %1

Phase 5: Verify & extend

With the server running, we can index a sample document and then search for a keyword to see the results. Using curl keeps the verification simple and reproducible.

Feel free to add more fields, switch the LIKE query to SQLite's FTS5 for real full‑text search, or build a tiny HTML front‑end that talks to these endpoints.

# Index a document
curl -s -X POST http://localhost:8080/index -H "Content-Type: application/json" -d '{"title":"Go Docs","body":"The Go programming language documentation"}' -w "\nStatus:%{http_code}\n'

# Search for the keyword "Go"
curl -s "http://localhost:8080/search?q=Go" -w "\nStatus:%{http_code}\n'

Verifying the Stack

Check	Command / Why it matters
Server starts without panic	./hister-clone shows "Listening on :8080" in console
Document indexed successfully	First curl returns HTTP 201 status
Search returns expected title	Second curl JSON includes "Go Docs" in results

What’s next?

You now have a functional personal search backend. Add a tiny HTML front‑end, hook a browser extension to POST pages, or replace the LIKE query with SQLite FTS5 for real full‑text search. Happy hacking!