Document‑Prep Tool for Gen‑AI (Python)

A minimal clone of Docling using free‑tier Python stack

Welcome to another official UVF IT build guide! In this quick tutorial we’ll craft a lightweight document‑preparation service inspired by Docling, perfect for feeding clean text into Gen‑AI models.

You’ll spin up a tiny FastAPI app, use the open‑source docling library to extract text from PDFs, store results in a local SQLite DB, and test it all in under an hour.

Before you start…

Minimal Docling Clone Architecture
LOCAL RUNTIME upload store extracted text PDF Input FastAPI Service Python + docling SQLite DB
1

Phase 1: Scaffold the project

Create a fresh folder, initialise a virtual environment, and install FastAPI, Uvicorn, SQLite driver and docling.

mkdir docling_clone && cd docling_clone
python3 -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
pip install fastapi uvicorn[standard] aiosqlite docling
2

Phase 2: Define the SQLite model and helper

A tiny table stores file_name and extracted content. The helper opens a DB connection and creates the table if missing.

db.py

import aiosqlite
import os

DB_PATH = os.getenv('DOC_DB', 'documents.db')

async def init_db():
    async with aiosqlite.connect(DB_PATH) as db:
        await db.execute(
            """CREATE TABLE IF NOT EXISTS docs (
                id INTEGER PRIMARY KEY AUTOINCREMENT,
                file_name TEXT NOT NULL,
                content TEXT NOT NULL
            )"""
        )
        await db.commit()

async def insert_doc(file_name: str, content: str):
    async with aiosqlite.connect(DB_PATH) as db:
        await db.execute(
            "INSERT INTO docs (file_name, content) VALUES (?, ?)",
            (file_name, content)
        )
        await db.commit()

async def fetch_all():
    async with aiosqlite.connect(DB_PATH) as db:
        async with db.execute("SELECT file_name, content FROM docs") as cursor:
            return await cursor.fetchall()
3

Phase 3: Core extraction endpoint (API/UI)

In this step we expose a FastAPI POST endpoint called /extract. It accepts a PDF file upload, hands the file to the docling library to pull out plain‑text, and then stores the filename together with the extracted snippet in a tiny SQLite database. The endpoint finally returns a short JSON preview so you can see the result instantly in Swagger UI or via curl. This gives you the core “document‑prep” service that downstream LLM components can call.

main.py

import io
import sqlite3
from fastapi import FastAPI, File, UploadFile, HTTPException
from fastapi.responses import JSONResponse
from docling.document_converter import DocumentConverter

app = FastAPI()

# Initialise SQLite (creates docs table if missing)
conn = sqlite3.connect('docs.db')
conn.execute('''CREATE TABLE IF NOT EXISTS docs (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    filename TEXT NOT NULL,
    snippet TEXT NOT NULL
)''')
conn.commit()

converter = DocumentConverter()

@app.post('/extract')
async def extract(file: UploadFile = File(...)):
    if file.content_type != 'application/pdf':
        raise HTTPException(status_code=400, detail='Only PDF files are accepted')
    content = await file.read()
    # Convert PDF bytes to plain text using docling
    doc = converter.convert(io.BytesIO(content))
    text = doc.get_text()
    snippet = text[:500]  # first 500 chars as preview
    # Store in SQLite
    conn.execute('INSERT INTO docs (filename, snippet) VALUES (?, ?)',
                 (file.filename, snippet))
    conn.commit()
    return JSONResponse(content={'filename': file.filename, 'preview': snippet})
4

Phase 4: Run locally and test

Now start the service with Uvicorn in reload mode so code changes are reflected automatically. Once the server is up, open the automatically generated Swagger UI at http://127.0.0.1:8000/docs – you’ll see the /extract endpoint ready to accept a file. You can also test it from the command line with curl, sending a small PDF and watching the JSON preview returned.

uvicorn main:app --reload
5

Phase 5: Verify & extend

After a successful upload, open the SQLite file to confirm the row was written – this proves persistence works. To make the service more useful, we add a simple GET endpoint that returns the full stored snippet for a given filename. This keeps the codebase tiny while demonstrating how you can grow the API with read‑only operations.

main.py

# Append to the existing main.py
from fastapi import Query

@app.get('/document')
def get_document(name: str = Query(..., description='Exact filename used during upload')):
    cur = conn.execute('SELECT snippet FROM docs WHERE filename = ?', (name,))
    row = cur.fetchone()
    if not row:
        raise HTTPException(status_code=404, detail='Document not found')
    return JSONResponse(content={'filename': name, 'full_text': row[0]})

Verifying the Stack

CheckCommand / Why it matters
Server starts without errorsuvicorn main:app --reload → should show 'Uvicorn running on http://127.0.0.1:8000'
Upload a PDF via Swagger UI (http://127.0.0.1:8000/docs) and see a JSON previewOpen browser, use /extract endpoint, select a small PDF, submit
SQLite file docs.db contains a row after uploadsqlite3 docs.db "SELECT * FROM docs;" → should list the inserted record

What’s next?

You now have a functional document‑prep microservice. Hook it up to your LLM pipeline, add OCR via pytesseract for scanned PDFs, or containerise with Docker for easy deployment.