UVF IT: Mini Feature‑Flag A/B Testing Dashboard

Welcome to another official UVF IT guide where we roll up our sleeves and build a tiny A/B‑testing dashboard inspired by Flagsmith Experiments. In just ~45 minutes you’ll have a local web app that lets you define experiments, assign variants, and see simple conversion counts—all without any paid services.

We’ll use Python 3, Flask, and SQLite – all free‑tier, open‑source tools – to mimic the core experience of Flagsmith’s experiment UI. The result is a minimal, runnable clone you can extend as you learn.

Before you start

You need Python 3.9+ and pip installed; the guide assumes a Unix‑like shell (Linux/macOS or WSL).
The clone only stores data locally and lacks authentication, real‑time analytics, and multi‑project support that the full Flagsmith product provides.
Running the app on a different port than 5000 may require adjusting the URLs in the checklist steps.

Mini A/B Testing Architecture

Phase 1: Scaffold the project

Create a virtual environment and install Flask; this gives us a clean workspace.

A virtual environment isolates our dependencies so the guide won’t interfere with any other Python projects you might have.

python3 -m venv venv
source venv/bin/activate
pip install --upgrade pip
pip install Flask

Phase 2: Define the data model

We’ll use SQLite via Flask’s built‑in g object; the schema holds experiments, variants, and conversion counts.

The tiny schema lives in a single file and is created automatically when the app starts for the first time.

app.py

import sqlite3
from flask import Flask, g, request, jsonify

app = Flask(__name__)
DB_PATH = 'ab_test.db'

# ---------- Database helpers ----------

def get_db():
    if 'db' not in g:
        g.db = sqlite3.connect(DB_PATH)
        g.db.row_factory = sqlite3.Row
    return g.db

@app.teardown_appcontext
def close_db(exc):
    db = g.pop('db', None)
    if db is not None:
        db.close()

def init_db():
    db = get_db()
    db.executescript('''
        CREATE TABLE IF NOT EXISTS experiments (
            id INTEGER PRIMARY KEY AUTOINCREMENT,
            name TEXT NOT NULL UNIQUE
        );
        CREATE TABLE IF NOT EXISTS variants (
            id INTEGER PRIMARY KEY AUTOINCREMENT,
            experiment_id INTEGER NOT NULL,
            name TEXT NOT NULL,
            weight INTEGER NOT NULL,
            conversions INTEGER DEFAULT 0,
            FOREIGN KEY (experiment_id) REFERENCES experiments(id)
        );
    ''')
    db.commit()

# Initialize DB on first request
@app.before_first_request
def setup():
    init_db()

# ---------- Simple API endpoints (placeholder) ----------
@app.route('/experiments', methods=['GET'])
def list_experiments():
    rows = get_db().execute('SELECT * FROM experiments').fetchall()
    return jsonify([dict(row) for row in rows])

if __name__ == '__main__':
    app.run(debug=True)

Phase 3: Core API & simple UI

In this step we expose a handful of HTTP endpoints that let you create an experiment, add variants, record conversions, and see a tiny dashboard. The API stores everything in a SQLite database that Flask creates for us, so you can focus on the logic rather than infrastructure.

The UI is just a single Jinja template that reads the experiment data and prints variant names with their conversion counts. Keeping the UI minimal makes it easy to extend later with charts or a React front‑end.

app.py

from flask import Flask, request, jsonify, render_template, redirect, url_for
import sqlite3

app = Flask(__name__)

# Helper to get a DB connection
def get_db():
    conn = sqlite3.connect('abtest.db')
    conn.row_factory = sqlite3.Row
    return conn

# Initialise tables if they don't exist
def init_db():
    db = get_db()
    db.executescript('''
        CREATE TABLE IF NOT EXISTS experiment (id INTEGER PRIMARY KEY AUTOINCREMENT, name TEXT);
        CREATE TABLE IF NOT EXISTS variant (id INTEGER PRIMARY KEY AUTOINCREMENT, exp_id INTEGER, name TEXT, conversions INTEGER DEFAULT 0);
    ''')
    db.commit()
    db.close()

init_db()

# Create a new experiment
@app.route('/exp', methods=['POST'])
def create_experiment():
    name = request.json.get('name') if request.is_json else request.form.get('name')
    db = get_db()
    cur = db.execute('INSERT INTO experiment (name) VALUES (?)', (name,))
    exp_id = cur.lastrowid
    db.commit()
    db.close()
    return jsonify({'id': exp_id, 'name': name})

# Add a variant to an experiment
@app.route('/exp/<int:exp_id>/variant', methods=['POST'])
def add_variant(exp_id):
    vname = request.json.get('name') if request.is_json else request.form.get('name')
    db = get_db()
    db.execute('INSERT INTO variant (exp_id, name) VALUES (?, ?)', (exp_id, vname))
    db.commit()
    db.close()
    return redirect(url_for('view_experiment', exp_id=exp_id))

# Record a conversion for a variant
@app.route('/exp/<int:exp_id>/variant/<int:var_id>/convert', methods=['POST'])
def record_conversion(exp_id, var_id):
    db = get_db()
    db.execute('UPDATE variant SET conversions = conversions + 1 WHERE id = ? AND exp_id = ?', (var_id, exp_id))
    db.commit()
    db.close()
    return jsonify({'status': 'recorded'})

# Simple dashboard view
@app.route('/exp/<int:exp_id>')
def view_experiment(exp_id):
    db = get_db()
    exp = db.execute('SELECT * FROM experiment WHERE id = ?', (exp_id,)).fetchone()
    variants = db.execute('SELECT * FROM variant WHERE exp_id = ?', (exp_id,)).fetchall()
    db.close()
    return render_template('dashboard.html', experiment=exp, variants=variants)

if __name__ == '__main__':
    app.run(debug=True)

Phase 4: Run locally

Now that the Flask app contains all the routes, you can start the development server on your machine. The server will bind to http://127.0.0.1:5000 and automatically reload when you edit the code, which is perfect for quick iteration.

Make sure you have the required packages installed (flask and sqlite3 is built‑in). If you haven't, run the pip install command first.

pip install flask
python app.py

Phase 5: Verify and extend

With the server running, use curl or a browser to exercise the API: create an experiment, add a couple of variants, fire some conversion events, and finally visit the dashboard URL to see the counts. This confirms the core flow works.

From here you can add weighting logic (store a weight column on variants and pick one randomly), or build a tiny JavaScript snippet that hits the /convert endpoint when a user performs the target action.

# 1. Create an experiment
curl -X POST http://127.0.0.1:5000/exp -H "Content-Type: application/json" -d '{"name":"Button Color Test"}'
# 2. Add two variants (replace <ID> with the experiment id returned above)
curl -X POST http://127.0.0.1:5000/exp/<ID>/variant -d 'name=Blue'
curl -X POST http://127.0.0.1:5000/exp/<ID>/variant -d 'name=Green'
# 3. Record a conversion for variant 1 (replace <VAR_ID>)
curl -X POST http://127.0.0.1:5000/exp/<ID>/variant/<VAR_ID>/convert
# 4. Open the dashboard in a browser
open http://127.0.0.1:5000/exp/<ID>

Verifying the Stack

Check	Command / Why it matters
Flask server starts without error	python app.py → should show Running on http://127.0.0.1:5000/
POST /exp returns JSON with an id	curl -X POST http://127.0.0.1:5000/exp -d '{"name":"Button Color"}' → response like {"id":1,"name":"Button Color"}
Dashboard displays conversion counts	Visit http://127.0.0.1:5000/exp/1 → page shows variant names and numbers

You did it!

Your minimal A/B‑testing dashboard is up and running. Feel free to experiment with weighting, add a tiny front‑end client, or swap SQLite for Postgres when you’re ready to scale.