UVF IT: Local Voice Cloning Demo with VoiceStudio (Python)

Welcome to another official UVF IT build guide! In this tutorial we’ll spin up a tiny, fully‑local voice‑cloning demo inspired by the open‑source VoiceStudio project. You’ll get a Flask API that accepts text and returns a short WAV generated with a lightweight TTS model – no cloud keys required.

We’ll use Python, the free‑tier TTS library (which ships a tiny pretrained model) and SQLite for a tiny metadata store. In ~45 minutes you’ll have a runnable clone you can extend into dubbing or voice‑design experiments.

Before you start…

You need Python 3.9‑3.11 and git installed; older versions may fail to build the torch wheel used by the TTS library.
The demo uses the smallest “ljspeech” model (~30 MB). Real VoiceStudio ships many larger multilingual models – our clone only supports English text‑to‑speech.
Running on CPU is fine for the demo but will be slower than the GPU‑accelerated original; expect a few seconds per utterance.

Minimal VoiceStudio Clone Architecture

Phase 1: Scaffold the project

Create a clean folder for the demo, initialise a Python virtual environment and install only the tiny dependencies we actually need. This gives us an isolated workspace and keeps the install fast on modest hardware.

We’ll pull in Flask for the API, the TTS library for the tiny pretrained model, and sqlite3 (built‑in) for our metadata store. All of this fits in a few megabytes and runs on CPU.

mkdir voice_clone_demo && cd voice_clone_demo
python3 -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
pip install Flask TTS

Phase 2: Add a minimal data layer

We’ll store each generated clip’s original text and the path to its WAV file in a SQLite database. SQLite needs no server, works out‑of‑the‑box, and gives us a simple way to list or replay past clips later.

The helper below creates the table on first run and provides two tiny functions: one to insert a new record and another to fetch all records ordered by creation time. Keep this file alongside your Flask app.

db.py

import sqlite3
from pathlib import Path

DB_PATH = Path('clips.db')

def _connect():
    return sqlite3.connect(DB_PATH)

def init_db():
    with _connect() as conn:
        conn.execute('''
            CREATE TABLE IF NOT EXISTS clips (
                id INTEGER PRIMARY KEY AUTOINCREMENT,
                text TEXT NOT NULL,
                wav_path TEXT NOT NULL,
                created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
            )
        ''')

def add_clip(text: str, wav_path: str):
    with _connect() as conn:
        conn.execute('INSERT INTO clips (text, wav_path) VALUES (?, ?)', (text, wav_path))
        conn.commit()

def list_clips():
    with _connect() as conn:
        cur = conn.execute('SELECT id, text, wav_path, created_at FROM clips ORDER BY created_at DESC')
        return cur.fetchall()

# Initialise on import so the table exists before any API call
init_db()

Phase 3: Core API & UI

In this step we wire up a tiny Flask server that accepts a POST to /speak with a JSON payload containing the text you want spoken. The server runs the TTS model (here we use the simple gTTS wrapper for speed) and writes a WAV file into the static folder, then returns the file so the browser can download it.

We also add a minimal HTML page served at the root that lets you type text, hit a button, and hear the generated voice instantly. This UI demonstrates the whole flow without any extra JavaScript frameworks, keeping the clone lightweight and easy to understand.

app.py

from flask import Flask, request, send_from_directory, jsonify, render_template_string
from gtts import gTTS
import os, uuid
app = Flask(__name__)
STATIC_DIR = 'static'
os.makedirs(STATIC_DIR, exist_ok=True)
# In‑memory list of generated clips
clips = []
@app.route('/')
def index():
    html = '''
    <!doctype html><html><head><title>VoiceStudio Demo</title></head><body>
    <h2>Enter text to speak</h2>
    <input id='txt' type='text' size='40'>
    <button onclick='speak()'>Speak</button>
    <script>
    function speak(){
        fetch('/speak',{
            method:'POST',
            headers:{'Content-Type':'application/json'},
            body:JSON.stringify({text:document.getElementById('txt').value})
        })
        .then(r=>r.blob())
        .then(b=>{const url=URL.createObjectURL(b); const a=document.createElement('a'); a.href=url; a.download='output.wav'; a.click();});
    }
    </script>
    </body></html>'''
    return render_template_string(html)
@app.route('/speak', methods=['POST'])
def speak():
    data = request.get_json()
    txt = data.get('text','')
    if not txt:
        return jsonify(error='No text provided'),400
    filename = f"{uuid.uuid4()}.wav"
    path = os.path.join(STATIC_DIR, filename)
    gTTS(txt).save(path)
    clips.append({'id':len(clips)+1,'text':txt,'file':filename})
    return send_from_directory(STATIC_DIR, filename, as_attachment=True)
@app.route('/clips')
def get_clips():
    return jsonify(clips)
if __name__=='__main__':
    app.run(debug=True)

Phase 4: Run locally

Now that the Flask app is ready, start it with a single command. The development server will listen on localhost and serve both the API and the tiny UI we just created. Open your browser to the shown address and you’ll see the text box ready for testing.

python app.py

Phase 5: Verify & extend

Use the web page to type a phrase and click Speak – the browser should download a .wav file that plays your spoken text. Then, query the /clips endpoint to confirm the server stored metadata about the generated clip. From here you can expand the UI to list past clips or connect the API to a video‑dubbing script that pulls audio automatically.

curl http://127.0.0.1:5000/clips

Verifying the Stack

Check	Command / Why it matters
Server starts without errors	python app.py → console shows Running on http://127.0.0.1:5000/
Browser UI loads at http://127.0.0.1:5000/	Open the URL and see a text box + Speak button
Audio file downloads and plays when submitting text	Enter "Hello world", click Speak, and the browser downloads a .wav that plays
GET /clips returns JSON with at least one entry	curl http://127.0.0.1:5000/clips → shows [{"id":1,"text":"Hello world","file":"...wav"}]

What’s next?

You now have a functional local voice‑cloning demo. Try swapping the TTS model for a multilingual one, add a React front‑end, or integrate the /clips endpoint into a simple video‑dubbing pipeline. Happy hacking!