UVF IT: Minimal PDF Inspector Clone

Welcome to another official UVF IT build guide. Today we are constructing a lightweight PDF inspector inspired by Firecrawl's high-performance Rust library, but simplified for educational purposes.

You will build a tool that classifies PDFs as 'Text-Based' or 'Scanned' and extracts raw text with position awareness, using only standard Rust crates and no external APIs.

Prerequisites & Simplifications

Requires Rust 1.70+ installed via rustup; this guide assumes a Unix-like environment (Linux/macOS) for CLI commands.
This clone simplifies the original by skipping complex OCR routing, multi-column layout heuristics, and table detection to focus on core classification and text extraction.
We use lopdf and pdf-extract which may struggle with heavily encrypted or malformed PDFs compared to production-grade engines.

PDF Inspector Architecture

Phase 1: Scaffold Project

Initialize a new Rust binary project and add the necessary dependencies for PDF parsing and JSON serialization. We use lopdf for low-level PDF object access and serde_json for structured output.

Run the following commands to create the project structure and update your Cargo.toml with the required libraries. This sets the foundation for our inspector tool.

cargo new pdf-inspector-clone
cd pdf-inspector-clone
cargo add lopdf serde_json

Phase 2: Core Feature 1 (Data/Model)

Define the data structures for our classification result. We will classify a PDF as 'TextBased' or 'Scanned' based on the presence of text objects in the content stream.

Create a simple struct to hold the classification confidence and extracted text. This struct will be serialized to JSON for our final output.

src/main.rs

use serde::{Serialize, Deserialize};
use lopdf::Document;
use std::env;
use std::path::Path;

#[derive(Serialize, Deserialize, Debug, Clone)]
#[serde(rename_all = "camelCase")]
pub struct PdfInspectionResult {
    pub classification: String,
    pub confidence: f32,
    pub text_preview: String,
    pub page_count: u32,
}

fn main() {
    let args: Vec<String> = env::args().collect();
    if args.len() < 2 {
        eprintln!("Usage: pdf-inspector <path-to-pdf>");
        std::process::exit(1);
    }

    let pdf_path = &args[1];
    match inspect_pdf(pdf_path) {
        Ok(result) => {
            let json = serde_json::to_string_pretty(&result).unwrap();
            println!("{}", json);
        }
        Err(e) => {
            eprintln!("Error inspecting PDF: {}", e);
            std::process::exit(1);
        }
    }
}

fn inspect_pdf(path: &str) -> Result<PdfInspectionResult, Box<dyn std::error::Error>> {
    let doc = Document::load(path)?;
    let page_count = doc.get_pages().len() as u32;
    
    // Placeholder for logic in Phase 3
    let mut has_text = false;
    let mut text_preview = String::new();

    // TODO: Implement page iteration and text detection

    let classification = if has_text { "TextBased" } else { "Scanned" }.to_string();
    let confidence = if has_text { 0.95 } else { 0.80 };

    Ok(PdfInspectionResult {
        classification,
        confidence,
        text_preview,
        page_count,
    })
}

Phase 3: Core Feature 2 (API/UI)

Implement the core logic: load the PDF, iterate through pages, and check for text operators. If text operators are found, classify as 'TextBased'; otherwise, assume 'Scanned'.

We will extract the first 200 characters of text found. Note that lopdf provides low-level access, so we must manually parse the content streams to find text commands like 'Tj' or 'TJ'.

src/main.rs

use serde::{Serialize, Deserialize};
use lopdf::Document;
use std::env;
use std::path::Path;

#[derive(Serialize, Deserialize, Debug, Clone)]
#[serde(rename_all = "camelCase")]
pub struct PdfInspectionResult {
    pub classification: String,
    pub confidence: f32,
    pub text_preview: String,
    pub page_count: u32,
}

fn main() {
    let args: Vec<String> = env::args().collect();
    if args.len() < 2 {
        eprintln!("Usage: pdf-inspector <path-to-pdf>");
        std::process::exit(1);
    }

    let pdf_path = &args[1];
    match inspect_pdf(pdf_path) {
        Ok(result) => {
            let json = serde_json::to_string_pretty(&result).unwrap();
            println!("{}", json);
        }
        Err(e) => {
            eprintln!("Error inspecting PDF: {}", e);
            std::process::exit(1);
        }
    }
}

fn inspect_pdf(path: &str) -> Result<PdfInspectionResult, Box<dyn std::error::Error>> {
    let doc = Document::load(path)?;
    let page_count = doc.get_pages().len() as u32;
    
    let mut has_text = false;
    let mut text_preview = String::new();

    for (page_id, _page) in doc.get_pages() {
        let content_stream = doc.get_page_content(page_id)?;
        let content_str = String::from_utf8_lossy(&content_stream.0);
        
        // Simple heuristic: check for common PDF text operators
        if content_str.contains("Tj") || content_str.contains("TJ") {
            has_text = true;
            // Extract a simple preview by finding text between parentheses
            // This is a naive extraction for demonstration purposes
            if text_preview.is_empty() {
                let start = content_str.find('(');
                if let Some(start_idx) = start {
                    let end = content_str[start_idx..].find(')');
                    if let Some(end_idx) = end {
                        let raw_text = &content_str[start_idx + 1..start_idx + end_idx];
                        // Remove escape sequences and clean up
                        let cleaned = raw_text.replace("\\n", " ").replace("\\t", " ");
                        text_preview = cleaned.chars().take(200).collect();
                    }
                }
            }
            break; // Found text, no need to check further pages for classification
        }
    }

    let classification = if has_text { "TextBased" } else { "Scanned" }.to_string();
    let confidence = if has_text { 0.95 } else { 0.80 };

    Ok(PdfInspectionResult {
        classification,
        confidence,
        text_preview,
        page_count,
    })
}

Phase 4: Run Locally

Now that the logic is in place, we compile the Rust binary and execute it against a sample PDF file. You should have a test file ready, such as a simple text-based PDF or a scanned image PDF, to see how the classifier behaves.

Running the binary with cargo run will trigger the PDF parsing routine. The program will read the file path provided as an argument, analyze the content streams, and print the resulting JSON classification to your terminal.

# Compile the project in release mode for optimized performance
cargo build --release

# Run the binary against a sample PDF file
# Replace 'sample.pdf' with the path to your actual test file
./target/release/pdf-inspector sample.pdf

Phase 5: Verify and Extend

Verify the output by piping the result into jq to pretty-print the JSON. This allows you to confirm that the 'classification' field correctly identifies the PDF as 'TextBased' or 'Scanned'.

To extend this minimal clone, consider integrating the pdf-extract crate. This library can pull actual text strings from the PDF, allowing you to populate the 'extracted_text' field with real content rather than just a placeholder or empty string.

# Verify the JSON structure and classification result
./target/release/pdf-inspector sample.pdf | jq .

# Example of extending with pdf-extract (requires adding to Cargo.toml)
# cargo add pdf-extract
# Then update main.rs to use pdf_extract::extract_text_from_file

Verifying the Stack

Check	Command / Why it matters
Project compiles without errors	cargo build --release
JSON output is valid and contains 'classification' field	cargo run --release -- sample.pdf \| jq .classification
Correctly identifies a text-based PDF	Run against a known text PDF and check for 'TextBased'
Handles missing file gracefully	cargo run --release -- nonexistent.pdf && echo 'Should have errored'

Next Steps

To make this production-ready, integrate a full text extraction library like pdf-extract or tesseract for OCR support on scanned pages.