OCR Pipeline: Extracting Text from Product Labels with Python

The Yuga Yatra jewellery inventory contained 10,000+ products with handwritten labels — weight, purity, making charges, and SKU codes. Manual data entry would take weeks. I built an OCR pipeline in Python that processed all labels in under 2 hours with 94% accuracy.

The Challenge

Handwritten labels are notoriously difficult for OCR. Variations in handwriting style, lighting conditions, and label positioning meant off-the-shelf Tesseract achieved only 60% accuracy initially. I needed a preprocessing pipeline.

Preprocessing Pipeline

The preprocessing stage is where most of the accuracy gains come from. Here's the pipeline I ended up with:

import cv2
import numpy as np

def preprocess_image(image_path):
    img = cv2.imread(image_path)
    # 1. Convert to grayscale
    gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
    # 2. Denoise (preserve edges)
    denoised = cv2.fastNlMeansDenoising(gray, h=10)
    # 3. Adaptive thresholding (handles uneven lighting)
    thresh = cv2.adaptiveThreshold(
        denoised, 255,
        cv2.ADAPTIVE_THRESH_GAUSSIAN_C,
        cv2.THRESH_BINARY, 11, 2
    )
    # 4. Deskew using minAreaRect
    coords = np.column_stack(np.where(thresh > 0))
    angle = cv2.minAreaRect(coords)[-1]
    if angle < -45: angle = 90 + angle
    (h, w) = thresh.shape
    center = (w // 2, h // 2)
    M = cv2.getRotationMatrix2D(center, angle, 1.0)
    rotated = cv2.warpAffine(thresh, M, (w, h),
        borderMode=cv2.BORDER_REPLICATE)
    return rotated

Tesseract Configuration

With preprocessed images, I configured Tesseract with custom PSM (Page Segmentation Mode) and whitelisted characters specific to jewellery labels (digits, decimal points, "g" for grams, "K" for karat):

import pytesseract

custom_config = r'--psm 6 -c tessedit_char_whitelist=0123456789.gGKk%₹'
text = pytesseract.image_to_string(preprocessed, config=custom_config)

# Parse extracted text into structured data
import re
weight = re.search(r'(\d+\.?\d*)\s*g', text)
purity = re.search(r'(\d+)K', text)
price = re.search(r'₹(\d+)', text)

Results & Accuracy

• 94.2% overall accuracy (up from 60% without preprocessing)
• Processed 10,200 labels in 1 hour 47 minutes
• Manual verification queue for low-confidence results (<80%)
• Saved approximately 160 person-hours of manual data entry

Lessons Learned

The biggest insight: OCR accuracy is 80% preprocessing, 20% the actual OCR engine. Invest time in image cleanup — it compounds. Also, always build a confidence threshold; automatic extraction should flag uncertain results for human review rather than silently producing wrong data.