Text Recognition From Crops

Text recognition reads a cropped word strip left to right, turning pixel columns into characters without anyone cutting the letters apart first.

Eight frame predictions collapse through CTC into the four letters of halo, showing how a strip becomes a string with no letter cuts.

Why Does This Exist?

Cutting a word into letters before reading fails exactly where it matters: cursive joins every letter, cheap print touches, and stylized fonts share strokes between neighbours. Any segment-then-classify pipeline inherits every cut error as a reading error. Meanwhile text detection only delivers whole-word strips, so something must turn an uncut strip into a string. Text recognition is that second half of OCR: a CNN reads visual columns left to right, a sequence layer adds context, and a decoder emits characters. Receipt scanners, license-plate readers and handwriting digitization all depend on it.

This page covers strip-to-string decoding, CTC collapsing with a hand-traced example, and attention decoders. Box-finding itself lives on the detection page.

Think of It Like This

Reading a timetable through a slit

Slide a narrow slit across one row of a train timetable and call out what passes: digit, dash, blank, digit. You never cut the row apart; the position of the slit plus your memory of what came before tells a smudged 3 from an 8. CTC decoding works like the caller, collapsing repeats and blanks into the final string, while attention decoding glances back at the most informative columns before speaking each character.

The analogy stops at training. The caller follows rules someone taught them, while the network learns its reading from thousands of strip-and-transcript pairs with no letter positions marked.

How It Actually Works

The strip-to-sequence pipeline

The input strip $X$ has fixed height $H$ and width $W$ . A CNN compresses it into a feature sequence of length $T$ (one vector per column group), and optional recurrent or transformer layers mix context across positions so each frame knows its neighbours. The decoder then maps $T$ frame predictions to a string over an alphabet $\Sigma$ plus a blank symbol $\varnothing$ meaning "no character here". CTC (connectionist temporal classification) trains this mapping with only the transcript as supervision: it sums over all frame-level paths $\pi$ that collapse to the transcript, so no character positions are ever labelled.

Worked CTC collapse

Eight frame predictions for a strip read $h, h, \varnothing, a, a, l, l, o$ . CTC decodes in two steps. First merge consecutive repeats inside each blank-separated run: $h,h$ becomes $h$ , $a,a$ becomes $a$ , $l,l$ becomes $l$ , giving $h, \varnothing, a, l, o$ . Then drop the blank $\varnothing$ , yielding "halo". Fourteen raw outputs were never needed; eight frames compress into four characters with no position labels supplied. The same rule turns the doubled predictions that wide letters cause into single characters instead of stutters.

CTC versus attention decoders

CTC assumes frames are conditionally independent given the features, which makes training fast and parallel but weak on strongly curved or reordered text. Attention decoders emit one character at a time while weighting the relevant columns, which handles irregular layouts and implicit language modelling at higher compute cost and slower sequential inference. Short clean strips go CTC; bent, artistic or heavily occluded text goes attention.

Code

1def ctc_collapse(frames, blank="-"):2    merged = []3    for frame in frames:4        if not merged or merged[-1] != frame:5            merged.append(frame)6    return "".join(f for f in merged if f != blank)7
8print(ctc_collapse(["h", "h", "-", "a", "a", "l", "l", "o"]))9# -> halo10print(ctc_collapse(["b", "o", "-", "o", "k"]))11# -> book

Watch Out For

A charset that never saw your symbols

A model trained on English alphanumerics has no output class for accents, currency signs or rare glyphs, so it substitutes confidently: every é becomes e across the whole document. The symptom is systematic character replacement, not random noise. Extend the alphabet $\Sigma$ and fine-tune on lines containing your real symbols before trusting a new deployment.

Genuine doubles collapsed away

CTC merges repeats, so the word "book" decodes as "bok" unless the model predicts a blank between the two o's. The symptom is dropped double letters in otherwise clean reads. During training this resolves itself from data, but at inference time never strip blanks with a naive deduplication pass: the blank between identical neighbours carries meaning.

The Quick Version

Recognition reads whole word strips into strings, never pre-cut letters.
CNN columns plus sequence context feed CTC or attention decoders.
CTC trains on transcripts alone by summing over all collapsing paths.
Eight frames $h, h, \varnothing, a, a, l, l, o$ collapse to "halo".
The training charset must contain every symbol the deployment will meet.