The model loads first. Then type a line, write one, or pick a sample.
The line, and where the model looks for the selected word piece What it wrote, one word piece at a time · click a pieceRead a line on the first tab, then click a patch here.
Each small square is one attention head: where the selected patch gathers information from. Heads specialise: some look only at the patch itself or its neighbours, many gather the whole column it sits in (the same stretch of the line, from top to bottom), and some spread over all the ink. Click one to show it large.
CTC is forced to move left to right. TrOCR is free to look anywhere, yet a left-to-right diagonal appears on its own: each piece looks at the next stretch of the line. This is the line from the first tab, cross-attention summed over the image's height.
Attention can lose its place in ways CTC cannot: it can skip or repeat words, or invent plausible text, and it squashes every line into the same 384 pixels. Run the real model on these and see what happens.