OCR

In short

OCR stands for optical character recognition. It is the technique that takes the image of a text and returns that text as characters, so a system can treat it as data instead of pixels.

Applied to an identity document, the job does not end at recognizing letters. It ends at returning fields: this is the document number, this is the date of birth, this is the last name. Recognizing the characters is half the work; knowing what each group of characters means within that specific document is the other half.

That is why document reading is not solved with a generic recognizer. It requires knowing how each document type is laid out, in every version still in circulation.

How a field is extracted from a photo

The sequence is short, and each step can ruin the next one.

• Capture. A photograph of the document, taken with the user's camera, under whatever lighting happens to be available and from whatever distance occurs to them.
• Normalization. Cropping the document from the background, correcting perspective, straightening it and compensating contrast. This is what turns a skewed photo into a rectangular image comparable to a template.
• Document identification. Determining which document type and version it is, because the position of the fields depends on that.
• Field segmentation. Locating in the image the regions where each piece of data should be, according to the template of the identified document.
• Recognition. Converting the content of each region into characters.
• Validation. Checking that what came back has the right shape: that the date is a date, that the number follows the country's format, that the expiration date is later than the issue date.

Step 3 is the most underestimated one. A single country keeps several versions of its document in circulation at the same time, because older ones stay valid until they expire. Each version has its own layout, and the wrong template returns well-recognized data in the wrong field.

Why the MRZ reads better than the front

On the same document, recognition does not perform equally everywhere, and the difference is not accidental.

The MRZ is designed to be read by a machine. It uses a typeface meant to minimize confusion between characters, a reduced character set, fixed spacing and a stable position. It also carries check digits, so the data itself says whether it was read correctly.

The front of the document is designed to be read by a person. Variable typefaces, security-pattern backgrounds, reflective holograms, text over image, fields that move between versions. All of that is correct for its purpose and hostile to automatic recognition.

Hence a practical rule: when the same data is available in the strip or in the [barcode](/glosario/pdf417) on the back, that is the preferred source, and the front is used to cross-check it. Not the other way around.

Where it fails, and what to do about it

Recognition errors on documents are fairly predictable, and that predictability is what makes them manageable.

• Confusion between similar characters — zero and the letter O, one and the letter I, five and S, eight and B. It is the classic error and mostly affects document numbers.
• Capture conditions — glare on the plastic, an out-of-focus shot, low light, a finger covering a field. Most onboarding rejections start here.
• Document wear — a credential that is years old, scratched or faded, has fields that are no longer legible even to a person.
• Unrecognized version — a document whose layout the system does not know. It does not return a clear error: it returns misassigned fields, which is worse.
• Transliteration — names with accents, the letter ñ or non-Latin characters appear differently in the strip and on the front. Treating that difference as a discrepancy rejects legitimate documents.

What fixes these errors is not a better recognizer, it is having more than one source for the same data. The front-and-back cross-check catches a faulty reading the moment the two paths do not match, and the MRZ's check digits do the same without leaving the strip.

OCR reads, it does not verify

It is the most costly confusion on this topic, and it is worth spelling out.

Recognition returns what is printed. Nothing more. If the document was altered and the alteration is well made, recognition returns the altered data completely normally, because its job is to transcribe what it sees.

A completely fake document, with consistent data and a well-reproduced layout, produces a flawless extraction. The quality of the recognition says nothing about the legitimacy of the document.

What speaks to authenticity is something else: analyzing the security features of the physical document, the cryptographic signature of the chip when the document has one, the contrast between representations of the same data and, where the country allows it, checking against an official source. None of those checks is character recognition, and none can be replaced by it.

That is why OCR is one step within document verification and never its equivalent. And that is why the region's anti-money-laundering frameworks, such as SARLAFT in Colombia or the anti-money-laundering law in Mexico, require keeping evidence of how identity was checked, not just the extracted data.