Recognition of handwritten text using CRAFT text detection and TrOCR

CRAFT(Character Region Awareness for Text Detection) is a way of doing OCR on images by exploring the region around the text. On high level the algorithm generates heatmap of the image and convolute upon them rather than on the image directly. Also for generating bounding boxes it will model respective thresholds with anchor points between individual character and individual words

TrOCR is essentially an encoder-decoder model, where encoder network creates an representation of the image using image encoding transformers models(ViT, DEiT) and the decoder network (language models) converts the processed repsentation into target strings.

from craft_hw_ocr import OCR
import cv2
import numpy as np

img = OCR.load_image('/content/example_1.png')

# do the below step if your image is tilted by some angle else ignore
# img = OCR.process_image(img)

ocr_models = OCR.load_models()

img, results = OCR.detection(img, ocr_models[2])

bboxes, text = OCR.recoginition(img, results, ocr_models[0], ocr_models[1])

pilImage = OCR.visualize(img, results)

Huggingface Space

craft-text-detector