Arabic Calligraphy

The github repository :

In this project we were a team of 3 students :

• Mustapha Ajeghrir
• Nouamane Tazi
• Asmae Khald

This project is separated in two parts, the first one is an introduction to the project and it aims to recognise the style of the Arabic calligraphy. The second part and the main part aims to recognize the characters and words from the calligraphy.

ACSR (Arabic Calligraphy Style Recognition):

Our implementation is highly inspired from ArbMl’s work and the data (synthetic and real) from Mahmoud Aslan’s collection. The main problem in ACSR (Arabic Calligraphy Style Recognition) is the scarcity of data. Hopefully, it is possible to generate data using digital fonts to help the training, the model is then tested on the real data reaching a score of 97% in accuracy. The used architecture is a simple CNN network (more on that in the full report in the end of this article). We have also managed to perform some saliency maps as shown in the figure below.

Demo

A demo could be accessed is the following HuggingFace spaces link : https://huggingface.co/spaces/mustapha/ACSR

OCR (optical character recognition):

The main inspiration of this part is TrOCR paper. It uses a Vision-Encoder-Decoder model. For the encoder we used the ViT model, which applies a pure transformer directly to sequences of image patches to classify the full image. Its embeddings are then passed to the decoder. This decoder uses the weights of AraBERT. The figure below shows the architecture of the model.

For A detailed description :

Below is our report for this project, please feel free to read it and email me if necessary.