Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
khufkens authored Oct 13, 2023
1 parent fadf217 commit 4ec1c8c
Showing 1 changed file with 3 additions and 1 deletion.
4 changes: 3 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ The [“Congo basin eco-climatological data recovery and valorisation (COBECORE,

Within the context of COBECORE handwritten text recognition happens within the very specific context of tabulated data. Given the context of this problem, i.e. snippets of tables with numbers, it can be approached solving a [captcha](https://en.wikipedia.org/wiki/CAPTCHA). With the "noise" of table cell boundary lines and other smudges on old paper this comparison is more than fitting.

Generally, solving captcha problems can be done using a RCNN + CTC loss setup. The Keras introduction into the [captcha problem](https://keras.io/examples/vision/captcha_ocr/) provides the baseline, to be expanded to [handwritten text](https://keras.io/examples/vision/handwriting_recognition/) in another demo - accounting for varying input image sizes. In a quick test I used the vanilla Keras handwritten text recognition code, adapted to the COBECORE climate data formatting (separate labels and images, some cropping) to learn to recognize the value of climate variables.
Generally, solving captcha problems can be done using a CRNN + CTC loss setup. The Keras introduction into the [captcha problem](https://keras.io/examples/vision/captcha_ocr/) provides the baseline, to be expanded to [handwritten text](https://keras.io/examples/vision/handwriting_recognition/) in another demo - accounting for varying input image sizes. In a quick test I used the vanilla Keras handwritten text recognition code, adapted to the COBECORE climate data formatting (separate labels and images, some cropping) to learn to recognize the value of climate variables.

A simple test on a subset of the data (~10K images, instead of the total dataset of ~350K) shows reasonable performance (see image below). Given these results the exercise should be expanded to the full dataset, including many more writing styles to increase model robustness.

Expand All @@ -21,6 +21,8 @@ A simple test on a subset of the data (~10K images, instead of the total dataset
- expand training data and matching testing
- generate true out of sample data (extracted from novel tables)
- consider image augmentation and contrast boosting options to increase accuracy
- other model setups simpleHTR
- layout detection instead of template matching
- ...

----
Expand Down

0 comments on commit 4ec1c8c

Please sign in to comment.