Remove Text From Image With Python From Scratch

How does it work?

Steps:

1. Take original image;
2. Detect words rectangles in image (with CRAFT-pytorch image);
3. Create words mask;
4. Delete text with Automated objects removal inpainter

Possible improvement:

1. Create more accurasy mask. For example create mask from words symbols, not from word rectangles (Create and train autoencoder by symbols);
2. Play with function create_craft_args - change args.text_threshold, args.low_text, args.link_threshold - it's can give better result;
3. Autotune edge-connect models;
4. Try another pretrained edge-connect models (psv/celeba/places2);
5. Try change args.model value in load_object_remover_config;
6. ...

Installation

Installation on *nix system:

1. Open console;
2. Run command git clone https://github.com/pnzr00t/remove-text-from-image (current repository URL);
3. Run command cd ./remove-text-from-image/ (cloned folder);
4. Run command bash ./install_project.sh (Downloading libs, and models);
5. Run command pip install -r ./requirements.txt;
6. Run main.py script, you can chage original image URL in function test_remover_func():. Output image will save in local folder ./results_images.

Installation and run FastAPI service:

1. Open console;
2. Run command git clone https://github.com/pnzr00t/remove-text-from-image (current repository URL);
3. Run command cd ./remove-text-from-image/ (cloned folder);
4. Run command bash ./install_project.sh (Downloading libs, and models);
5. Run command pip install -r ./requirements.txt;
6. Run command pip install -r ./requirements-fast-api.txt (modules for FastAPI service);
7. Run command for start up FastAPI service uvicorn app:app;
8. Remove text from image by HTTP request http://127.0.0.1:8000/image_remover/?url=https://img-9gag-fun.9cache.com/photo/axMNd31_460s.jpg (IP and port will print in console when you start up service step 7. url= -- URL to original image).

Installation and run FastAPI service with gunicorn:

1. Open console;
2. Run command git clone https://github.com/pnzr00t/remove-text-from-image (current repository URL);
3. Run command cd ./remove-text-from-image/ (cloned folder);
4. Run command bash ./install_project.sh (Downloading libs, and models);
5. Run command pip install -r ./requirements.txt;
6. Run command pip install -r ./requirements-fast-api.txt (modules for FastAPI service);
7. Run command for start up FastAPI service gunicorn -w 1 -k uvicorn.workers.UvicornWorker app:app --timeout 600 --max-requests 5;
8. Remove text from image by HTTP request http://127.0.0.1:8000/image_remover/?url=https://img-9gag-fun.9cache.com/photo/axMNd31_460s.jpg (IP and port will print in console when you start up service step 7. url= -- URL to original image).

Note: FastAPI with unicorn "eat" a lot of memory and have memory leak, thats why you can use gunicorn service, witch will restart and clean memory every --max-requests COUNT_REQUEST

Run in google colab:

1. Go https://colab.research.google.com/ ;
2. File->Open notebook->Git Hub;
3. Copy and paste URL for current repository;
4. Chose colab file;
5. Run all cells;
6. Copy and paste original image URL in special cell with "input_image_url" parameter.

More examples:

Different edge-connect models

Celeba/Places/PSV(Paris Street View)

FAQ

A lot of memory usage

This script use a lot of memory, so i recommended restart you service or use gunicorn version (Install gunicorn service and run)

Uvicorn service memory leaks

Use gunicorn version and set properly --max-requests COUNT COUNT parameter (according of you RAM capacity)

Can script working faster

Yes, but you need torch lib with GPU, script automatically detecting you GPU device and run on them. (Additional info: At this time it must be NVIDIA GPU with cuda drivers and >= 4GB RAM)

Additional info and links

Create as part of dlschool.org project. Deep Learning School -- organization supported by PSAMI MIPT and Lab of Innovation (MIPT).

MIPT links:

• MIPT official site;
• MIPT Stepik DLS course;
• Deep learning school

Based on

• CRAFT text detecting;
• Automated objects removal inpainter;
• Generative Image Inpainting with Adversarial Edge Learning as part of Automated objects removal inpainter

Special thanks

• Artem Chumachenko - project MIPT curator;
• Viktor Savin - This project teammate

License

Licensed under a Creative Commons Attribution-NonCommercial 4.0 International.

Except where otherwise noted, this content is published under a CC BY-NC license, which means that you can copy, remix, transform and build upon the content as long as you do not use the material for commercial purposes and give appropriate credit and provide a link to the license.