Review/sanity check on a custom framework

[gregd33]

I've created a custom framework for EasyOCR https://github.com/JaidedAI/EasyOCR .

It seems to work but I just wanted to get a second pair of eyes to make sure it works. I was not quite sure how the metadata worked and I need to use those arguments.

The key part to know about EasyOCR is that a model is instantiated like:

model = easyocr.Reader(lang_list=lang_list, model_storage_directory=path, gpu=gpu,
                download_enabled=False, recog_network=recog_network) 

lang_list, recog_network and gpu are potentially things a service using this artifact will customize. The first two determine what actual file artifacts are needed. model_storage_directory is where to read weights from.

So basically my Artifact takes care of those downloaded weights and wrapping the predict endpoint.

I'm happy to contribute this as a framework if it is useful.

import os

from bentoml.exceptions import MissingDependencyException
from bentoml.service.artifacts import BentoServiceArtifact
from bentoml.service.env import BentoServiceEnv

class EasyOCRArtifact(BentoServiceArtifact):
    """
    Abstraction for saving/loading EasyOCR models
    Args:
        name (str): Name for the artifact
    Raises:
        MissingDependencyError: easyocr>=1.3 package is required for EasyOCRArtifact 
    """

    def __init__(self, name):
        super(EasyOCRArtifact, self).__init__(name)

        self._model = None

    def set_dependencies(self, env: BentoServiceEnv):
        env.add_pip_packages(["easyocr>=1.3.0"])

    def _model_file_path(self, base_path):
        return os.path.join(base_path, self.name)

    def _parse_metadata(self):
        #this is where i'm unsure a bit. But this particularly is setting some defaults. 
        try:
            recog_network =  self._metadata['recog_network']
        except:
            recog_network = "english_g2"

        try:
            lang_list = self._metadata['lang_list']
        except:
            lang_list = ['en']
        
        try:
            gpu = self._metadata['gpu'].astype(bool)
        except:
            gpu = False

        self._recog_network = recog_network
        self._lang_list = lang_list
        self._gpu = gpu


    def pack(self, easyocr_model, metadata=None):  # pylint:disable=arguments-differ
        try:
            import easyocr  # noqa # pylint: disable=unused-import
            assert easyocr.__version__ >= "1.3"
        except ImportError:
            raise MissingDependencyException(
                "easyocr>=1.3 package is required to use EasyOCRArtifact"
            )
        self._model = easyocr_model
        self._metadata = metadata
        self._parse_metadata()

        return self

    def load(self, path):
        try:
            import easyocr  # noqa # pylint: disable=unused-import
            assert easyocr.__version__ >= "1.3"
        except ImportError:
            raise MissingDependencyException(
                 "easyocr>=1.3 package is required to use EasyOCRArtifact"
            )

        self._parse_metadata()
        model = easyocr.Reader(lang_list=self._lang_list, model_storage_directory=path, gpu=self._gpu,
                download_enabled=False, recog_network=self._recog_network) 

        return self.pack(model)

    def get(self):
        return self._model

    def save(self, dst):
        # here i move files to the bento path. I hardcode one thing which is the text detector. probably not ideal but i can change that`  
        from shutil import copyfile
        src_folder = self._model.model_storage_directory
    
        if not os.path.exists(os.path.join(dst, "craft_mlt_25k.pth")):
            copyfile(os.path.join(src_folder, "craft_mlt_25k.pth"), os.path.join(dst,"craft_mlt_25k.pth"))

        fname = self._recog_network+".pth"
        copyfile(os.path.join(src_folder, fname), os.path.join(dst,fname))

    def predict(self, image_array):
        return self._model.readtext(image_array)

and this is how I would use it:

service = EasyOCR()

model = easyocr.Reader(lang_list=lang_list, model_storage_directory=model_storage_directory, gpu=False,
  download_enabled=True, recog_network=recog)

service.pack("my_model", model, metadata={"lang_list": lang_list, "recog_network":recog})

saved_path = service.save()

[yubozhao]

Hey @gregd33 This is awesome. I think this would be a great addition to BentoML.

I think there are couple things we can do.

1. I think we could just provide the Reader instance to end users. They can decided to call different methods on it like readtext or recognize

2. I think there are two paths that users can use to configure about the reader

   a. Users set the configuration they need when they write the BentoService.

   # my_service.py
   import bentoml
   @bentoml.artifacts([EasyReader('model', lang_list=[], recog_network='', is_gpu=False)])
   class MyService(bentoml.BentoService)

   For this approach, this is an open ticket that tracking an issue with artifact initialization Loading SKLearn model with "mmap_mode=r", raise ValueError: buffer source array is read only,  #1219

   b. Users set the configuration when they packing and saving the bento bundle.

   from my_other_service import MyOtherService
   svc = MyOtherService()
   svc.pack('model', model, recog_network='abc', lang_list=[], is_gpu=False)
   svc.save()

   For this approach, you might have to preserve the user's configuration (maybe with pickle) and then make sure we load those configuration during the load method.

Let me know what you think, happy to discuss further.

[gregd33]

For 1. I agree. This is what I originally had done but it didn't seem to work but I didn't try at all to debug since I was just trying to get things working. So I can try again.

For 2. I do think b) would be better. It seems my metadata approach is close (I looked at Detectron for a bit of a guide) but I don't think I'm fully sure about how the metadata works. Or am I misunderstanding the purpose of it.

I also think I will have to do a little improvement to the detector path (hardcoded as craft_mlt_25k.pth) but should be easy.

In any case, I will try to put a PR for this if I get a chance

[yubozhao]

For 1. I agree. This is what I originally had done but it didn't seem to work but I didn't try at all to debug since I was just trying to get things working. So I can try again.

Let me know any issues when you tryst out. Happy to help

For 2. I do think b) would be better.

Totally agree 🙌

It seems my metadata approach is close ... but I don't think I'm fully sure about how the metadata works. Or am I misunderstanding the purpose of it.

I think it could be confusing, we need to do better with our documentation. The metadata designed for users(human) able to identifying about the model or any other relevant information that's helpful for them in the workflow.

I looked at Detectron for a bit of a guide

That's great. I think for detectron, it is using the extra option input_model_yaml only for saving. For EasyOCR, we will need to preserve those configuration options during our loading. I think the best approach is to pickle those configuration in the same model directory and we can load from there.

[yubozhao]

@gregd33 Let me know anything I can help with to move this forward.

Love to get this merge in and help the community

[gregd33]

Started a PR here, need to clean up and add tests/docstring but I think the artifact is working.

#1616

I used JSON instead of Pickle. I can use Pickle if its preferred but I tend to avoid pickling since it can be picky with versions etc. Since the stuff is easily saved as JSON I stuck to that.