[Migrated] Failed to find libmysqlclient.so.18 when using slim_handler

[jneves]

Originally from: Miserlou/Zappa#2208 by NomeChomsky

This issue has been posted here and here but so far no fix.

When trying to use Postgres (psycopg2 and psyopg2-binary) it seems Zappa still tries to import libmysqlclient.so.18, which it cannot find, causing the program (in this case, Flask migration scripts) to hang, then exit.

This is a breaking issue for deployment It means that Flask projects connecting to Aurora using psycopg2 binaries cannot be deployed as this issue stops database migrations/interactions

The library should either be available automatically, or not included if its not needed. The fact the library cannot be found seems to create a breaking error.

Yes

Expected Behavior

Running a Flask-Migrate command works locally (with my local dependencies) works correctly. But when I call that command on the server via zappa, I get a breaking error which causes the program to hang. I expect to be able to run the command on the server the same way/in the same environment as I can run it locally. I'm expecting my command to work - it doesn't.

Actual Behavior

A 30 second command timeout, with this in the tail:
[1612507108211] Instancing..
[1612507108215] Failed to find library: libmysqlclient.so.18 ...right filename?

If I add "include": []" to settings.json, the problem persists. This fix was suggested in the above mentioned threads, but it has not worked.

The line in question is below - see how it does not allow for overriding of include: [] with nothing.

https://github.com/Miserlou/Zappa/blob/ba20c850eeca00edd6ea39fda1ab976cfee193ea/zappa/handler.py#L105

Possible Fix

There is a possible fix for the problem listed here

                include = self.stage_config.get('include', [])
                if include is None:
                    settings_s += 'INCLUDE=[]\n'
                elif len(include) >= 1:
                    settings_s += "INCLUDE=" + str(include) + '\n'

Steps to Reproduce

Create a flask app which uses Postgres db, connect to it with Flask-sqlalchemy, write a script which wraps Flask-Migrate commands so they're callable from zappa, and invoke either migrate() or update() - anything that uses the postgresdb.

Your Environment

• Zappa version used:
  {
  "dev": {
  "app_function": "app.app",
  "aws_region": "us-east-1",
  "profile_name": "default",
  "project_name": "zask",
  "runtime": "python3.8",
  "s3_bucket": "zappa-mlrijq5z6",
  "environment_variables": {
  "POSTGRES_USER": "
  "POSTGRES_PASSWORD":
  "POSTGRES_URL":
  "POSTGRES_DB":
  "POSTGRES_PORT":
  "ENV":
  },
  "slim_handler": "true",
  "include": [
  ""
  ]
  }
  }

[john-parton]

I'm having the same issue.

I'm using zappa==0.53.0 and
python 3.8.11

[sumit1201]

facing the same issue
pyhotn 3.7
zappa=0.52.0

[atindersingh67]

Facing same issue. Any solution you guys found.

[jottenlips]

facing this same issue on 0.54.0

[jottenlips]

I have a PR up to resolve this for a lot of use cases like people not using MySQL. #1060. A workaround, for now, is include: [] in Zappa settings. So it won't look for "libmysqlclient.so.18".

https://pypi.org/project/lambda_packages/

This package contains "libmysqlclient.so.18" from the include part of the code. It was removed in this release Miserlou/Zappa@f4e846f.

Another option if you are using MySQL is to switch to PyMySQL, a python-only implementation, and set include: [] so you will no longer rely on "libmysqlclient.so.18".

Hope this helps!

[jottenlips]

And this is why lambda-packages were removed? Miserlou/Zappa#398, blog post has been deleted though 🙃

[monkut]

considered supported in #1060 closing.