Skip to content
This repository has been archived by the owner on Apr 18, 2018. It is now read-only.

Added --always-copy to virtualenv call #55

Open
wants to merge 2 commits into
base: develop
Choose a base branch
from

Conversation

justquick
Copy link
Contributor

This make it so that the files in the vent are copied from the source
machine when creating the jar files. If this is not present, recent versions
of virtualenv will symlink and cause bad links in the jar files. Bad links
mean that the files will not exist when the jar is actually run. This fix
ensures that all required files in the venv are copied over so you do not
actually have to have python installed on your pyleus cluster.

This is the root cause of the ImportError: No module named 'encodings' problem in #50

This make it so that the files in the vent are copied from the source 
machine when creating the jar files. If this is not present, recent versions 
of virtualenv will symlink and cause bad links in the jar files. Bad links 
mean that the files will not exist when the jar is actually run. This fix 
ensures that all required files in the venv are copied over so you do not 
actually have to have python installed on your pyleus cluster.

This is the root cause of the `ImportError: No module named 'encodings'` problem in Yelp#50
@patricklucas
Copy link
Contributor

How does this affect the size of a Pyleus jar?

@justquick
Copy link
Contributor Author

in testing the example exclamation_topology it's an increase of ~15% (25MB vs 30MB)

@patricklucas
Copy link
Contributor

This is something I'd like to bring in, but I'm concerned about JAR bloat—I'm going to open another ticket regarding the removal of the scala/ and kafka/ dirs from the JAR if the user is not using a Kafka spout. I would prefer to have that change in the same release or earlier as this one.

@patricklucas
Copy link
Contributor

Opened #57.

@justquick
Copy link
Contributor Author

@patricklucas I could add a "--always-copy" flag to pyleus build which would pass that flag onto virtualenv when initing the environment. it could b optional since only in specific cases (eg python3.4) is this absolutely required to get the stdlib installed and functioning properly. we can trust that users are smart enough to use this flag when running pyleus build only when they absolutely need to.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants