Allow private GIT repo in pip_requeriments

[jbianquetti]

This is my application definition. All code is in a private GIT server, so I need to use application_git cookbook who provides deploy_key resource to checkout code successfully.

application  "/data/envs/app_name" do
   application_git "/data/envs/app_name" do
         repository  "[email protected]:atomic/app_name.git"
         revision 'develop'
         deploy_key key
   end
   virtualenv
   pip_requirements "/data/envs/app_name/requeriments/development.txt"
   django do
      database do
              engine 'mysql'
              username 'someuser'
              password 'somepassword'
              host     'localhost'
        end
   end
  gunicorn do
        app_module 'django'
        port 8100
        timeout 800
  end
end

So far, so good. The problem arise when pip_requirements finds this line into requeriments/development.txt, who points to an egg archive into another private GIT repo:

-e git+ssh://[email protected]/atomic/puzzlequestserver.git@develop#egg=pqs

In this case, the command fails because lack of permissions in the GIT repo

  Permission denied, please try again.
  Permission denied, please try again.
  Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).
  fatal: Could not read from remote repository.

  Please make sure you have the correct access rights
  and the repository exists.
  Command "git clone -q ssh://[email protected]/atomic/puzzlequestserver.git /data/envs/app_name/.virtualenv/src/pqs" failed with error code 128 in None
       ---- End output of ["/data/envs/app_name/.virtualenv/bin/python", "-m", "pip.__main__", "install", "--requirement", "/data/envs/app_name/requeriments/development.txt"] ----
       Ran ["/data/envs/app_name/.virtualenv/bin/python", "-m", "pip.__main__", "install", "--requirement", "/data/envs/app_name/requeriments/development.txt"] returned 1

How can I install this egg if cookbook don't provides this functionality ?

[coderanger]

You have to solve this the same way you normally would with pip install -r. Usually this is through an SSH config file, but you could also use the GIT_SSH environment variable.

[zepheiryan]

As this is one of the first results when searching on the topic, I wanted to note that pip_requirements doesn't appear to recognize the application's environment attribute and has no way to accept any directly (poise/poise-python#77). Perhaps because of how python_shell_out works?

In order to use GIT_SSH, I had to replicate a pip install using python_execute in conjunction with a deploy wrapper script and key instead of using pip_requirements at all.

  python_execute do
    command "-m pip.__main__ install -r requirements.txt"
    environment({
      "GIT_SSH" => ".deploy_wrapper"
    })
  end

and

#!/usr/bin/env bash
# .deploy_wrapper
/usr/bin/env ssh -o StrictHostKeyChecking=no -i ./.ssh_deploy_key $@

and .ssh_deploy_key is an SSH key used for deployment with your private repository.

The wrapper is necessary as git will try to use the entire value of GIT_SSH as a command instead of a command with arguments. You may need to adjust permissions and directories according to your setup.

Note that this executes on every Chef client run, so it sends a restart to every service on every Chef client run. This may also not be acceptable for your purposes.