Cannot run make requirements from an empty virtualenv for cookiecutter-django-ida

[pshiu]

The Makefile target piptools, which installs pip-sync, was removed from cookiecutter-django-ida (but not other cookiecutters) in [link to line]. This means that make requirements in a clean virtualenv will fail for that cookiecutter's outputs.

The other cookiecutters vary in how they install the pip-tools package.

Acceptance criteria:

• There is a pre-requirements target (or similar) that installs pip and pip-tools pin files, as a dependency of the upgrade and *requirements targets.
  • fix: Add and depend on piptools target from all IDA make targets #330
• Unit tests check that make requirements works in a fresh cookiecutter output
  • make upgrade is a separate post-setup manual step, so this work is deferred to Run make upgrade automatically when running a cookiecutter #337

[pshiu]

How each repo handles installing pip-tools / pip-sync via make requirements now:

• cookiecutter-django-app uses a piptools recipe
• cookiecutter-django-ida doesn't install it
• cookiecutter-xblock doesn't install it
• python-template runs pip install -r requirements/pip.txt and pip install -r requirements/pip-tools.txt in the make requirements recipe

Manually edited shell search:

edx-cookiecutters % git grep -n -A5 '^requirements:' **/Makefile
cookiecutter-django-app/{{cookiecutter.repo_name}}/Makefile:63:requirements: piptools ## install development environment requirements
cookiecutter-django-app/{{cookiecutter.repo_name}}/Makefile-64- pip-sync -q requirements/dev.txt requirements/private.*
--
cookiecutter-django-ida/{{cookiecutter.repo_name}}/Makefile:38:requirements: clean_pycrypto dev_requirements ## sync to default requirements
--
cookiecutter-xblock/{{cookiecutter.repo_name}}/Makefile:45:requirements: piptools## install development environment requirements
cookiecutter-xblock/{{cookiecutter.repo_name}}/Makefile-46-     pip-sync -q requirements/dev.txt requirements/private.*
--
python-template/{{cookiecutter.placeholder_repo_name}}/Makefile:61:requirements: ## install development environment requirements
python-template/{{cookiecutter.placeholder_repo_name}}/Makefile-62-     pip install -r requirements/pip.txt
python-template/{{cookiecutter.placeholder_repo_name}}/Makefile-63-     pip install -r requirements/pip-tools.txt
python-template/{{cookiecutter.placeholder_repo_name}}/Makefile-64-     pip-sync requirements/dev.txt requirements/private.*

[timmc-edx]

Hmm! Yeah, make requirements should be installing the pip and piptools pinning files first, one way or another. (There are a few ways that we have gone about this in various repos.) I also would have expected CI to catch this, as we do have automated tests that use the cookiecutters and then run tests in the new virtualenv. Perhaps that virtualenv is dirty somehow, though? I'll add that to the acceptance criteria.

[timmc-edx]

@pshiu I think the original scope of the issue is complete; cookiecutter-django-ida now has a working make requirements. However, I did want to check on something -- one thing about the cookiecutters is that you need to run make upgrade after using the cookiecutter, which is annoying manual step, and would also result in make requirements not working. Is that something you ran into? I have it listed in the A/C of this issue to look into making that automated (probably as a separate issue), but wanted to check first if that was something you encountered as a pain point.

[pshiu]

@timmc-edx Hooray! Thanks for taking on this work!

I never found make requirements inoperable after a make upgrade, but I was so frustrated back then & was so furiously running manual installs of pip that I'm sure I might have missed that being the case.

[timmc-edx]

Filed #337 for automatically running make upgrade