Audit and clean up edx-platform requirements

[kdmccormick]

Background

As part of openedx-unsupported/wg-developer-experience#146, we want to optimize pip install for edx-platform. The more packages that edx-platform requires, the longer it will take pip install to run, the more bandwidth it'll use, and the larger the resulting image will be.

Tasks

Find requirements that could either be (a) removed from requirements, or (b) moved from base requirements out to one of the other requirements contexts, like development.

Some ideas:

• Devise a way to check whether each edx-platform requirement (that is, the Python requirement lists in requirements/, and the JS requirement lists in package.json) is actually being used by the application.
• Manually scan the list of requirements.:
  • Look for things that seem like they might be unused.
  • Look for things in base.txt that look like they belongs in development.txt.
  • Look for things in package.json that are marked as dependencies but should be devDependencies.
  • Check out the "via" comments, which tell you where the requirements are coming from--oftentimes, a change may need to be made in a package repository in order to remove a transitive dependency.

As you find requirements that could be removed or moved, either make PRs yourself or write up an issue for our backlog. Add those PRs and issues here:

• Remove libsass from base requirements
  • Rearrange requirements files to avoid installing libsass in production #31766
  • Remove libsass from base requirements for edx-ora2 edx-ora2#1986
• Don't install squire and sinon into production asset build #31768

[bradenmacdonald]

In https://discuss.openedx.org/t/a-minimal-open-edx-distribution/9478 I mentioned some examples:

algoliasearch       # Algolia’s API client, used for "Learner Recommendations"
analytics-python    # Used for Segment analytics
edx-braze-client    # "Customer Engagement Platform"
optimizely-sdk      # Optimizely full stack SDK for Python
py2neo              # Neo4j graph DB driver used for Coursegraph
edx-enterprise      # Might be too hard to remove though
crowdsourcehinter-xblock
recommender-xblock               # https://github.com/edx/RecommenderXBlock
done-xblock
xblock-google-drive              # XBlock for google docs and calendar
xblock-poll                      # Xblock for polling users

These in turn introduce 20+ other python dependencies that would no longer be part of the base install.