Skip to content

edX Managing the Production Stack

Jump to bottom Edit New page
John Jarvis edited this page Aug 27, 2014 · 70 revisions

NOTE: These are instructions for the Production Stack configuration, not the Developer Stack.

Managing the server

manage.py commands

login to the server and change to the /edx/app/edxapp/edx-platform directory to use manage.py

  • list all manage.py commands - sudo -u www-data /edx/bin/python.edxapp ./manage.py lms --settings aws help

  • create a final grade and generate a certificate for a single user - sudo -u www-data /edx/bin/python.edxapp ./manage.py lms --settings aws regenerate_user -u [email protected] -c edX/Open_DemoX/edx_demo_course

  • create a new user - sudo -u www-data /edx/bin/python.edxapp ./manage.py lms --settings aws create_user -e [email protected]

  • set or change password - sudo -u www-data /edx/bin/python.edxapp ./manage.py lms --settings aws changepassword user

  • set user to staff - sudo -u www-data /edx/bin/python.edxapp ./manage.py lms --settings aws set_staff [email protected]

  • launch the django shell - sudo -u www-data /edx/bin/python.edxapp ./manage.py lms --settings aws shell

  • create superuser -

    sudo -u www-data /edx/bin/python.edxapp ./manage.py lms --settings aws create_user -e [email protected]
sudo -u www-data /edx/bin/python.edxapp ./manage.py lms --settings aws changepassword user
sudo -u www-data /edx/bin/python.edxapp ./manage.py lms --settings aws shell

from django.contrib.auth.models import User
me = User.objects.get(username="user")
me.is_superuser = True
me.is_staff = True
me.save()

  • importing a course from github

Example using the demo course

cd /var/tmp
git clone https://github.com/edx/edx-demo-course.git
cd /edx/app/edxapp/edx-platform
sudo -u www-data /edx/bin/python.edxapp ./manage.py cms --settings=aws import /edx/var/edxapp/data  /var/tmp/edx-demo-course

  • run migrations

    • LMS: sudo -u www-data /edx/bin/python.edxapp ./manage.py lms syncdb --migrate --settings aws
    • CMS: sudo -u www-data /edx/bin/python.edxapp ./manage.py cms syncdb --migrate --settings aws

  • delete course - sudo -u www-data /edx/bin/python.edxapp ./manage.py cms --settings aws delete_course Organization/CourseNumber/CourseNumber commit

See help and help <cmd> for more information!

Use runserver instead of gunicorn (helpful for debugging)

  • Shutdown the lms and cms (and optionally the workers)
sudo /edx/bin/supervisorctl -c /edx/etc/supervisord.conf
> stop edxapp: edxapp_worker:
  • Launch the lms using the debug server
cd /edx/app/edxapp/edx-platform
sudo -u www-data /edx/bin/python.edxapp ./manage.py lms runserver 8000 --settings aws

When you connect your browser requests will go to the debug server instead of gunicorn through nginx. You will still need to compile static assets of they change

See what services are running

sudo /edx/bin/supervisorctl -c /edx/etc/supervisord.conf status

you should see something like the following:

certs                            RUNNING    pid 19862, uptime 0:00:04
discern                          RUNNING    pid 19926, uptime 0:00:02
discern_celery                   RUNNING    pid 19857, uptime 0:00:04
edxapp:cms                       RUNNING    pid 19884, uptime 0:00:03
edxapp:lms                       RUNNING    pid 19879, uptime 0:00:03
edxapp_worker:cms_default_4      RUNNING    pid 19915, uptime 0:00:03
edxapp_worker:cms_high_1         RUNNING    pid 19901, uptime 0:00:03
edxapp_worker:cms_low_3          RUNNING    pid 19890, uptime 0:00:03
edxapp_worker:lms_default_3      RUNNING    pid 19897, uptime 0:00:03
edxapp_worker:lms_high_4         RUNNING    pid 19922, uptime 0:00:03
edxapp_worker:lms_high_mem_2     RUNNING    pid 19894, uptime 0:00:03
edxapp_worker:lms_low_1          RUNNING    pid 19908, uptime 0:00:03
forum                            RUNNING    pid 19855, uptime 0:00:04
ora                              RUNNING    pid 19875, uptime 0:00:04
ora_celery                       RUNNING    pid 19866, uptime 0:00:04
xqueue                           RUNNING    pid 19853, uptime 0:00:04
xqueue_consumer                  RUNNING    pid 19860, uptime 0:00:04
xserver                          RUNNING    pid 19854, uptime 0:00:04

Restarting Services

  • LMS/CMS - sudo /edx/bin/supervisorctl -c /edx/etc/supervisord.conf restart edxapp:
  • Workers - sudo /edx/bin/supervisorctl -c /edx/etc/supervisord.conf restart edxapp_worker:

(Note: the colon character : at the end of the line is mandatory!)

Updating Versions using edX repos

Use the /edx/bin/update to update repo versions on the server.

Allowed repo names [edx-platform, xqueue, cs_comments_service, xserver, ease, edx-ora, configuration, read-only-certificate-code]

Examples:

  • sudo /edx/bin/update edx-platform master - change edx-platform to use the latest on the master branch
  • sudo /edx/bin/update cs_comments_service master - change the forums service to use the latest on the master branch

Generate certificates after a course ends

Note: the [email protected] and [email protected] users have been pre-added to the certificate whitelist in the vagrant image so you can generate a certificate even if they have not passed the course

  • Certificates should only be generated after the course ends. The reason for this is because we store a final grade in the certificate table. To ensure your course has ended login to studio and confirm the end date end date
  • Once the end date has passed for a course the users will see that 'final course details' are being wrapped up. final course details
  • Next you will can either run the ungenerated_certs django admin command or the regenerate_user django admin command to generate certificates for an entire course or for a single user. This command does the following:
  • Grade the student
  • Update the generated certificate table with the students score and certificate status
  • Send a certificate request to the certificate process
  • The certificate server will generate a pdf, sign it and copy it to the certificate publish directory
  • The student's dashboard will update to show the link to the pdf and the validation page.

cert download

These are the django commands to generate certificates for an entire course or a single student. Note the --insecure flag which is so that http is used for the lms callback url. It is not necessary if https is supported on the lms.

# Generate or regenerate a certificate for a single user in a course
sudo -u www-data /edx/bin/python.edxapp ./manage.py lms --settings aws regenerate_user -u [email protected] -c edX/Open_DemoX/edx_demo_course --insecure
# Grade and generate certs for all users in a course
sudo -u www-data /edx/bin/python.edxapp ./manage.py lms --settings aws ungenerated_certs -c edX/Open_DemoX/edx_demo_course --insecure

Using an edx-platform fork

  • sudo rm -rf /edx/app/edxapp/edx-platform - to change the remote you need to remove the existing edx-platform checkout
  • Add the following line to /edx/app/edx_ansible/server-vars.yml - edx_platform_repo: "https://github.com/<user>/edx-platform.git"
  • sudo /edx/bin/update edx-platform master - update edx-platform using the forked repo

Compile assets manually

To compile javascript and css outside of the update script run the following commands:

  • sudo -H -u edxapp bash
  • source /edx/app/edxapp/edxapp_env
  • cd /edx/app/edxapp/edx-platform
  • paver update_assets cms --settings=aws
  • paver update_assets lms --settings=aws

Troubleshooting

Unable to connect to the LMS

Check to see if nginx is running:

  • sudo service nginx status
  • sudo service nginx start

NGINX logs are located in /edx/var/log/nginx NGINX configurations are located in /edx/app/nginx/sites-enabled

Check to see if the lms/cms is running

  • sudo /edx/bin/supervisorctl -c /edx/etc/supervisord.conf status edxapp:

If you are still unable to connect to the LMS run the following commands:

  • sudo service nginx stop
  • sudo service supervisor stop
  • sudo service supervisor.devpi stop
  • sudo pkill -u www-data
  • sudo service nginx start
  • sudo service supervisor start
  • sudo service supervisor.devpi start
  • stop everything except the LMS and CMS
sudo /edx/bin/supervisorctl -c /edx/etc/supervisord.conf stop discern discern_celery \
  forum ora ora_celery xqueue xqueue_consumer xserver edxapp_worker:
  • sudo tail -f /edx/var/log/{lms,cms,nginx}/*log

While tailing the log files try to connect your browser to the LMS or Studio

Log files

  • The production stack sends all application logs to rsyslog which writes config files to the /edx/var/log directory.
  • Supervisor logs stderr and stdout, its logs are located in /edx/var/log/supervisor

server-vars.yml does not make any changes after running the update bash script

Take a look at this line in the /edx/bin/update bash script to see where the server-vars.yml file should be:

  • This means you are using an older version of the configuration repo, so create and add your settings in /edx/var/edx_ansible/server-vars.yml:
if [[ -f /edx/var/edx_ansible/server-vars.yml ]]; then
    extra_args="-e@/edx/var/edx_ansible/server-vars.yml"
fi
  • The following line in /edx/bin/update means you are using the current version of the configuration repo. If so, create and modify your settings in /edx/app/edx_ansible/server-vars.yml:
if [[ -f /edx/app/edx_ansible/server-vars.yml ]]; then
    extra_args="-e@/edx/app/edx_ansible/server-vars.yml"
fi
  • Notes: this may be changed in the future
Add a custom footer
Add a custom sidebar
Clone this wiki locally