diff --git a/DeploymentWeb/DeployForProduction.pillar b/DeploymentWeb/DeployForProduction.pillar index 7f263a4..974dc5d 100644 --- a/DeploymentWeb/DeployForProduction.pillar +++ b/DeploymentWeb/DeployForProduction.pillar @@ -1,45 +1,52 @@ -!Deploying a Pharo application for production +!Deploying a Pharo Web Application in Production -It is very easy to develop with Pharo and its ecosystem. Once your development needs to go into production to produce value, a lot of questions comes out of the box: which operating system should I use? how do I run my application? how do I ensure y application will be restarted after a reboot or a crash? how do I log? +In the previous chapters we discussed several frameworks and libraries to facilitate the development of web application. In this chapter, we will focus on deploying such a web application. While doing so, we will try to answer some questions such as: which operating system should I use, how do I run my application, how do I ensure my application will be restarted after a reboot or a crash, and how do I log data. -There are a lot of questions and even more answers but following sections will try to make things clearer and to provide one guideline that works. +!!Where to Host your Application? -!! Where to host your application? -The easiest (and fastest) way to host your application is to host it in the cloud. +The easiest (and fastest) way to host your application is to host it in the cloud. -""*PharoCloud>http://pharocloud.com*"" proposes pre-packaged solutions (including seaside and pier, database support) as well as the possibility to use your own image. You could start very quickly from there but you do not have full control on your pharo stack. It is enough in most cases: PharoCloud manages defaults for you. +*PharoCloud>http://pharocloud.com*, for example, proposes pre-packaged solutions (including Seaside and Pier, database support) as well as the possibility to use your own Pharo image. You could start very quickly from there but you do not have full control on your pharo stack. It is enough in most cases: PharoCloud manages defaults for you. -There are a lot of other cloud providers : *Amazon AWS>https://aws.amazon.com/*, *Openshit>https://www.openshift.com/*, *Microsoft Azure>http://azure.microsoft.com*, etc. Many pharoers use ""*DigitalOcean>https://www.digitalocean.com/*"" (simplicity, cheaper). +There are many other cloud providers including *Amazon AWS>https://aws.amazon.com/*, *Openshit>https://www.openshift.com/* and *Microsoft Azure>http://azure.microsoft.com*. Many Pharo users use *DigitalOcean>https://www.digitalocean.com/* as it is both simple and cheap. Choose your cloud provider according to your needs. -!! Which Operating System? -A lot of pharo developers uses OS X to develop their applications but it is not really a solution for a production deployment as there is no more Apple server hardware. Most pharoers deploy on Linux because ""Linux"" is well know to be stable even with months of up-time. It is also a simpler solution (comparing to Windows) in many aspects (remote access, configurability) and the most supported OS by cloud providers. +In the rest of this chapter, we detail how to setup a server to host a Pharo web application. -You also have many flavors of Linux distributions. If you restrict your choice to well-known open-source distributions, competitors are ""centos"", ""debian"" and ""ubuntu"". Anyone will do the job, choose the most appropriate for you. A long-term support (""LTS"") version is welcomed if you do not want to update your OS too often. +!!Which Operating System? -Last but not least, take care that Pharo provides a *virtual machine>http://pharo.org/gnu-linux-installation* for the selected Operating System if you do not want to compile the VM by yourself. A big problem is that, waiting for Spur64, the Pharo VM is 32 bits and requires 32 bits libraries not availaible by default on recent distributions. Check that instructions to install required 32 bits libraries are available on the Pharo web site. +A lot of pharo developers use Mac OS X to develop their applications but it is not really a solution for a production deployment as there is no more Apple server hardware. Most Pharo developers prefer to deploy on GNU/Linux because GNU/Linux stability is well known. Deploying on Windows is a bit more complex and less supported by cloud providers. -!! Build your image -The best option to obtain a clean image to deploy is to start from a fresh *stable pharo image>http://files.pharo.org/image/stable/latest.zip* and to install needed packages through your application Metacello configuration and the command line handler. The configuration has to explicitely describes all dependencies used in your application. +There are many Linux distributions you can choose from. If you restrict your choice to well-known open-source distributions, competitors are *Centos>http://www.centos.org*, *Debian>http://www.debian.org* and *Ubuntu>http://www.ubuntu.com*. Most distribution will do the job, choose the most appropriate for you. A long-term support (''aka.'', LTS) version is welcomed if you do not want to update your operating system too often. The Pharo Virtual Machine (VM) comes pre-packages for some distributions. For other distributions, you will have to compile the VM sources yourself. Pay attention that the Pharo VM is still 32bits as of 2015 and you will have to install 32-bit libraries on your 64-bit Operating System. + +!!Build your Image + +The best option to obtain a clean image to deploy is to start from a fresh *stable pharo image>http://files.pharo.org/image/stable/latest.zip* and to install required packages through your application's Metacello configuration and the command line handler. The configuration has to explicitely describes all dependencies used in your application. First, create a copy of the clean image with your application name: -[[[language=bash­ -$ ./pharo Pharo.image save myapp]]] + +[[[language=bash +$ ./pharo Pharo.image save myapp +]]] + Then, install your dependencies: -[[[language=bash­ +[[[language=bash $ ./pharo myapp.image config http://www.smalltalkhub.com/mc/Me/MyApp/main ConfigurationOfMyApp --install=stable =============================================================================== Notice: Installing ConfigurationOfMyApp stable =============================================================================== +[...] ]]] -After loading all necessary code, the config option will also save the image so that it now permanently includes your code. +After loading all necessary code, the ==config== option will also save the image so that the image now permanently includes your code. + +To make sure that your deployment image is reproducible, the best approach is to create a Continuous Integration job that automatically produces clean deployment-ready images of your application. + +!!Run your Application -The best approach for this is to create a Continuous Integration job to automatically produce clean images of your application, deployment-ready. +When you have a Pharo image with your application inside, the next step is to start the application. To make this process reproducible, it is recommended to create a dedicated file (''e.g.'', named ==myapp.st==) with the instructions needed to start your application. Here is an example of a script used to start a web application using Zinc. -!! Run your application -Although you could try to write a (long) one line expression to start your application in a server and pass it to the eval option of the pharo command, it is better to write a small script. Create a file called ‘myapp.st’ with the instructions needed to start your application. Here is an example of a script used to start a web application using Zinc [[[language=Smalltalk ZnServer defaultOn: 8080. ZnServer default logToStandardOutput. @@ -47,9 +54,14 @@ ZnServer default delegate map: 'image' to: MyFirstWebApp new; map: 'redirect-to-image' to: [ :request | ZnResponse redirect: 'image' ]; map: '/' to: 'redirect-to-image'. -ZnServer default start ]]] +ZnServer default start +]]] + +@@authorToDo Damien: explain what the above script does + You can test the startup script like this: -[[[language=bash­ + +[[[language=bash $ ./pharo myapp.image myapp.st 2013-07-10 11:46:58 660707 I Starting ZnManagingMultiThreadedServer HTTP port 8080 2013-07-10 11:46:58 670019 D Initializing server socket @@ -62,17 +74,18 @@ $ ./pharo myapp.image myapp.st 2013-07-10 11:47:12 909356 I Wrote a ZnResponse(200 OK text/html;charset=utf-8 282B) 2013-07-10 11:47:12 909356 I Read a ZnRequest(GET /image?raw=true) 2013-07-10 11:47:12 909356 T GET /image?raw=true 200 18778B 82ms -2013-07-10 11:47:12 909356 I Wrote a ZnResponse(200 OK image/png 18778B)]]] -Type ctrl-c to kill the server. +2013-07-10 11:47:12 909356 I Wrote a ZnResponse(200 OK image/png 18778B) +]]] + +Type ==Ctrl-c== to kill the server. + +In the following we present how to configure recovery, backup and logging. -You could use the nohup utility command to have a long-running process (nohup starts pharo in the background in such a way that the subsequent terminal logout does not stop it) but it is not a good solution as it does not support recovery . -[[[language=bash­ -$ nohup ./pharo myapp.image run.st &]]] -It is now time to put the server in production mode, running for real and robust to faults. +!!Recovery -!! Recovery -!!! Automatic start -One important aspect for application production is that they need to be up as much as possible. To limit downtimes, you should have ""a simple script to start/stop your application"". This script should be configured to be __run automatically at system startup__. On linux servers, it could be done with a simple Shell script that you place in the ''/etc/init.d'' directory. +!!!Automatic Start + +One important requirement for application production is that they need to be up as much as possible. To limit downtimes, you should have ""a simple script to start/stop your application"". This script should be configured to be __run automatically at system startup__. On linux servers, it could be done with a simple Shell script that you place in the ''/etc/init.d'' directory. You can find a template named *pharo-service-script.sh>https://github.com/pharo-project/pharo-deployment-scripts/blob/master/pharo-service-script.sh* in the *GitHub pharo-project/pharo-deployment-scripts repository>https://github.com/pharo-project/pharo-deployment-scripts*. Copy it to set up your application service script and give it the name of your application. The ""pharo-service-script.sh"" script is derivated from the template provided by the Ubuntu distribution (/etc/init.d/skeleton). It manages the launch in background, with a given user, of another script named ""pharo-run-script.sh"". @@ -90,18 +103,18 @@ You will end with a file hierarchy like this one: Then, you can test / run your application first by running the ==./myapp== script, then by using the service command (uses the pharo init script): ==service myapp start==, ==service myapp stop==. -!!! Monitoring +!!!Monitoring Your application could also crash or be killed by the system or alive but not responding to requests. In such cases, you would like to stop the application and restart it. To do so, you need to first detect if your application is running or not. It can be done with the previous script with the status command. If your application is a web application, you may wish to check if your application is able to answer to HTTP requests. To do so, you need to define some 'ping' url to your application that will be used by the monitoring tool. A simple solution to both monitor your application and take appropriate actions (e.g. restart) is to use the *monit tool>https://mmonit.com/* that can be easily installed: -[[[language=bash­ +[[[language=bash $ sudo apt-get install monit]]] Then, the configuration of monit takes place in the ''/etc/monit/monitrc'' file. -!!!! Monit global configuration -!!!!! Monit dashboard +!!!!Monit Global Configuration +!!!!!Monit Dashboard You can first activate the embedded HTTP dashboard of monit. We will only allow local connections with a given username/password. -[[[language=bash­ +[[[language=bash set httpd port 2812 and use address localhost # only accept connection from localhost allow localhost # allow localhost to connect to the server and @@ -110,28 +123,31 @@ Do not forget to apply the new configuration: [[[language=bash $ sudo monit reload]]] It does not mean that you will not be able to connect to the monit dashboard from another host. You will just need to use an SSH tunnel to achieve that. If the server running your application and monit is named myserver.com, then you could simply run -[[[language=bash­ +[[[language=bash $ ssh -L 2812:localhost:2812 myserver.com]]] Keep the SSH connection open, and then browse *http://localhost:2812>http://localhost:2812* to display the monit dashboard. -!!!!! Email settings +!!!!!Email Settings If you want notifications from monit, you need to configure email settings so that monit can send emails. Edit the ''/etc/monit/monitrc'' file and add the following lines: -[[[language=bash­ +[[[language=bash set mailserver ]]] -!!!! Monitor system services + +!!!!Monitor System Services + Configuration files related to an application should be put into the ''/etc/monit/monitrc.d'' directory (more modular that everything in the core configuration file. To enable a configuration just symlink it to conf.d. We will first enable pre-defined configuration for SSH. -[[[language=bash­ +[[[language=bash $ sudo ln -s /etc/monit/monitrc.d/openssh-server /etc/monit/conf.d/openssh-server $ sudo monit reload]]] ""Warning"": default configurations fro well-known services are provided by monit but may require some adaptations (ex: wrong path to the PID file). To check errors, you may need to run monit in verbose mode: -[[[language=bash­ +[[[language=bash $ sudo monit -v]]] and check the monit error log (default location is ''/var/log/monit.log''). -!!!! Monit configuration for a simple Pharo application +!!!!Monit Configuration for a Simple Pharo Application + First create a new file (update myapp with your application name) in ''/etc/monit/monitrc.d''. We will check that the application is running -[[[language=bash­ +[[[language=bash alert me@domain.com check process myapp with pidfile /var/run/myapp.pid @@ -143,16 +159,17 @@ check process myapp with pidfile /var/run/myapp.pid The ''alert'' instruction is used to be notified about all alerts related to this service. The monitoring is described with the check command. We ask monit to check a given pid file. If there is no pid or no process associated to the pid, monit will start the program with the given instruction. The last instruction prevents infinite loops if there is a problem with the script. Start the monitoring -[[[language=bash­ +[[[language=bash $ sudo ln -s /etc/monit/monitrc.d/myapp /etc/monit/conf.d/myapp $ sudo monit reload]]] You can now check if your app is running. To test the recovery, kill your application and check that it will be relaunch. -!!!! Monit configuration for a Pharo web application +!!!!Monit Configuration for a Pharo Web Application + Most of applications depoyed on servers are web applications. What may happens is that your Pharo image is running (processus alive) but your image is not anymore able to handle HTTP requests (one reason may be a resource limit exceeded like the number of opened sockets or any other reason). In such cases, your application becomes unusable. A useful check would be to test if your application can deal with a simple HTTP request. We will ask monit to monitor your web server by doing regular checks to a predifined URL and validate the HTTP response content. Here is an example of monit configuration for a web application: -[[[language=bash­ +[[[language=bash alert me@domain.com check process myapp with pidfile /var/run/myapp.pid @@ -167,15 +184,15 @@ check process myapp with pidfile /var/run/myapp.pid ]]] It will try to connect to the given URL, here localhost on the port 8080, do a HTTP GET on ''/ping''. Monit will restart the process myapp if the connection fails or if the content of the HTTP response is not equal to "pong" or if there is no answer before 10 seconds. In addition, you may want to monitor apache if there is an apache server in front of your application. You could do it with a dedicated configuration. You can adapt the configuration provided by monit. You just need to remove the http protocol check that is already done for your application. -[[[language=bash­ +[[[language=bash if failed host localhost port 80 with protocol http with timeout 25 seconds for 4 times within 5 cycles then restart ]]] Then, activate the apache monitoring and relaod the monit configuration: -[[[language=bash­ +[[[language=bash $ sudo ln -s /etc/monit/monitrc.d/apache2 /etc/monit/conf.d/apache2 $ sudo monit reload]]] -!! Backup +!!Backup One essential point when you have an application into production is that you do not want to lose data! In most cases, your image hosting your application should be stateless, i.e. if a problem occurs, no need to save the image state, just restart it. If you need persitent data, the best is to use a database (postgres, mongo, fuel, etc). What needs to be saved carefully is the database content. The backup is often proposed by cloud providers. If not, you need to ensure a regular backup and keep backups for a moving window time (ex: all backups from the last 30 days). -!! Logging +!!Logging TODO diff --git a/pillar.conf b/pillar.conf index 3104216..25c65bb 100644 --- a/pillar.conf +++ b/pillar.conf @@ -46,6 +46,7 @@ "Zinc-HTTP-Server/Zinc-HTTP-Server.pillar", "Zinc-REST/Zinc-REST.pillar", "WebApp/WebApp.pier", - "WebSockets/WebSockets.pier" + "WebSockets/WebSockets.pier", + "DeploymentWeb/DeployForProduction.pillar" ] }