Skip to content

Buildbot HowTo (from Aayush)

Jump to bottom Edit New page
solin edited this page Oct 11, 2011 · 1 revision

How Buildbot works

The BuildBot is a system to automate the compile/test cycle required by most software projects to validate code changes. It not only can pinpont quickly the failure of compilation each time the code has been changed, but can run the builds on a variety of platforms as well. The overall goal is to reduce tree breakage and provide a platform to run tests or code-quality checks.

The following shows the control flow of Buildbot:

  1. A developer commits some source code changes to git. A trigger script will send this changes to buildmaster, the change will include who made the change, what files were modified, which revision, and checkin comments.

  2. Buildmaster distributes the changes to its schedulers. Changes will go into a list "tree-stable-timer" of new Build. A Build is started on each configured Builders in parallel.

  3. The build consists of a set of Steps. First, it checks for revision. Then it performs a compile and run unit tests. As each step runs, the buildslave reports back command output and return status ot the buildmaster.

  4. Status message "Build Started", "Step Started" and "Build Finished", etc are published to a collection of Status Targets, for example, HTML display.

  5. If a MailNotifier status target is active, then upon completion of a build, the developer whose changes were incorporated into this build will be notified.

Installation of Buildbot

1. Requirements

  Python 2.3 or later.
  Twisted-2.0.x or later. At least Twisted core package, TwistedMail, TwistedWeb and TwistedWords.
  ZopeInterface package required by Twisted.

2. a. Build on Linux(Ubuntu)*

# sudo aptitude install python-twisted-core python-twisted-mail python-twisted-web python-twisted-words    python-zope.interface
# sudo aptitude install python-dev
# wget http://downloads.sourceforge.net/buildbot/buildbot-0.7.12.tar.gz
# tar -zxf buildbot-0.7.12.tar.gz
# cd buildbot-0.7.12/
# python setup.py build
# sudo python setup.py install

After installation, run the trial test by:
# PYTHONPATH=. trial buildbot.test

b. Build on MacOSX(10.6.3)*

Install Xcode Developer Tools.
Twisted 10.0.0 is installed under MacOSX 10.5.x later.
# sudo aptitude install python-dev
# wget http://downloads.sourceforge.net/buildbot/buildbot-0.7.12.tar.gz
# tar -zxf buildbot-0.7.12.tar.gz
# cd buildbot-0.7.12/
# python setup.py build
# sudo python setup.py install

After installation, run the trial test by:
# PYTHONPATH=. trial buildbot.test

*3. Creating Buildmaster: *

Issue the command from root:
$ adduser buildmaster
$ mkdir ~/Buildmasters/hermes
$ buildbot create-master -r /home/buildmaster/Buildmasters/hermes

*4. Creating Buildslave: *

Issue the command from root on each of the platforms:
$ adduser buildslave
$ mkdir ~/Buildslaves/hermes
$ buildslave create-slave /home/buildslave/Buildslaves/hermes MASTERHOST:PORT SLAVENAME PASSWD

#Below are the settings for hermes buildslave
MASTERHOST:PORT = buildbot.hpfem.org:9994

SLAVENAME = vista64       | ubuntu64 | macosx
PASSWD =    coffee_maker  | lamp_R3  | Paper_mate

#Below are the settings for hermes-dev buildslave
MASTERHOST:PORT = buildbot.hpfem.org:9998

SLAVENAME = vista64     | ubuntu64     | macosx
PASSWD =    pass_vista  | pass_ubuntu  | pass_mac

5. Start | Stop | Restart Buildslave:

hermes buildslave in ubuntu can be start \ stop as
$ sudo /etc/init.d/hermes_buildslave start
$ sudo /etc/init.d/hermes_buildslave stop
$ sudo /etc/init.d/hermes_buildslave restart

hermes-dev buildslave in ubuntu can be start \ stop as
$ sudo /etc/init.d/hermes-dev_buildslave start
$ sudo /etc/init.d/hermes-dev_buildslave stop
$ sudo /etc/init.d/hermes-dev_buildslave restart

5. Start | Stop | Restart Buildmaster:

hermes buildmaster in Spilka can be start \ stop as
$ sudo /etc/init.d/hermes_buildmaster start
$ sudo /etc/init.d/hermes_buildmaster stop
$ sudo /etc/init.d/hermes_buildmaster restart

hermes-dev buildmaster in Spilka can be start \ stop as
$ sudo /etc/init.d/hermes-dev_buildmaster start
$ sudo /etc/init.d/hermes-dev_buildmaster stop
$ sudo /etc/init.d/hermes-dev_buildmaster restart

for Start buildbot on MacOSX, see 7. Using Launchd

6. Git Hook

Hook used by hermes to trigger build process
Post-Receive URLs, http://hooks.hpfem.org/hermes-update.py

Hook used by hermes-dev to trigger build process
Post-Receive URLs, http://hooks.hpfem.org/hermes-dev-update.py

these hooks are located at /var/www3/hooks in Spilka.

7. Using Launchd

Launchd is Mac OSX substitude for /etc/rc/, init.d, and cron. If you launch buildbot in Mac OSX by the way described in section 5, buildbot process will vanish eventually.

The proper way to launch buildbot in Mac OSX is to use launchd, whereas you will first need to create a plist file in /Library/LaunchDaemons/. For example, you could create a file /Library/LaunchDaemons/buildbot.slave.hermes.plist as following:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>StandardOutPath</key>
    <string>twistd.log</string>
    <key>StandardErrorPath</key>
    <string>twistd-err.log</string>
    <key>EnvironmentVariables</key>
    <dict>
            <key>PATH</key>
           <string>/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/local/git/bin:/usr/X11/bin</string>
            <key>SHELL</key>
            <string>/bin/bash</string>
    </dict>
    <key>GroupName</key>
    <string>daemon</string>
    <key>KeepAlive</key>
    <dict>
            <key>SuccessfulExit</key>
            <false/>
    </dict>
    <key>Label</key>
    <string>buildbot.slave.hermes</string>
    <key>Program</key>
    <string>/usr/bin/twistd</string>
    <key>ProgramArguments</key>
    <array>
            <string>-no</string>
            <string>-y</string>
            <string>/Users/buildslave/Buildslaves/hermes/buildbot.tac</string>
    </array>
    <key>RunAtLoad</key>
    <true/>
    <key>UserName</key>
    <string>buildslave</string>
    <key>WorkingDirectory</key>
    <string>/Users/buildslave/Buildslaves/hermes</string>
</dict>
</plist>

where, Label should match the plist name; UserName is the account in which you run your buildbot, Working Directory is the place in which your project is built.

This file must be owned by root:wheel; to launch the daemon, issuing the following comamnd from root: # launchctl load -w buildbot.slave.hermes2d.plist

7. To trigger the build process

From the command line in Spilka we can hit the following command to trigger the build process for testing.
hermes
$ sudo buildbot sendchange --branch=master --master=buildbot.hpfem.org:9994 --username="buildbot" --comments="Build Test:" file_changed

hermes-dev
$ sudo buildbot sendchange --branch=master --master=buildbot.hpfem.org:9998 --username="buildbot" --comments="Build Test:" file_changed

*8. Buildbot master/slaves IP

Master: Spilka, 134.197.117.197
Slaves: 
        ubuntu : 134.197.117.93
        vista  : 134.197.116.127
        mac    : 134.197.117.8

For buildbot documentation, see http://buildbot.net/buildbot/docs/latest/

Add a custom footer
Add a custom sidebar
Clone this wiki locally