Changeset 2

Timestamp:

11/25/05 18:09:24 (3 years ago)

Author:

warner

Message:

[project @ 2005-11-26 02:09:24 by warner]
Revision: arch@buildbot.sf.net--2004/buildbot--dev--0--patch-438
Creator: Brian Warner <warner@lothar.com>

add cron-style 'scheduler.Nightly', thanks to Dobes Vandermeer

• docs/buildbot.texinfo (Scheduler Types): give a few hints about
  what Schedulers are available
  

• buildbot/scheduler.py (Nightly): add new Scheduler based upon
  work by Dobes Vandermeer and hacked mercilessly by me. This offers
  'cron'-style build scheduling at certain times of day, week,
  month, or year.
  
• buildbot/test/test_scheduler.py (Scheduling.testNightly): test it
  

Files:

• ChangeLog (modified) (1 diff)
• buildbot/scheduler.py (modified) (1 diff)
• buildbot/test/test_scheduler.py (modified) (2 diffs)
• docs/buildbot.texinfo (modified) (1 diff)

ChangeLog

@@ -1 +1 @@
-2005-11-25  Brian Warner  <warner@lothar.com>
+
+        * docs/buildbot.texinfo (Scheduler Types): give a few hints about
+        what Schedulers are available
+
+        * buildbot/scheduler.py (Nightly): add new Scheduler based upon
+        work by Dobes Vandermeer and hacked mercilessly by me. This offers
+        'cron'-style build scheduling at certain times of day, week,
+        month, or year.
+        * buildbot/test/test_scheduler.py (Scheduling.testNightly): test it
-
-        * buildbot/scheduler.py (Scheduler): change fileIsImportant

buildbot/scheduler.py

@@ -339 +339 @@
-                               SourceStamp(branch=self.branch))
-        self.submit(bs)
+
+
+
+class Nightly(BaseUpstreamScheduler):
+    """Imitate 'cron' scheduling. This can be used to schedule a nightly
+    build, or one which runs are certain times of the day, week, or month.
+
+    Pass some subset of minute, hour, dayOfMonth, month, and dayOfWeek; each
+    may be a single number or a list of valid values. The builds will be
+    triggered whenever the current time matches these values. Wildcards are
+    represented by a '*' string. All fields default to a wildcard except
+    'minute', so with no fields this defaults to a build every hour, on the
+    hour.
+
+    For example, the following master.cfg clause will cause a build to be
+    started every night at 3:00am:
+
+     s = Nightly('nightly', ['builder1', 'builder2'], hour=3, minute=0)
+     c['schedules'].append(s)
+
+    This scheduler will perform a build each monday morning at 6:23am and
+    again at 8:23am:
+
+     s = Nightly('BeforeWork', ['builder1'],
+                 dayOfWeek=0, hour=[6,8], minute=23)
+
+    The following runs a build every two hours:
+
+     s = Nightly('every2hours', ['builder1'], hour=range(0, 24, 2))
+
+    And this one will run only on December 24th:
+
+     s = Nightly('SleighPreflightCheck', ['flying_circuits', 'radar'],
+                 month=12, dayOfMonth=24, hour=12, minute=0)
+
+    For dayOfWeek and dayOfMonth, builds are triggered if the date matches
+    either of them.  Month and day numbers start at 1, not zero.
+    """
+
+    compare_attrs = ('name', 'builderNames',
+                     'minute', 'hour', 'dayOfMonth', 'month',
+                     'dayOfWeek', 'branch')
+
+    def __init__(self, name, builderNames, minute=0, hour='*',
+                 dayOfMonth='*', month='*', dayOfWeek='*',
+                 branch=None):
+        # Setting minute=0 really makes this an 'Hourly' scheduler. This
+        # seemed like a better default than minute='*', which would result in
+        # a build every 60 seconds.
+        BaseUpstreamScheduler.__init__(self, name)
+        self.builderNames = builderNames
+        self.minute = minute
+        self.hour = hour
+        self.dayOfMonth = dayOfMonth
+        self.month = month
+        self.dayOfWeek = dayOfWeek
+        self.branch = branch
+        self.delayedRun = None
+        self.nextRunTime = None
+
+    def addTime(self, timetuple, secs):
+        return time.localtime(time.mktime(timetuple)+secs)
+    def findFirstValueAtLeast(self, values, value, default=None):
+        for v in values:
+            if v >= value: return v
+        return default
+
+    def setTimer(self):
+        self.nextRunTime = self.calculateNextRunTime()
+        self.delayedRun = reactor.callLater(self.nextRunTime - time.time(),
+                                            self.doPeriodicBuild)
+
+    def startService(self):
+        BaseUpstreamScheduler.startService(self)
+        self.setTimer()
+
+    def stopService(self):
+        BaseUpstreamScheduler.stopService(self)
+        self.delayedRun.cancel()
+
+    def isRunTime(self, timetuple):
+        def check(ourvalue, value):
+            if ourvalue == '*': return True
+            if isinstance(ourvalue, int): return value == ourvalue
+            return (value in ourvalue)
+
+        if not check(self.minute, timetuple[4]):
+            #print 'bad minute', timetuple[4], self.minute
+            return False
+
+        if not check(self.hour, timetuple[3]):
+            #print 'bad hour', timetuple[3], self.hour
+            return False
+
+        if not check(self.month, timetuple[1]):
+            #print 'bad month', timetuple[1], self.month
+            return False
+
+        if self.dayOfMonth != '*' and self.dayOfWeek != '*':
+            # They specified both day(s) of month AND day(s) of week.
+            # This means that we only have to match one of the two. If
+            # neither one matches, this time is not the right time.
+            if not (check(self.dayOfMonth, timetuple[2]) or
+                    check(self.dayOfWeek, timetuple[6])):
+                #print 'bad day'
+                return False
+        else:
+            if not check(self.dayOfMonth, timetuple[2]):
+                #print 'bad day of month'
+                return False
+
+            if not check(self.dayOfWeek, timetuple[6]):
+                #print 'bad day of week'
+                return False
+
+        return True
+
+    def calculateNextRunTime(self):
+        return self.calculateNextRunTimeFrom(time.time())
+
+    def calculateNextRunTimeFrom(self, now):
+        dateTime = time.localtime(now)
+
+        # Remove seconds by advancing to at least the next minue
+        dateTime = self.addTime(dateTime, 60-dateTime[5])
+
+        # Now we just keep adding minutes until we find something that matches
+
+        # It not an efficient algorithm, but it'll *work* for now
+        yearLimit = dateTime[0]+2
+        while not self.isRunTime(dateTime):
+            dateTime = self.addTime(dateTime, 60)
+            #print 'Trying', time.asctime(dateTime)
+            assert dateTime[0] < yearLimit, 'Something is wrong with this code'
+        return time.mktime(dateTime)
+
+    def listBuilderNames(self):
+        return self.builderNames
+
+    def getPendingBuildTimes(self):
+        # TODO: figure out when self.timer is going to fire next and report
+        # that
+        if self.nextRunTime is None: return []
+        return [self.nextRunTime]
+
+    def doPeriodicBuild(self):
+        # Schedule the next run
+        self.setTimer()
+
+        # And trigger a build
+        bs = buildset.BuildSet(self.builderNames,
+                               SourceStamp(branch=self.branch))
+        self.submit(bs)
+
+    def addChange(self, change):
+        pass
+
+
-
-class TryBase(service.MultiService, util.ComparableMixin):

buildbot/test/test_scheduler.py

@@ -1 +1 @@
-# -*- test-case-name: buildbot.test.test_scheduler -*-
-
-
+, time
-
-from twisted.trial import unittest
@@ -46 +46 @@
-        s1 = self.master.sets[0]
-        self.failUnlessEqual(s1.builderNames, ["a","b"])
+
+    def testNightly(self):
+        # now == 15-Nov-2005, 00:05:36 AM . By using mktime, this is
+        # converted into the local timezone, which happens to match what
+        # Nightly is going to do anyway.
+        MIN=60; HOUR=60*MIN; DAY=24*3600
+        now = time.mktime((2005, 11, 15, 0, 5, 36, 1, 319, 0))
+
+        s = scheduler.Nightly('nightly', ["a"], hour=3)
+        t = s.calculateNextRunTimeFrom(now)
+        self.failUnlessEqual(int(t-now), 2*HOUR+54*MIN+24)
+
+        s = scheduler.Nightly('nightly', ["a"], minute=[3,8,54])
+        t = s.calculateNextRunTimeFrom(now)
+        self.failUnlessEqual(int(t-now), 2*MIN+24)
+
+        s = scheduler.Nightly('nightly', ["a"],
+                              dayOfMonth=16, hour=1, minute=6)
+        t = s.calculateNextRunTimeFrom(now)
+        self.failUnlessEqual(int(t-now), DAY+HOUR+24)
+
+        s = scheduler.Nightly('nightly', ["a"],
+                              dayOfMonth=16, hour=1, minute=3)
+        t = s.calculateNextRunTimeFrom(now)
+        self.failUnlessEqual(int(t-now), DAY+57*MIN+24)
+
+        s = scheduler.Nightly('nightly', ["a"],
+                              dayOfMonth=15, hour=1, minute=3)
+        t = s.calculateNextRunTimeFrom(now)
+        self.failUnlessEqual(int(t-now), 57*MIN+24)
+
+        s = scheduler.Nightly('nightly', ["a"],
+                              dayOfMonth=15, hour=0, minute=3)
+        t = s.calculateNextRunTimeFrom(now)
+        self.failUnlessEqual(int(t-now), 30*DAY-3*MIN+24)
+
-
-    def isImportant(self, change):

docs/buildbot.texinfo

@@ -1 @@
-\input texinfo @c -*-texinfo-*-
-@c %**start of header
-@setfilename buildbot.info
-@settitle BuildBot Manual 0.7.0+
-@c %**end of header
-
-@copying
-This is the BuildBot manual.
-
-Copyright (C) 2005 Brian Warner
-
-Copying and distribution of this file, with or without
-modification, are permitted in any medium without royalty
-provided the copyright notice and this notice are preserved.
-
-@end copying
-
-@titlepage
-@title BuildBot
-@page
-@vskip 0pt plus 1filll
-@insertcopying
-@end titlepage
-
-@c Output the table of the contents at the beginning.
-@contents
-
-@ifnottex
-@node Top, Introduction, (dir), (dir)
-@top BuildBot
-
-@insertcopying
-@end ifnottex
-
-@menu
-* Introduction::                What the BuildBot does.
-* Installation::                Creating a buildmaster and buildslaves,
-                                running them.
-* Concepts::                    What goes on in the buildbot's little mind.
-* Configuration::               Controlling the buildbot.
-* Getting Source Code Changes::  Discovering when to run a build.
-* Build Process::               Controlling how each build is run.
-* Status Delivery::             Telling the world about the build's results.
-* Command-line tool::           
-* Resources::                   Getting help.
-* Developer's Appendix::        
-* Index::                       Complete index.
-
-@detailmenu
- --- The Detailed Node Listing ---
-
-Introduction
-
-* History and Philosophy::      
-* System Architecture::         
-* Control Flow::                
-
-Installation
-
-* Requirements::                
-* Installing the code::         
-* Creating a buildmaster::      
-* Creating a buildslave::       
-* Launching the daemons::       
-* Logfiles::                    
-* Shutdown::                    
-* Maintenance::                 
-* Troubleshooting::             
-
-Troubleshooting
-
-* Starting the buildslave::     
-* Connecting to the buildmaster::  
-* Forcing Builds::              
-
-Concepts
-
-* Version Control Systems::     
-* Schedulers::                  
-* BuildSet::                    
-* BuildRequest::                
-* Builder::                     
-* Users::                       
-
-Version Control Systems
-
-* Generalizing VC Systems::     
-* Source Tree Specifications::  
-* How Different VC Systems Specify Sources::  
-* Attributes of Changes::       
-
-Users
-
-* Doing Things With Users::     
-* Email Addresses::             
-* IRC Nicknames::               
-* Live Status Clients::         
-
-Configuration
-
-* Config File Format::          
-* Loading the Config File::     
-* Defining the Project::        
-* Listing Change Sources and Schedulers::  
-* Setting the slaveport::       
-* Buildslave Specifiers::       
-* Defining Builders::           
-* Defining Status Targets::     
-* Debug options::               
-
-Listing Change Sources and Schedulers
-
-* Build Dependencies::          
-
-Getting Source Code Changes
-
-* Change Sources::              
-
-Change Sources
-
-* Choosing ChangeSources::      
-* CVSToys - PBService::         
-* CVSToys - mail notification::  
-* Other mail notification ChangeSources::  
-* PBChangeSource::              
-
-Build Process
-
-* Build Steps::                 
-* Interlocks::                  
-* Build Factories::             
-
-Build Steps
-
-* Common Parameters::           
-* Source Checkout::             
-* ShellCommand::                
-* Simple ShellCommand Subclasses::  
-
-Source Checkout
-
-* CVS::                         
-* SVN::                         
-* Darcs::                       
-* Arch::                        
-* Bazaar::                      
-* P4Sync::                      
-
-Simple ShellCommand Subclasses
-
-* Configure::                   
-* Compile::                     
-* Test::                        
-
-Build Factories
-
-* BuildStep Objects::           
-* BuildFactory::                
-* Process-Specific build factories::  
-
-BuildFactory
-
-* BuildFactory Attributes::     
-* Quick builds::                
-
-Process-Specific build factories
-
-* GNUAutoconf::                 
-* CPAN::                        
-* Python distutils::            
-* Python/Twisted/trial projects::  
-
-Status Delivery
-
-* HTML Waterfall::              
-* IRC Bot::                     
-* PBListener::                  
-
-Command-line tool
-
-* Administrator Tools::         
-* Developer Tools::             
-* Other Tools::                 
-* .buildbot config directory::  
-
-Developer Tools
-
-* statuslog::                   
-* statusgui::                   
-* try::                         
-
-Other Tools
-
-* sendchange::                  
-* debugclient::                 
-
-@end detailmenu
-@end menu
-
-@node Introduction, Installation, Top, Top
-@chapter Introduction
-
-@cindex introduction
-
-The BuildBot is a system to automate the compile/test cycle required by most
-software projects to validate code changes. By automatically rebuilding and
-testing the tree each time something has changed, build problems are
-pinpointed quickly, before other developers are inconvenienced by the
-failure. The guilty developer can be identified and harassed without human
-intervention. By running the builds on a variety of platforms, developers
-who do not have the facilities to test their changes everywhere before
-checkin will at least know shortly afterwards whether they have broken the
-build or not. Warning counts, lint checks, image size, compile time, and
-other build parameters can be tracked over time, are more visible, and
-are therefore easier to improve.
-
-The overall goal is to reduce tree breakage and provide a platform to
-run tests or code-quality checks that are too annoying or pedantic for
-any human to waste their time with. Developers get immediate (and
-potentially public) feedback about their changes, encouraging them to
-be more careful about testing before checkin.
-
-Features:
-
-@itemize @bullet
-@item
-run builds on a variety of slave platforms
-@item
-arbitrary build process: handles projects using C, Python, whatever
-@item
-minimal host requirements: python and Twisted
-@item
-slaves can be behind a firewall if they can still do checkout
-@item
-status delivery through web page, email, IRC, other protocols
-@item
-track builds in progress, provide estimated completion time
-@item
-flexible configuration by subclassing generic build process classes
-@item
-debug tools to force a new build, submit fake Changes, query slave status
-@item
-released under the GPL
-@end itemize
-
-@menu
-* History and Philosophy::      
-* System Architecture::         
-* Control Flow::                
-@end menu
-
-
-@node History and Philosophy, System Architecture, Introduction, Introduction
-@section History and Philosophy
-
-@cindex Philosophy of operation
-
-The Buildbot was inspired by a similar project built for a development
-team writing a cross-platform embedded system. The various components
-of the project were supposed to compile and run on several flavors of
-unix (linux, solaris, BSD), but individual developers had their own
-preferences and tended to stick to a single platform. From time to
-time, incompatibilities would sneak in (some unix platforms want to
-use @code{string.h}, some prefer @code{strings.h}), and then the tree
-would compile for some developers but not others. The buildbot was
-written to automate the human process of walking into the office,
-updating a tree, compiling (and discovering the breakage), finding the
-developer at fault, and complaining to them about the problem they had
-introduced. With multiple platforms it was difficult for developers to
-do the right thing (compile their potential change on all platforms);
-the buildbot offered a way to help.
-
-Another problem was when programmers would change the behavior of a
-library without warning its users, or change internal aspects that
-other code was (unfortunately) depending upon. Adding unit tests to
-the codebase helps here: if an application's unit tests pass despite
-changes in the libraries it uses, you can have more confidence that
-the library changes haven't broken anything. Many developers
-complained that the unit tests were inconvenient or took too long to
-run: having the buildbot run them reduces the developer's workload to
-a minimum.
-
-In general, having more visibility into the project is always good,
-and automation makes it easier for developers to do the right thing.
-When everyone can see the status of the project, developers are
-encouraged to keep the tree in good working order. Unit tests that
-aren't run on a regular basis tend to suffer from bitrot just like
-code does: exercising them on a regular basis helps to keep them
-functioning and useful.
-
-The current version of the Buildbot is additionally targeted at
-distributed free-software projects, where resources and platforms are
-only available when provided by interested volunteers. The buildslaves
-are designed to require an absolute minimum of configuration, reducing
-the effort a potential volunteer needs to expend to be able to
-contribute a new test environment to the project. The goal is for
-anyone who wishes that a given project would run on their favorite
-platform should be able to offer that project a buildslave, running on
-that platform, where they can verify that their portability code
-works, and keeps working.
-
-@node System Architecture, Control Flow, History and Philosophy, Introduction
-@comment  node-name,  next,  previous,  up
-@section System Architecture
-
-The Buildbot consists of a single @code{buildmaster} and one or more
-@code{buildslaves}, connected in a star topology. The buildmaster
-makes all decisions about what and when to build. It sends commands to
-be run on the build slaves, which simply execute the commands and
-return the results. (certain steps involve more local decision making,
-where the overhead of sending a lot of commands back and forth would
-be inappropriate, but in general the buildmaster is responsible for
-everything).
-
-The buildmaster is usually fed @code{Changes} by some sort of version
-control system @xref{Change Sources}, which may cause builds to be
-run. As the builds are performed, various status messages are
-produced, which are then sent to any registered Status Targets
-@xref{Status Delivery}.
-
-@ifinfo
-@smallexample
-@group
- TODO: picture of change sources, master, slaves, status targets
- should look like docs/PyCon-2003/sources/overview.svg
-@end group
-@end smallexample
-@end ifinfo
-@ifnotinfo
-@c @image{images/overview}
-@end ifnotinfo
-
-The buildmaster is configured and maintained by the ``buildmaster
-admin'', who is generally the project team member responsible for
-build process issues. Each buildslave is maintained by a ``buildslave
-admin'', who do not need to be quite as involved. Generally slaves are
-run by anyone who has an interest in seeing the project work well on
-their platform.
-
-
-@node Control Flow,  , System Architecture, Introduction
-@comment  node-name,  next,  previous,  up
-@section Control Flow
-
-A day in the life of the buildbot:
-
-@itemize @bullet
-
-@item
-A developer commits some source code changes to the repository. A hook
-script or commit trigger of some sort sends information about this
-change to the buildmaster through one of its configured Change
-Sources. This notification might arrive via email, or over a network
-connection (either initiated by the buildmaster as it ``subscribes''
-to changes, or by the commit trigger as it pushes Changes towards the
-buildmaster). The Change contains information about who made the
-change, what files were modified, which revision contains the change,
-and any checkin comments.
-
-@item
-The buildmaster distributes this change to all of its configured
-Schedulers. Any ``important'' changes cause the ``tree-stable-timer''
-to be started, and the Change is added to a list of those that will go
-into a new Build. When the timer expires, a Build is started on each
-of a set of configured Builders, all compiling/testing the same source
-code. Unless configured otherwise, all Builds run in parallel on the
-various buildslaves.
-
-@item
-The Build consists of a series of Steps. Each Step causes some number
-of commands to be invoked on the remote buildslave associated with
-that Builder. The first step is almost always to perform a checkout of
-the appropriate revision from the same VC system that produced the
-Change. The rest generally perform a compile and run unit tests. As
-each Step runs, the buildslave reports back command output and return
-status to the buildmaster.
-
-@item
-As the Build runs, status messages like ``Build Started'', ``Step
-Started'', ``Build Finished'', etc, are published to a collection of
-Status Targets. One of these targets is usually the HTML ``Waterfall''
-display, which shows a chronological list of events, and summarizes
-the results of the most recent build at the top of each column.
-Developers can periodically check this page to see how their changes
-have fared. If they see red, they know that they've made a mistake and
-need to fix it. If they see green, they know that they've done their
-duty and don't need to worry about their change breaking anything.
-
-@item
-If a MailNotifier status target is active, the completion of a build
-will cause email to be sent to any developers whose Changes were
-incorporated into this Build. The MailNotifier can be configured to
-only send mail upon failing builds, or for builds which have just
-transitioned from passing to failing. Other status targets can provide
-similar real-time notification via different communication channels,
-like IRC.
-
-@end itemize
-
-
-@node Installation, Concepts, Introduction, Top
-@chapter Installation
-
-@menu
-* Requirements::                
-* Installing the code::         
-* Creating a buildmaster::      
-* Creating a buildslave::       
-* Launching the daemons::       
-* Logfiles::                    
-* Shutdown::                    
-* Maintenance::                 
-* Troubleshooting::             
-@end menu
-
-@node Requirements, Installing the code, Installation, Installation
-@section Requirements
-
-At a bare minimum, you'll need the following (for both the buildmaster
-and a buildslave):
-
-@itemize @bullet
-@item
-Python: http://www.python.org
-
-Buildbot requires python-2.2 or later, and is primarily developed
-against python-2.3. The buildmaster uses generators, a feature which
-is not available in python-2.1, and both master and slave require a
-version of Twisted which only works with python-2.2 or later. Certain
-features (like the inclusion of build logs in status emails) require
-python-2.2.2 or later. The IRC ``force build'' command requires
-python-2.3 (for the shlex.split function).
-
-@item
-Twisted: http://twistedmatrix.com
-
-Both the buildmaster and the buildslaves require Twisted-1.3.0 or
-later. It has been mainly developed against Twisted-2.0.1, but has
-been tested against Twisted-2.1.0 (the most recent as of this
-writing), and might even work on versions as old as Twisted-1.1.0, but
-as always the most recent version is recommended.
-
-Twisted-1.3.0 and earlier were released as a single monolithic
-package. When you run Buildbot against Twisted-2.0.0 or later (which
-are split into a number of smaller subpackages), you'll need at least
-"Twisted" (the core package), and you'll also want TwistedMail,
-TwistedWeb, and TwistedWords (for sending email, serving a web status
-page, and delivering build status via IRC, respectively).
-@end itemize
-
-Certain other packages may be useful on the system running the
-buildmaster:
-
-@itemize @bullet
-@item
-CVSToys: http://purl.net/net/CVSToys
-
-If your buildmaster uses FreshCVSSource to receive change notification
-from a cvstoys daemon, it will require CVSToys be installed (tested
-with CVSToys-1.0.10). If the it doesn't use that source (i.e. if you
-only use a mail-parsing change source, or the SVN notification
-script), you will not need CVSToys.
-
-@end itemize
-
-And of course, your project's build process will impose additional
-requirements on the buildslaves. These hosts must have all the tools
-necessary to compile and test your project's source code.
-
-
-@node Installing the code, Creating a buildmaster, Requirements, Installation
-@section Installing the code
-
-@cindex installation
-
-The Buildbot is installed using the standard python @code{distutils}
-module. After unpacking the tarball, the process is:
-
-@example
-python setup.py build
-python setup.py install
-@end example
-
-where the install step may need to be done as root. This will put the
-bulk of the code in somewhere like
-/usr/lib/python2.3/site-packages/buildbot . It will also install the
-@code{buildbot} command-line tool in /usr/bin/buildbot.
-
-To test this, shift to a different directory (like /tmp), and run:
-
-@example
-buildbot --version
-@end example
-
-If it shows you the versions of Buildbot and Twisted, the install went
-ok. If it says @code{no such command} or it gets an @code{ImportError}
-when it tries to load the libaries, then something went wrong.
-@code{pydoc buildbot} is another useful diagnostic tool.
-
-Windows users will find these files in other places. You will need to
-make sure that python can find the libraries, and will probably find
-it convenient to have @code{buildbot} on your PATH.
-
-If you wish, you can run the buildbot unit test suite like this:
-
-@example
-PYTHONPATH=. trial -v buildbot.test
-@end example
-
-This should run up to 175 tests, depending upon what VC tools you have
-installed. On my desktop machine it takes about four minutes to
-complete. Nothing should fail, a few might be skipped. If any of the
-tests fail, you should stop and investigate the cause before
-continuing the installation process, as it will probably be easier to
-track down the bug early.
-
-If you cannot or do not wish to install the buildbot into a site-wide
-location like @file{/usr} or @file{/usr/local}, you can also install
-it into the account's home directory. Do the install command like
-this:
-
-@example
-python setup.py install --home=~
-@end example
-
-That will populate @file{~/lib/python} and create
-@file{~/bin/buildbot}. Make sure this lib directory is on your
-@code{PYTHONPATH}.
-
-
-@node Creating a buildmaster, Creating a buildslave, Installing the code, Installation
-@section Creating a buildmaster
-
-As you learned earlier (@pxref{System Architecture}), the buildmaster
-runs on a central host (usually one that is publically visible, so
-everybody can check on the status of the project), and controls all
-aspects of the buildbot system. Let us call this host
-@code{buildbot.example.org}.
-
-You may wish to create a separate user account for the buildmaster,
-perhaps named @code{buildmaster}. This can help keep your personal
-configuration distinct from that of the buildmaster and is useful if
-you have to use a mail-based notification system (@pxref{Change
-Sources}). However, the Buildbot will work just fine with your regular
-user account.
-
-You need to choose a directory for the buildmaster, called the
-@code{basedir}. This directory will be owned by the buildmaster, which
-will use configuration files therein, and create status files as it
-runs. @file{~/Buildbot} is a likely value. If you run multiple
-buildmasters in the same account, or if you run both masters and
-slaves, you may want a more distinctive name like
-@file{~/Buildbot/master/gnomovision} or
-@file{~/Buildmasters/fooproject}. If you are using a separate user
-account, this might just be @file{~buildmaster/masters/fooprojects}.
-
-Once you've picked a directory, use the @command{buildbot master}
-command to create the directory and populate it with startup files:
-
-@example
-buildbot master @var{basedir}
-@end example
-
-You will need to create a configuration file (@pxref{Configuration})
-before starting the buildmaster. Most of the rest of this manual is
-dedicated to explaining how to do this. A sample configuration file is
-placed in the working directory, named @file{master.cfg.sample}, which
-can be copied to @file{master.cfg} and edited to suit your purposes.
-
-(Internal details: This command creates a file named
-@file{buildbot.tac} that contains all the state necessary to create
-the buildmaster. Twisted has a tool called @code{twistd} which can use
-this .tac file to create and launch a buildmaster instance. twistd
-takes care of logging and daemonization (running the program in the
-background). @file{/usr/bin/buildbot} is a front end which runs twistd
-for you.)
-
-In addition to @file{buildbot.tac}, a small @file{Makefile.sample} is
-installed. This can be used as the basis for customized daemon startup,
-@xref{Launching the daemons}.
-
-
-@node Creating a buildslave, Launching the daemons, Creating a buildmaster, Installation
-@section Creating a buildslave
-
-Typically, you will be adding a buildslave to an existing buildmaster,
-to provide additional architecture coverage. The buildbot
-administrator will give you several pieces of information necessary to
-connect to the buildmaster. You should also be somewhat familiar with
-the project being tested, so you can troubleshoot build problems
-locally.
-
-The buildbot exists to make sure that the project's stated ``how to
-build it'' process actually works. To this end, the buildslave should
-run in an environment just like that of your regular developers.
-Typically the project build process is documented somewhere
-(@file{README}, @file{INSTALL}, etc), in a document that should
-mention all library dependencies and contain a basic set of build
-instructions. This document will be useful as you configure the host
-and account in which the buildslave runs.
-
-Here's a good checklist for setting up a buildslave:
-
-@enumerate
-@item
-Set up the account
-
-It is recommended (although not mandatory) to set up a separate user
-account for the buildslave. This account is frequently named
-@code{buildbot} or @code{buildslave}. This serves to isolate your
-personal working environment from that of the slave's, and helps to
-minimize the security threat posed by letting possibly-unknown
-contributors run arbitrary code on your system. The account should
-have a minimum of fancy init scripts.
-
-@item
-Install the buildbot code
-
-Follow the instructions given earlier (@pxref{Installing the code}).
-If you use a separate buildslave account, and you didn't install the
-buildbot code to a shared location, then you will need to install it
-with @code{--home=~} for each account that needs it.
-
-@item
-Set up the host
-
-Make sure the host can actually reach the buildmaster. Usually the
-buildmaster is running a status webserver on the same machine, so
-simply point your web browser at it and see if you can get there.
-Install whatever additional packages or libraries the project's
-INSTALL document advises. (or not: if your buildslave is supposed to
-make sure that building without optional libraries still works, then
-don't install those libraries).
-
-Again, these libraries don't necessarily have to be installed to a
-site-wide shared location, but they must be available to your build
-process. Accomplishing this is usually very specific to the build
-process, so installing them to @file{/usr} or @file{/usr/local} is
-usually the best approach.
-
-@item
-Test the build process
-
-Follow the instructions in the INSTALL document, in the buildslave's
-account. Perform a full CVS (or whatever) checkout, configure, make,
-run tests, etc. Confirm that the build works without manual fussing.
-If it doesn't work when you do it by hand, it will be unlikely to work
-when the buildbot attempts to do it in an automated fashion.
-
-@item
-Choose a base directory
-
-This should be somewhere in the buildslave's account, typically named
-after the project which is being tested. The buildslave will not touch
-any file outside of this directory. Something like @file{~/Buildbot}
-or @file{~/Buildslaves/fooproject} is appropriate.
-
-@item
-Get the buildmaster host/port, botname, and password
-
-When the buildbot admin configures the buildmaster to accept and use
-your buildslave, they will provide you with the following pieces of
-information:
-
-@itemize @bullet
-@item
-your buildslave's name
-@item
-the password assigned to your buildslave
-@item
-the hostname and port number of the buildmaster, i.e. buildbot.example.org:8007
-@end itemize
-
-@item
-Create the buildslave
-
-Now run the 'buildbot' command as follows:
-
-@example
-buildbot slave @var{BASEDIR} @var{MASTERHOST}:@var{PORT} @var{SLAVENAME} @var{PASSWORD}
-@end example
-
-This will create the base directory and a collection of files inside,
-including the @file{buildbot.tac} file that contains all the
-information you passed to the @code{buildbot} command.
-
-@item
-Fill in the hostinfo files
-
-When it first connects, the buildslave will send a few files up to the
-buildmaster which describe the host that it is running on. These files
-are presented on the web status display so that developers have more
-information to reproduce any test failures that are witnessed by the
-buildbot. There are sample files in the @file{info} subdirectory of
-the buildbot's base directory. You should edit these to correctly
-describe you and your host.
-
-@file{BASEDIR/info/admin} should contain your name and email address.
-This is the ``buildslave admin address'', and will be visible from the
-build status page (so you may wish to munge it a bit if
-address-harvesting spambots are a concern).
-
-@file{BASEDIR/info/host} should be filled with a brief description of
-the host: OS, version, memory size, CPU speed, versions of relevant
-libraries installed, and finally the version of the buildbot code
-which is running the buildslave.
-
-If you run many buildslaves, you may want to create a single
-@file{~buildslave/info} file and share it among all the buildslaves
-with symlinks.
-
-@end enumerate
-
-
-@node Launching the daemons, Logfiles, Creating a buildslave, Installation
-@section Launching the daemons
-
-Both the buildmaster and the buildslave run as daemon programs. To
-launch them, pass the working directory to the @code{buildbot}
-command:
-
-@example
-buildbot start @var{BASEDIR}
-@end example
-
-This command will start the daemon and then return, so normally it
-will not produce any output. To verify that the programs are indeed
-running, look for a pair of files named @file{twistd.log} and
-@file{twistd.pid} that should be created in the working directory.
-@file{twistd.pid} contains the process ID of the newly-spawned daemon.
-
-When the buildslave connects to the buildmaster, new directories will
-start appearing in its base directory. The buildmaster tells the slave
-to create a directory for each Builder which will be using that slave.
-All build operations are performed within these directories: CVS
-checkouts, compiles, and tests.
-
-Once you get everything running, you will want to arrange for the
-buildbot daemons to be started at boot time. One way is to use
-@code{cron}, by putting them in a @@reboot crontab entry:
-
-@example
-@@reboot buildbot @var{BASEDIR}
-@end example
-
-It is important to remember that the environment provided to cron jobs
-and init scripts can be quite different that your normal runtime.
-There may be fewer environment variables specified, and the PATH may
-be shorter than usual. It is a good idea to test out this method of
-launching the buildslave by using a cron job with a time in the near
-future, with the same command, and then check @file{twistd.log} to
-make sure the slave actually started correctly. Common problems here
-are for @file{/usr/local} or @file{~/bin} to not be on your
-@code{PATH}, or for @code{PYTHONPATH} to not be set correctly.
-Sometimes @code{HOME} is messed up too.
-
-To modify the way the daemons are started (perhaps you want to set
-some environment variables first, or perform some cleanup each time),
-you can create a file named @file{Makefile.buildbot} in the base
-directory. When the @file{buildbot} front-end tool is told to
-@command{start} the daemon, and it sees this file (and
-@file{/usr/bin/make} exists), it will do @command{make -f
-Makefile.buildbot start} instead of its usual action (which involves
-running @command{twistd}). When the buildmaster or buildslave is
-installed, a @file{Makefile.sample} is created which implements the
-same behavior as the the @file{buildbot} tool uses, so if you want to
-customize the process, just copy @file{Makefile.sample} to
-@file{Makefile.buildbot} and edit it as necessary.
-
-@node Logfiles, Shutdown, Launching the daemons, Installation
-@section Logfiles
-
-@cindex logfiles
-
-While a buildbot daemon runs, it emits text to a logfile, named
-@file{twistd.log}. A command like @code{tail -f twistd.log} is useful
-to watch the command output as it runs.
-
-The buildmaster will announce any errors with its configuration file
-in the logfile, so it is a good idea to look at the log at startup
-time to check for any problems. Most buildmaster activities will cause
-lines to be added to the log.
-
-@node Shutdown, Maintenance, Logfiles, Installation
-@section Shutdown
-
-To stop a buildmaster or buildslave manually, use:
-
-@example
-buildbot stop @var{BASEDIR}
-@end example
-
-This simply looks for the @file{twistd.pid} file and kills whatever
-process is identified within.
-
-At system shutdown, all processes are sent a @code{SIGKILL}. The
-buildmaster and buildslave will respond to this by shutting down
-normally.
-
-The buildmaster will respond to a @code{SIGHUP} by re-reading its
-config file. The following shortcut is available:
-
-@example
-buildbot sighup @var{BASEDIR}
-@end example
-
-@node Maintenance, Troubleshooting, Shutdown, Installation
-@section Maintenance
-
-It is a good idea to check the buildmaster's status page every once in
-a while, to see if your buildslave is still online. Eventually the
-buildbot will probably be enhanced to send you email (via the
-@file{info/admin} email address) when the slave has been offline for
-more than a few hours.
-
-If you find you can no longer provide a buildslave to the project, please
-let the project admins know, so they can put out a call for a
-replacement.
-
-The Buildbot records status and logs output continually, each time a
-build is performed. The status tends to be small, but the build logs
-can become quite large. Each build and log are recorded in a separate
-file, arranged hierarchically under the buildmaster's base directory.
-To prevent these files from growing without bound, you should
-periodically delete old build logs. A simple cron job to delete
-anything older than, say, two weeks should do the job. The only trick
-is to leave the @file{buildbot.tac} and other support files alone, for
-which find's @code{-mindepth} argument helps skip everything in the
-top directory. You can use something like the following:
-
-@example
-@@weekly cd BASEDIR && find . -mindepth 2 -type f -mtime +14 -exec rm @{@