2008-03-29 Brian Warner * buildbot/__init__.py (version): Releasing buildbot-0.7.7 * docs/buildbot.texinfo: same 2008-03-29 Brian Warner * NEWS: update with all changes since the last release. I left the bugs-fixed section somewhat terse, but there's a link to the Trac page with more details. * CREDITS: update * buildbot/status/web/waterfall.py (WaterfallStatusResource): accept category= arguments in the query-arg portion of the URL. These behave like the builder= queryargs: if any category= queryargs are present, we'll only show Builders that fit into one of the specified categories. Using multiple category= arguments will show more Builders. Thanks to Jerome Davann (hollowman) for the idea and the patch. Closes #117. * docs/buildbot.texinfo (Buildbot Web Resources): document it * buildbot/changes/svnpoller.py (SVNPoller.create_changes): pass revision as a string, not an int. This fixes a problem in MailNotifier, which expects a string. Thanks to Dan Locks for the patch, and Dustin Mitchell for the test fixups. Closes #208. * buildbot/steps/source.py (Source.commandComplete): treat got_revision as a string too. * buildbot/test/test_svnpoller.py: update test to match * buildbot/test/test_vc.py: same 2008-03-27 Brian Warner * buildbot/scripts/runner.py (doCheckConfig): patch from Ben Hearsum to implement 'buildbot checkconfig': validate that your master.cfg is good without actually touching the buildmaster. Closes #37. Thanks! * buildbot/scripts/checkconfig.py: same * docs/buildbot.texinfo (Testing the Config File): document it 2008-03-22 Brian Warner * buildbot/test/test_vc.py (Mercurial.serveHTTP): another attempt to make Mercurial.testCheckoutHTTP not fail on heavily loaded machines: avoid overlapping poll() calls. * NEWS: preliminary list of changes for the upcoming release. Needs editing and explanations. * buildbot/status/web/base.py (make_stop_form): add a label= argument so that the button says "Stop Builds" or "Stop Build" as appropriate. It used to say "Stop Builder", which was wrong. * buildbot/status/web/baseweb.py (OneLinePerBuild.body): same (OneBoxPerBuilder.body): same * buildbot/process/builder.py (Builder): use self.old_building to track Builds from a previous version of this Builder and are still running. This makes it possible for IBuilderControl to get at them, allowing the "Stop Build" button to work after a Builder-replacing reconfig. Closes #22. (Builder.consumeTheSoulOfYourPredecessor): grab weakrefs from the old Builder's self.building attribute, instead of strongrefs, since there's no good place to explicitly forget about them. (Builder.getBuild): factor this out of BuilderControl * buildbot/process/buildstep.py (LogLineObserver.setMaxLineLength): new method to override the default 16KiB line length. Lines that are longer than this will cause the rest of the log file to be ignored. Thanks to Greg Ward for the patch. Closes #201. * buildbot/test/test_buildstep.py: new tests for log observers * docs/buildbot.texinfo (Adding LogObservers): document it * buildbot/slave/commands.py (ShellCommand): record more information in the 'header' section: how much we write into stdin, do we close stdin, are we using a PTY. There are lingering problems with PTYs and with closing stdin (see #198 and #158). This doesn't fix those problems, but it should provide more information to developers. * buildbot/status/client.py: hush pyflakes, log.msg vs log * buildbot/status/mail.py: same * buildbot/locks.py (debuglog): hush pyflakes * buildbot/steps/source.py (SVN.__init__): fix pyflakes: it doesn't like a variable named 'warnings' that shadows the import * buildbot/test/test_web.py: fix some pyflakes warnings, caused by having a variable name shadow a top-level import * buildbot/test/test_run.py (Disconnect.testBuild2): fix old cut-and-paste typo that was causing us to run the wrong callback * Makefile (pyflakes): sort+uniqify, since newer pyflakes isn't smart enough to do this on its own * buildbot/test/runutils.py (myGetProcessOutputAndValue): move this method out from test_vc.py, hush pyflakes complaints * buildbot/test/test_vc.py (Mercurial.serveHTTP): wait for the 'hg serve' command to be ready, by polling with twisted.web's client.getPage . Unfortunately mercurial doesn't flush its stdout at startup, so we can't just wait for the 'listening at..' line to be printed. This should fix the sporadic "connection refused" test failures I've seen on the metabuildbot when the buildslave is overloaded and 'hg serve' takes a while to start up. * buildbot/steps/python_twisted.py (TrialTestCaseCounter): modify regexp to count "Doctest:" lines as well as normal tests. Thanks to Martin Geisler for the patch. Closes #196. 2008-03-20 Brian Warner * buildbot/status/web/waterfall.py (CurrentBox.formatETA): apply patch from TedMielczarek to display hours/minutes/seconds in the ETA display. Thanks Ted! Also make 'next build starts in' box use the same format. Closes #95. * buildbot/process/buildstep.py (render_properties): more cleanup: rename 'render' to 'render_properties'. * buildbot/steps/shell.py: same (ShellCommand._interpolateProperties): tolerate None in the value, clean up the isinstance checks, stop using the 'types' module * buildbot/steps/transfer.py (FileUpload.start): same, other cleanups * buildbot/steps/shell.py (render): oops, remove redundant definition that probably crept in during a merge conflict * buildbot/scheduler.py (Triggerable): cleanup: wrap docstring * buildbot/steps/trigger.py (Trigger): cleanup: rename schedulers= to schedulerNames=, make updateSourceStamp=True by default, tolerate missing got_revision property (i.e. if updateSourceStamp=True but there was no checkout/update step run), add TODO notes for further cleanup. * buildbot/test/test_run.py (Triggers): update to match * docs/buildbot.texinfo (Build Dependencies): same * buildbot/slave/commands.py (SVN.doVCFull): merge Dustin's patch: add --no-auth-cache option to svn invocations to avoid caching buildbot's credentials. This also helps when users run a buildslave from their personal account, to keep the buildslave from clobbering their manually-provided svn password. Closes #188. (SVN.doVCUpdate): do it for 'svn update' too. * buildbot/steps/shell.py: merge Greg Ward's patch (via Dustin): refactor WithProperties handling, allow WithProperties in more places. Closes #115. * buildbot/process/buildstep.py: same * buildbot/steps/transfer.py: same, allow WithProperties in filenames * buildbot/steps/trigger.py: remove unused imports to hush pyflakes * docs/buildbot.texinfo (Build Properties): fix up nodes+menus 2008-03-19 Brian Warner * buildbot/status/web/xmlrpc.py (XMLRPCServer.xmlrpc_getAllBuilders): merge Etienne Pierre's patch (via Dustin): add new XMLRPC methods. (XMLRPCServer.xmlrpc_getStatus): same (XMLRPCServer.xmlrpc_getLastBuilds): same, returns the last N completed builds for a given Builder * buildbot/test/test_config.py (ConfigTest.setUp): merge Dustin's patch: suppress DeprecationWarnings in our tests of deprecated features that aren't scheduled for removal quite yet. In the long run, we'll remove the old features and the tests. * NEWS: add a deprecation schedule * buildbot/changes/freshcvsmail.py: remove this file, it was replaced by mail.FCMaildirSource in the previous release. * buildbot/test/runutils.py: set usePTY=False everywhere we aren't explicitly testing =True, to avoid SIGHUP-based failures that I now see about 10% of the time. * buildbot/scheduler.py (Triggerable): merge Dustin's patch: add "Triggerable" Schedulers, to start builds at specific phases of other builds rather than in response to source code changes or periodic timers. These phases are marked by "Trigger" steps, which are configured with a list of Triggerable schedulers that they should fire. Closes #56. * buildbot/steps/trigger.py (Trigger): buildstep to match * buildbot/test/test_run.py (Triggers): test for it * docs/buildbot.texinfo (Build Dependencies): document it (Triggering Schedulers): same 2008-03-18 Brian Warner * buildbot/interfaces.py (ISourceStamp.getAbsoluteSourceStamp): merge patch from Dustin to add "absolute source stamps": just like normal SourceStamp instances except that they have a .revision attribute copied out of the got_revision property of a build. If the input SourceStamp says .revision=None (i.e. HEAD), and the build is done when the tree is at r1234, then the "absolute" SourceStamp should have .revision=1234 . * buildbot/sourcestamp.py (SourceStamp.getAbsoluteSourceStamp): same * buildbot/status/builder.py (BuildStatus.getSourceStamp): add absolute= argument * buildbot/test/runutils.py (SetTestFlagStep): test utility class, adding 'flags' that can be set by specific buildsteps. This makes it easier to write tests that detect whether certain buildsteps have been reached or not. Patch by Dustin Mitchell. * buildbot/test/test_run.py (TestFlag.testTestFlag): test for it. * buildbot/buildslave.py (BuildSlave.__repr__): protect against .botmaster being empty, to allow pretty-printing of configuration files before the twisted reactor starts. Patch from Dustin Mitchell. * docs/buildbot.texinfo (Shutdown): make it clear that 'buildbot reconfig' (which uses SIGHUP) doesn't work on windows. Acknowledge the occasional failures in reconfig (and suggest 'buildbot restart' as a fallback). Closes #105. * buildbot/steps/shell.py (ShellCommand._interpolateProperties): apply patch from Albert Hofkamp to allow WithProperties in environment variables and the workdir= argument. Closes #52. * buildbot/test/test_properties.py (Interpolate.testCustomBuildStep): remove this test, it was pretty weird and didn't work with the new _interpolateProperties approach. * buildbot/test/test_vc.py (VCBase.connectSlave): turn off usePTY for all slave commands: I'm seeing 'cp' and 'rm' fail with SIGHUP, probably when we close stdin. This happens on my debian/sid system, but not on older unixes, so I suspect some change in behavior in the kernel or glibc or something similar. 2008-02-05 Brian Warner * buildbot/buildslave.py (BuildSlave._missing_timer_fired): add a To: header to the email message we're composing. Oops. Thanks to Zandr Milewski for the catch. * buildbot/test/test_slaves.py (BuildSlave.test_mail_on_missing): test for it 2007-12-31 Brian Warner * buildbot/test/test_transfer.py (Upload.testLotsOfBlocks): remove the short timeouts, these can cause false test failures on slow systems, and were only needed while developing the recent patch (Download.testLotsOfBlocks): same * buildbot/scripts/reconfig.py (Reconfigurator.run): if we're on windows, make 'buildbot reconfig' emit a warning message (since there's no SIGHUP to be had) rather than just hanging. Addresses #105, but we still need to document this somewhere. Thanks to Ben Hearsum for the patch. * buildbot/slave/commands.py (SlaveFileUploadCommand.start): rewrite the main loop to avoid tail recursion, since Deferreds don't optimize those out. Without this, at the end of the transfer we'd hit a stack depth of about two or three frames per block transferred, and for large transfers this would run into a "stack depth exceeded" error. Closes #106. (SlaveFileDownloadCommand.start): same, for downloads. * buildbot/test/test_transfer.py (Upload.testLotsOfBlocks): test it (Download.testLotsOfBlocks): same 2007-12-22 Brian Warner * buildbot/steps/shell.py (TreeSize): This never worked properly. Fix it. Also, stash the tree size in a build property named 'tree-size-kb'. Note that this records size in KiB (1024 bytes), not KB (1000 bytes), because that's what du will give us. This closes #47. * docs/buildbot.texinfo (TreeSize): document it * buildbot/test/test_steps.py (TreeSize.testTreeSize): test it * buildbot/steps/shell.py (WarningCountingShellCommand): utility subclass which uses a regular expression to count warning messages in the output. This also puts warning messages in a new 'warnings' logfile, sets the 'warnings-count' build property with the total count, and marks the step results as WARNINGS if there are any warnings. Thanks to Roch Gadsdon (rochg) for the patch. Closes #74. (Compile): derive from WarningCountingShellCommand (Test): same * docs/buildbot.texinfo (Compile): document it * buildbot/test/test_steps.py (Warnings): test it * buildbot/slave/commands.py (ShellCommand._chunkForSend): don't send more than 128kB of stdout/stderr/logfile text in a single PB string, since PB has a hardwired limit of 640kB. This should fix crashes that occur when the test process generates output very very quickly. Patch by chops@demiurgestudios.com, thanks! I haven't been able to reproduce this locally, but the potential bug is clear. This should close #129. 2007-12-21 Brian Warner * buildbot/scripts/startup.py (start): check to see if there's a buildbot.tac before doing anything else. This improves the error message when you do 'buildbot start' in the wrong directory. * buildbot/scripts/logwatcher.py (LogWatcher.start): rewrite to spawn /usr/bin/tail to follow the target's twistd.log . It turns out that OS-X file EOF semantics are different, and the naive read-then-read-again approach I was using doesn't work (the first EOF we hit is sticky). tail gets it right, so rely upon it instead of reimplementing the wheel. This should make 'buildbot start' correctly follow the logfile on OS-X, thus closing #141. 2007-12-14 Brian Warner * buildbot/steps/shell.py (ShellCommand.__init__): stash the RemoteShellCommands args too, so things like 'workdir' and 'env' and 'timeout' are preserved. This ought to fix #150. * buildbot/test/test_steps.py (Steps.testClone): test it 2007-12-11 Brian Warner * buildbot/scripts/startup.py (launch): use twisted.__version__, rather than twisted.version, since the latter didn't appear until Twisted-2.4.0 . This unbreaks 'buildbot start' on older Twisteds. 2007-12-08 Brian Warner * docs/buildbot.texinfo (Git): document the fact that we require Git 1.2.0 or later, and that the one shipped in dapper is too old. * buildbot/scripts/startup.py (launch): twisted-2.5.0 and later doesn't need twisted.scripts._twistw.run; instead, it uses twisted.scripts.twistd.run on all platforms. Closes #53. 2007-11-30 Brian Warner * buildbot/status/builder.py (BuilderStatus): small docs patches from Greg Ward.. thanks! * buildbot/process/base.py (Build): same 2007-11-29 Brian Warner * buildbot/slave/commands.py (Mercurial._maybeFallback): add an _abandonOnFailure call to the fallback, to skip the 'update' step if the checkout fails. (Mercurial._maybeFallback): More fixes: look for error messages generated by every version of hg from 0.7 to the current 0.9.5, including development versions as of rev 92991422a847. Also make sure to return the exit code of non-fallbackable errors so that other steps can attempt a workaround. 2007-11-28 Brian Warner * buildbot/slave/commands.py (Mercurial._maybeFallback): improve hg fallback so that it works when the client hg is new and the server hg is old (the error message is different in this case). 2007-11-27 Brian Warner * buildbot/test/test_p4poller.py: hush pyflakes warning * buildbot/slave/commands.py (Mercurial._maybeFallback): when hg is doing a 'clone' operation, to a remote repository (e.g. over HTTP), and either the server or the client is hg-0.9.1 or older, 'hg clone' cannot accept a --rev argument: instead, you have to clone the tip and then update back to the desired revision. Added code to detect this failure mode and fall back to the clone+update scheme. Closes #122 and #103. (ShellCommand.__init__): add a keepStderr= flag to support this * buildbot/test/test_vc.py (VCBase._do_vctest_clobber_2): add a test that does a checkout to a specific version, to exercise the mercurial limitation workaround. (Mercurial.serveHTTP): add code to test hg checkout over HTTP, using mercurial's built-in 'hg serve' command. Because of limitations in hg, this code uses a randomly-selected TCP port (rather than claiming a free one), so every once in a while this test is going to fail because of a port-number collision. * buildbot/test/test_vc.py (GitHelper.capable): skip Git tests if the version of Git installed is older than 1.2.x . Git 1.1.3 doesn't accept 'git init' (it wants 'git init-db' instead) and the branch tests fail. I know that Git 1.5.3.6 works. I don't know what the dividing line is.. if someone can figure it out, please update this check. Also, if someone can make buildbot's git support handle older versions, please do that too! Addresses #130. 2007-11-21 Brian Warner * lots: improve Git support, AMAZING patch from Haavard Skinnemoen, complete with unit tests and docs, thanks! Closes #130. * buildbot/scripts/tryclient.py (GitExtractor): add 'try' support * buildbot/slave/commands.py (Git): accept branch and revisions, and use 'git' instead of the 'cogito' wrapper * buildbot/steps/source.py (Git.__init__): same * buildbot/test/test_vc.py (BaseHelper.runCommand): offer control over env= (BaseHelper.do): same (BaseHelper.dovc): same (Git): unit tests for Git support. Wow! * docs/buildbot.texinfo (Git): docs for Git support. Double Wow! * contrib/git_buildbot.py: commit-hook script for git * buildbot/changes/p4poller.py (P4Source._finished): don't let a failure cause us to stop polling. Thanks to John Backstrand for the patch. Closes #135. * buildbot/test/test_p4poller.py (TestP4Poller.testFailedChanges): change this to match (TestP4Poller.testFailedDescribe): same * buildbot/status/web/waterfall.py (WaterfallStatusResource.body): fix timezone calculation, we were always emitting DST, even in winter. Thanks to Charles Lepple and John Saxton for the patch. Closes #137. * buildbot/test/test_web.py (Waterfall.test_waterfall._check1): same * lots: a whole batch of patches from Benoit Sigoure, referenced in ticket #138. Thanks! * buildbot/master.py (BuildMaster.loadConfig): builder names that begin with an underscore are now reserved, so prohibit them from appearing in the config file. * NEWS: announce this change. * buildbot/status/web/base.py: add buttons (to one_line_per_build and one_box_per_builder) to force/stop builds on all Builders at once. * buildbot/status/web/baseweb.py: same (OneBoxPerBuilder.body): add links to the per-Builder page * buildbot/status/web/builder.py: same * buildbot/status/web/builder.py (StatusResourceBuilder.body): factor out force/stop form generation * buildbot/status/web/base.py: same * buildbot/status/web/build.py: same * buildbot/status/web/builder.py (StatusResourceBuilder.force): simplify some 'if' statements (StatusResourceBuilder.build_line): add a 'Stop Build' button next to the 'Force Build' button, so you can start and stop a build from the same place. (StatusResourceBuilder.force): redirect back to the waterfall, rather than the index page (StatusResourceBuilder.ping): same * buildbot/status/web/build.py (StatusResourceBuild.stop): same * buildbot/slave/bot.py (Bot.remote_setBuilderList): don't consider the 'info' directory as a leftover directory. Patch from Benoit Sigoure, ticket #138. (others): remove trailing whitespace 2007-11-01 Brian Warner * buildbot/status/web/builder.py (StatusResourceBuilder.build_line): if there's no current step, don't try to display its description. Same crash as in #96, this time in the web page. Closes #111. * buildbot/status/web/base.py (OneLineMixin.make_line): tolerate results == None (i.e. when the build hasn't finished yet); previously this code exploded when trying to find a CSS class for that results value. Patch from Greg Ward. Closes #118. * buildbot/status/client.py (RemoteBuildStep.remote_getLogs): IBuildStepStatus.getLogs() returns a list, not a dict. Patch from Fabrice Crestois. Closes #121. 2007-10-31 Brian Warner * buildbot/status/words.py (Contact.emit_status): if there's no current step, don't try to display its description. This should fix the crash when asking the irc bot about builds which are waiting for a build-wide Lock. Closes #96. * buildbot/status/tinderbox.py (TinderboxMailNotifier): add 'columnName' argument, from a patch by Ben Hearsum. Closes #120. 2007-10-16 Brian Warner * buildbot/status/web/slaves.py (BuildSlavesResource.body): fix timestamp, I was using %M (minutes) where I meant to be using %b (abbreviated month name). Closes #109. 2007-10-13 Brian Warner * README (COPYING): make it clear that buildbot is distributed under the GPL (version 2) * COPYING: add a copy of the GPLv2 to the source tree * MANIFEST.in: .. and to the source distribution too 2007-10-10 Brian Warner * buildbot/status/builder.py (Status.getURLForThing): update this to match new URL layout, I completely forgot about it. Closes #112. * buildbot/test/test_web.py (GetURL): add unit tests * buildbot/status/web/build.py (BuildsResource.getChild): fix silly typo which prevented the Build page from ever offering control options like 'Stop Build'. Thanks to Aaron Hsieh for the catch. Closes #114. * buildbot/test/test_webparts.py (Webparts._do_page_tests): it'd be nice to test this * buildbot/__init__.py (version): bump to 0.7.6+ while between releases * docs/buildbot.texinfo: same 2007-09-30 Brian Warner * buildbot/__init__.py (version): Releasing buildbot-0.7.6 * docs/buildbot.texinfo: same 2007-09-30 Brian Warner * NEWS: small updates, remove duplicate items from the 0.7.5 notes * setup.py: install all test files from buildbot/test/mail/* . Also update URL to point at buildbot.net * MANIFEST.in: update with new files * buildbot/status/web/baseweb.py (WebStatus.setupSite): ugh, yes auto-create an empty public_html directory, otherwise we get internal server errors instead of 404s. * README: reference buildbot.net, remove CVSToys section (does anyone still use it?) * buildbot/status/web/baseweb.py (WebStatus): emit a useful log message when public_html/ is missing (indicating that you should probably run 'upgrade-master'), and don't auto-create an empty directory in that case. * contrib/darcs_buildbot.py: refactor internals a bit, make it possible to use as an importable module (to make it easier to write some unit tests for it) * buildbot/status/web/baseweb.py (Waterfall.__init__): add a DeprecationWarning for Waterfall, advising users to switch over to WebStatus instead. * NEWS: update with user-visible changes since the last release * buildbot/status/web/waterfall.py (WaterfallStatusResource.get_reload_time): if the suggest a reload time of less than 15 seconds, give them 15 seconds instead of ignoring their request entirely, because that'd be confusing. * buildbot/steps/source.py (Mercurial.computeSourceRevision): log a warning if we must invoke the last-change-is-most-recent guess * buildbot/status/web/baseweb.py (Waterfall.__init__): if an old Waterfall is running in an upgraded master (which is the usual case for folks who have followed the instructions to run 'upgrade-master' but who have not yet resolved all the DeprecationWarnings), prefer the buildbot.css from public_html/ rather than the one passed in on the buildbot= argument. Otherwise, the resulting pages don't quite use CSS right.. sometimes it works, sometimes it doesn't, and I don't know why. Sites that are using custom CSS will see the default CSS until they modify public_html/buildbot.css to include their customizations. * docs/buildbot.texinfo (WebStatus): explain robots.txt a bit more * buildbot/status/web/robots.txt (Disallow): update to cover all dynamically-generated pages * buildbot/status/web/base.py (map_branches): when we see 'trunk', include both 'trunk' and None in the output list, since some VC systems (SVN, frequently) refer to trunk as 'trunk'. * buildbot/status/web/waterfall.py (WaterfallStatusResource): point the 'welcome' link explicitly at index.html, rather than '.', so that old Waterfall users can reach it and discover all the new pages. * buildbot/status/web/base.py (OneLineMixin.make_line): add the build's text to the end of the line, without a CSS class so it remains uncolored. * buildbot/interfaces.py (IStatus.generateFinishedBuilds): add a 'max_search=' argument, to limit the number of builds that are examined while trying to find ones that match the other criteria. This helps to limit the work we do, since otherwise we might have to trawl through all history. (IBuilderStatus.generateFinishedBuilds): same * buildbot/status/builder.py (Status.generateFinishedBuilds): same (BuilderStatus.generateFinishedBuilds): same * buildbot/status/web/base.py (OneLineMixin.make_line): stringify our got_revision before checking its length: SVN (at least) still reports numeric values for this. * buildbot/status/web/build.py (StatusResourceBuild.body): same * buildbot/status/web/xmlrpc.py (XMLRPCServer.xmlrpc_getAllBuildsInInterval): same, also stop serving a completely fake revision * buildbot/scripts/runner.py (Maker.check_master_cfg): prepend basedir to sys.path, to mimic what a real buildmaster would have access to, since it is common for master.cfg to import helper classes from .py files in the buildmaster directory, and we want config-file checking to accomodate this. * buildbot/status/web/base.py (map_branches): when referring to branches through WebStatus (i.e. by appending ?branch=FOO query arguments to the URL), make "trunk" mean trunk, by mapping it to None before passing it to generateFinishedBuilds() and the like. * buildbot/status/web/baseweb.py (OneLinePerBuild.body): same (OneBoxPerBuilder.body): same, also improve display of branches in the HTML a bit * buildbot/status/web/waterfall.py (BuildTopBox.getBox): same (WaterfallStatusResource.buildGrid): same. Note that this filters Changes as well as Builds. * docs/buildbot.texinfo (Buildbot Web Resources): remove the caveat about trunk branches, now that it's fixed * buildbot/status/web/base.py (OneLineMixin): refactor, move this from builder.py, use it on the Builder page too * buildbot/status/web/builder.py (StatusResourceBuilder): same * buildbot/status/web/baseweb.py (OneLinePerBuild): display branch names, add some CSS classes * buildbot/status/web/classic.css (td.box): new class, for OneBoxPerBuild * docs/buildbot.texinfo (WebStatus): update docs * buildbot/slave/commands.py (SVN.parseGotRevision._parse): remove the trailing "M" indicator that tells us this is a modified file. This should help with mode="update" builds that modify files in-place. Thanks to "Oz" (chris at santacruzgames.com) for the patch. * buildbot/scripts/runner.py (upgradeMaster): improvements: write new copies of files (to .new) when those files already exist, check the master.cfg file for errors (and give DeprecationWarnings a chance to be displayed). * buildbot/test/test_runner.py: add more tests * docs/buildbot.texinfo (Upgrading an Existing Buildmaster): docs 2007-09-29 Brian Warner * buildbot/steps/source.py (Mercurial.computeSourceRevision): take the revision id from the last change in our list, since that's the best we can do without knowing the full ancestry graph. Closes #103. * buildbot/scripts/sample.cfg: use buildstep instances in f.addStep, to demonstrate the (correct) modern usage. * docs/examples/hello.cfg: bring this up to date. Closes #79. * docs/examples/twisted_master.cfg: add a big warning about how old this is * docs/examples/glib_master.cfg: delete this one entirely, it is ancient and not a very useful example anyways * buildbot/status/web/base.py (HtmlResource.render): keep track of HTTPChannels, so we can shut them down at reconfig time (when the WebStatus goes away). Closes #102. * buildbot/status/web/baseweb.py (WebStatus.__init__): same (WebStatus.registerChannel): same (WebStatus.stopService): do .loseConnection() here * buildbot/test/test_webparts.py (Webparts.testPages): reload the WebStatus, make sure all pages are still retrievable. I thought this would exercise a bug, but it turns out that it was firefox caching connections (and continuing to talk to the old WebStatus even though I'd loaded a new one). * buildbot/status/web/baseweb.py (WebStatus): add some comments (WebStatus.__repr__): include id() in the repr * buildbot/status/tinderbox.py (TinderboxMailNotifier): allow this to specify an errorparser. From Ted Mielczarek and the mozilla crew. Closes #94. * buildbot/changes/bonsaipoller.py (BonsaiParser): "Make bonsaipoller ignore funkyness in xml output for empy log messages", from Axel Hecht. Closes #90. * buildbot/test/test_bonsaipoller.py (TestBonsaiPoller.testMergeEmptyLogMsg): same * buildbot/status/tinderbox.py (TinderboxMailNotifier.buildStarted): respect 'builders' arg, from Ben Hearsum. Closes #89. * buildbot/scripts/runner.py (stop): increase the timeout from 5 seconds to 10, to give heavily loaded machines a better chance to finish shutting down. Addresses (partially) #68. * buildbot/scripts/logwatcher.py (LogWatcher.TIMEOUT_DELAY): same, on startup * buildbot/scripts/reconfig.py (Reconfigurator.run): same * buildbot/scripts/startup.py (Follower._failure): update message * buildbot/changes/bonsaipoller.py: apply changes from the Mozilla team, to fix some bugs and avoid the use of blocking urlopen(). closes #61. * buildbot/test/test_bonsaipoller.py: same 2007-09-28 Brian Warner * buildbot/buildslave.py (BuildSlave): add notify_on_missing= argument, to send email about buildslaves which are disconnected for more than an hour. Closes #64. * buildbot/test/test_slaves.py (BuildSlave.test_mail_on_missing): test it * docs/buildbot.texinfo (When Buildslaves Go Missing): document it (MailNotifier): add docs for this too * buildbot/status/web/baseweb.py (OneBoxPerBuilder): make this page respect branch= queryargs, by restricting the builds displayed to those matching the given branch names * buildbot/status/web/waterfall.py (BuildTopBox): same * buildbot/test/test_web.py (WaterfallSteps.test_urls.FakeRequest): fix a test failure by adding .prepath to this fake object * buildbot/status/web/*: big set of changes to WebStatus, cleaning up generation of relative URLs. With luck, this should play with reverse proxies much better now. * buildbot/test/test_webparts.py: new test for most of the subpages * buildbot/test/test_web.py (base_config): add enough of a config to exercise more Waterfall code 2007-09-27 Brian Warner * buildbot/status/web/build.py (StatusResourceBuild.body): use a relative URL to the buildbot welcome page, rather than getBuildbotURL. * buildbot/status/web/builder.py (StatusResourceBuilder.body): same * buildbot/status/web/base.py (HtmlResource.title): change the default title of all Buildbot-generated page to "Buildbot" * buildbot/status/web/builder.py (StatusResourceBuilder.getTitle): change the title to include the Builder name * buildbot/status/web/build.py (StatusResourceBuild.getTitle): same, and also show the build number (StatusResourceBuild.rebuild): after using the 'Rebuild' button, redirect the browser to the Builder page 2007-09-26 Brian Warner * buildbot/status/web/baseweb.py (OneBoxPerBuilder.body): fix URL problem with the build links 2007-09-25 Brian Warner * buildbot/status/web/base.py (HtmlResource.render): fix addSlash to emit relative URLs instead of using URLPath, * docs/buildbot.texinfo (WebStatus): provide an example * buildbot/status/web/baseweb.py (OneBoxPerBuilder): new status page, shows a simple table with one row per Builder. Still pretty ugly, but functional. * buildbot/status/web/index.html: reference it * buildbot/status/web/waterfall.py (Spacer): make this *not* inherit from builder.Event, so that show_events=false doesn't hide spacers, since that would make the waterfall weirdly compressed. * buildbot/status/web/waterfall.py (HELP): put the View button in its own section, so it doesn't get visually confused with the reload-timer section (WaterfallStatusResource.body): make the '[help]' links more visually distinct from each other, and add a link to the Welcome page * buildbot/status/web/slaves.py (BuildSlavesResource): handle unused buildslaves without exploding * buildbot/status/web/waterfall.py (WaterfallStatusResource.body.with_args): don't use req.URLPath, it gives us absolute paths that break proxies * buildbot/status/web/base.py (HtmlResource.path_to_root): fix this, it was giving bad results when the Waterfall is behind a reverse proxy. * buildbot/scripts/runner.py (upgradeMaster): first phase of the new 'upgrade-master' command, which will bring an old buildmaster directory up-to-date. Right now the only thing it does is populate public_html/ if it wasn't already there. Still to do: check the other files, add documentation. * buildbot/test/test_runner.py (Create.testUpgradeMaster): test it (Create.failUnlessSameFiles): use sets.Set() for 2.3 compatibility 2007-08-13 Brian Warner * buildbot/changes/changes.py (Change.get_HTML_box): add a tooltip that contains the checkin comments when you hover over the entry in the 'changes' column of the waterfall. Thanks to Frederic Leroy for the patch. * buildbot/status/web/waterfall.py (WaterfallStatusResource): improve 'help' page linkage * buildbot/status/web/about.py (AboutBuildbot): add an 'about' page with versions of python, buildbot, and twisted * buildbot/status/web/baseweb.py (WebStatus.setupUsualPages): same * buildbot/status/web/index.html: add links to most pages * docs/buildbot.texinfo (Buildbot Web Resources): update docs * buildbot/status/web/robots.txt: put a robots.txt in new installs which discourages robot access to all large dynamically-generated pages. The intention is that the index page and the 'about' page are the only ones which should be crawlable. * buildbot/scripts/runner.py (Maker.public_html): same (createMaster): same * setup.py: ship index.html and robots.txt in source distributions * buildbot/status/web/waterfall.py: add 'refresh' query arg, to activate automatic reloading of the page (using a meta Refresh: tag). Closes #69. (WaterfallHelp): new page to explain all the nifty query arguments you can use on the Waterfall, along with forms to set them for you. From this page, you can add filters for builder=, branch=, show_events=, as well as turning on reload=. (WaterfallStatusResource.buildGrid): new query args: last_time=, first_time=, show_time=, num_events= . The old show= is still accepted, but the new builder= is preferred. We ignore empty branch= arguments, to make the WaterfallHelp form easier to implement. (WaterfallStatusResource.head): new method to add text to the of the page. * buildbot/status/web/base.py (HtmlResource.content): use head() * buildbot/status/web/baseweb.py (OneLinePerBuild.body): ignore empty branch= arguments (OneLinePerBuildOneBuilder.body): same 2007-08-12 Brian Warner * buildbot/status/web/waterfall.py (WaterfallStatusResource): add new query args: 'last_time' is a timestamp of the top-most event on the display, 'first_time' is a timestamp for the bottom-most event. 'show_time' is the difference between them, and overrides 'first_time'. 'num_events' puts an upper limit on the number of timestamps that will be displayed on the left hand edge. (WaterfallStatusResource.body): add a 'next page' link to the bottom, which shows the events that come just after those shown on the current page. This is a first step towards #67, but we need some more controls before we can close that one. * buildbot/buildslave.py (BuildSlave): these are now MultiService instances too, and live as children of the BotMaster. The main advantage of this approach is that BuildSlaves can know when they are being removed (so they can shut off the upcoming when-do-we-email-an-admin-because-we-lost-our-slave timer). * buildbot/interfaces.py (IBuildSlave): marker interface so we can identify which service children are BuildSlaves * buildbot/master.py (BuildMaster.loadConfig_Slaves): handle the Checker updates here, but delegate the rest to.. (BotMaster.loadConfig_Slaves): .. here, which looks at the BotMaster's service children to identify the old slaves. I think the previous approach of using master.slaves was broken in the face of config updates that modified BuildSlave instances but did not completely replace them. * docs/buildbot.texinfo (Developer's Appendix): note the change to the Service hierarchy * buildbot/status/web/slaves.py (BuildSlavesResource): new status page (at URL /buildslaves/) to view status of all buildslaves at once, including how long it's been since we last heard from them, and which Builders use them. * buildbot/status/web/baseweb.py: fix title of one-line-per-build pages * buildbot/interfaces.py (IStatus.getSlaveNames): new method to list all buildslaves * buildbot/buildslave.py (BuildSlave.messageReceivedFromSlave): use the BuildSlave to keep track of the last time we've heard from the buildslave. This is updated upon receipt of any message from any SlaveBuilder (which generally means when RemoteCommands are giving us status updates, as BuildSteps run). In the future this will include slave pings too. * buildbot/process/base.py (Build.setupBuild): give each BuildStep a reference to the BuildSlave * buildbot/process/buildstep.py: (BuildStep.setBuildSlave): accept that reference (BuildStep.runCommand): pass the reference on to the RemoteCommand (RemoteCommand.remote_update): update the timestamp (RemoteCommand.remote_complete): same * buildbot/test/runutils.py (StepTester.makeStep): match the change * buildbot/test/test_steps.py (BuildStep.testShellCommand1): same * buildbot/interfaces.py (ISlaveStatus.lastMessageReceived): new interface to retrieve this timestamp * buildbot/status/builder.py (SlaveStatus): track a copy of the timestamp * buildbot/status/web/baseweb.py (OneLineMixin.make_line): tighten up the format a bit (OneLinePerBuild.body): use a
    instead of
    s (OneLinePerBuildOneBuilder.body): same * contrib/darcs_buildbot.py (makeChange): handle moved files too, by adding the file's new name to the list of changed files. This ought to be sufficient for things like isFileImportant and trial tests that only exercise tests for files that have been modified. Closes #31. * buildbot/status/web/xmlrpc.py (XMLRPCServer): add preliminary XMLRPC server to the WebStatus page. This only has two methods right now (getAllBuildsInInterval and getBuild), but we'll be adding more in the future. This server is extracted from the trac-plugin branch. * buildbot/status/web/baseweb.py (OneLinePerBuild): refactor using generateFinishedBuilds, and improve the data displayed. * buildbot/status/web/build.py (StatusResourceBuild): rearrange the per-Build page, add some information like ETA and got_revision * buildbot/status/web/builder.py (StatusResourceBuilder): same, adding a list of recently completed builds * buildbot/status/web/base.py (css_classes): factor this out * buildbot/interfaces.py (IStatus.generateFinishedBuilds): define new method, to make status display classes easier to write, especially the WebStatus 'one_line_per_build' page. (IBuilderStatus.generateFinishedBuilds): same * buildbot/status/builder.py (BuilderStatus.generateFinishedBuilds): (Status.generateFinishedBuilds): implement it * buildbot/process/factory.py (GNUAutoconf.__init__): allow 'source' to be a BuildStep instance, since BuildFactory accepts them now. (CPAN.__init__): same (Distutils.__init__): same (Trial.__init__): same * buildbot/interfaces.py (IBuildStatus.getProperty): note that this might raise KeyError 2007-08-07 Brian Warner * buildbot/slave/commands.py (P4): use 'p4user' to construct the 'Owner:' field of the view, rather than 'p4logname', since p4logname comes from the buildslave's environment and seems unlikely to ever do the right thing. Thanks to Wade Brainerd for the patch. Closes #40. * buildbot/buildslave.py (BuildSlave.__init__): add max_builds=, which imposes a per-slave limit on how many builds are allowed to run simultaneously. This has a the same scope than the SlaveLock, but is different because max_builds= gives the buildmaster the freedom to assign the build to a different slave, whereas the SlaveLock doesn't get tested until after the build is irrevocably assigned to a slave. Therefore using max_builds= will improve utilization in the presence of multiple buildslaves that are attached to the same Builder. This completes the incorporation of Dustin Mitchell's patches, and closes ticket #48. Thanks Dustin! * buildbot/process/builder.py (SlaveBuilder.buildFinished): when any Builder finishes, potentially trigger *all* Builders, since max_builds= may have stalled someone else while waiting for the slave. * buildbot/scripts/sample.cfg: mention max_builds= * buildbot/test/test_run.py (ConcurrencyLimit): test it * docs/buildbot.texinfo (Buildslave Specifiers): document it * buildbot/test/test_run.py (CanStartBuild._do_test2): tests which inherit from RunMixin do not need to call master.stopService() themselves, since RunMixin.tearDown does that. The double call fails on Twisted-2.0.x, which didn't happen to protect against it. * buildbot/test/test_slaves.py (BuildSlave.test_track_builders): same * buildbot/buildslave.py (BuildSlave.addSlaveBuilder): third patch from #48, this one to have the BuildSlave objects track SlaveBuilders, so they'll be able to use that knowledge to influence their canStartBuild() response. * buildbot/process/builder.py (SlaveBuilder.attached): call it (SlaveBuilder.detached): same * buildbot/test/test_slaves.py (BuildSlave): test it * buildbot/buildslave.py (BuildSlave.canStartBuild): incorporate the second of Dustin's #48 patches, this one adding a method to the BuildSlave that allows it to decide whether it is willing to participate in a build or not. (SlaveBuilder.isAvailable): use canStartBuild() to decide * buildbot/test/test_run.py (CanStartBuild): test for it * all: incorporate the first of four patches by Dustin Mitchell from ticket #48, working towards improving control over slave concurrency. The first patch is to use a long-lived BuildSlave object per slave, on which other behavior can be added later. I've modified his patch considerably. * buildbot/buildslave.py (BuildSlave): move class out of slave/__init__.py (where it would get loaded by the slave as well as the master, making dependencies trickier), and merge it with BotPerspective. Now these BuildSlave instances are created once in the master.cfg file, and not destroyed until they are either removed from the config file or changed so much (name/password/class changes) that we cannot continue to use the old one. This affects config files: they must use 'from buildbot.buildslave import BuildSlave' instead of using 'from buildbot.slave import BuildSlave'. * buildbot/master.py: move BotPerspective out (BuildMaster.loadConfig_Slaves): examine old and new slaves, add or remove as necessary, update the rest using data from the new config file. * buildbot/slave/__init__.py: move BuildSlave out * buildbot/test/test*.py: import BuildSlave from the new place * buildbot/scripts/sample.cfg: update to match * docs/buildbot.texinfo: update to match * buildbot/process/builder.py: update comments to match 2007-08-02 Brian Warner * buildbot/status/web/baseweb.py (OneLinePerBuild): improve one-line-per-build pages a bit, add a header, refactor slightly * buildbot/test/test_maildir.py (MaildirTest.testMaildir): cleanup the TimeOutError timer, patch by Dustin Mitchell. 2007-08-01 Brian Warner * buildbot/status/web/baseweb.py (OneLinePerBuild): make this mostly work (branch= is still ignored, we need a useful header) (OneLinePerBuildOneBuilder.body): this too * buildbot/status/web/builder.py (StatusResourceBuilder.force): bring this up-to-date w.r.t. recent refactorings, and fix redirections to bounce the browser to the top page after hitting the 'force' button. This needs to be improved, but at least it now redirects to a valid page. (StatusResourceBuilder.ping): same (StatusResourceBuilder.getChild): same, although I think this code might be dead anyways * buildbot/status/web/waterfall.py (WaterfallStatusResource): add a branch= query argument, to filter the display down to builds and changes that involve the given branch. You can add multiple branch= arguments to see multiple branches (using a logical OR). Note that there is no way to include the default branch (i.e. None, i.e. trunk) yet, there is still work to be done to allow for simple+uniform names of branches (i.e. removing the VC-specific details of how branches are implemented, so calling the branch 'beta4' even though the SVN checkout step requires 'branches/beta4'). * buildbot/status/builder.py (BuilderStatus.eventGenerator): same * buildbot/changes/changes.py (ChangeMaster.eventGenerator): same * buildbot/interfaces.py (IEventSource.eventGenerator): same * docs/buildbot.texinfo (Buildbot Web Resources): document it * all: bring import statements up-to-date: don't import __future__, assume cPickle and cStringIO are always available. Also remove #!/usr/bin/python lines from non-scripts, my editor has been a bit over-enthusiastic about stuffing them into new .py files. * buildbot/status/web/index.html: put a link to the waterfall page on the index, otherwise first-time users of WebStatus aren't going to see anything very interesting. * buildbot/interfaces.py (IEventSource): document this interface. Thanks to Dustin Mitchell for the patch. Closes #60. * buildbot/status/builder.py (BuilderStatus): same * buildbot/changes/changes.py (ChangeMaster): same * buildbot/status/web/waterfall.py (WaterfallStatusResource.buildGrid): add a query arg of 'show_events=true' to display non-build events, like slaves attaching/detaching and reconfig events, or 'show_events=false' to hide them. The default is 'true'. * docs/buildbot.texinfo (Buildbot Web Resources): document it * web-parts: merge in web-parts branch. Lots of changes. The new functionality is to add buildbot.status.html.WebStatus, which is a superset of the Waterfall that adds new status pages and serves regular files from BASEDIR/public_html/ , which is now the preferred place to put robots.txt, buildbot.css, and index.html . * buildbot/scripts/runner.py: the 'create-master' command now creates public_html/ too, and populates it with a couple of static files. * buildbot/status/web: split waterfall code into smaller pieces 2007-07-31 Brian Warner * buildbot/scripts/startup.py (launch): import twistd.run in a different way to hush pyflakes * buildbot/process/builder.py: remove unused import 2007-07-30 Brian Warner * buildbot/interfaces.py (ISourceStamp): cleanup patch by Dustin Mitchell: pass SourceStamps around rather than branch/version/etc tuples. Thanks Dustin! Closes #70. * buildbot/sourcestamp.py (SourceStamp): insist that .changes is a tuple rather than a list, to avoid surprising mutations * buildbot/process/builder.py (BuilderControl.resubmitBuild): same * buildbot/status/mail.py (MailNotifier.buildMessage): same * buildbot/status/builder.py (BuildStatus.getSourceStamp): same * buildbot/test/test_status.py (Subscription): same * buildbot/test/test_control.py (Force): same * buildbot/test/test_buildreq.py (Request.testMerge): same 2007-07-29 Brian Warner * buildbot/scripts/sample.cfg: remove the spurious 'builders' variable, since it wasn't used and is confusing in the presence of c['builders'] (which *is* used). Thanks to Thomas Vander Stichele for the suggestion. * buildbot/status/web/waterfall.py (WaterfallStatusResource): update the Buildbot home-page link to point at buildbot.net instead of sourceforge, and add a current-version query parameter so we can get some statistics on which versions are in use out there. 2007-07-28 Brian Warner * buildbot/master.py (BuildMaster.loadConfig): replace c['sources'] with c['change_source'], leaving c['sources'] for backwards compatibility (but deprecated), scheduled for removal in 0.8.0 * docs/buildbot.texinfo: same * buildbot/test/test_config.py: verify that c['sources'] still works * buildbot/test/test_*.py: replace c['sources'] with c['change_source'] * buildbot/scripts/sample.cfg: update to match * buildbot/changes/mail.py (MaildirSource): parse with python's stdlib 'email' module, which has been available since python2.2, and drop use of the 'rfc822' module, which has been depreceated since python2.3 . * buildbot/test/test_maildir.py: same: use parse_file() not parse * buildbot/test/test_maildir.py (MaildirTest.testMaildir): oops, this needs to match the renaming in test/mail/msg* * README (SETTING UP A BUILD MASTER): fix capitalization-typo. Closes SF#1752648. * buildbot/test/test_mailparse.py: rename sample emails a bit * buildbot/test/mail/msg*: same * buildbot/changes/mail.py (SVNCommitEmailMaildirSource): add a parser for the "commit-email.pl" script that is shipped with SVN, written by Justin Mason. Thanks! Closes SF#1072845. * buildbot/test/mail/svn-commit.*: sample messages * buildbot/test/test_mailparse.py (TestSVNCommitEmail): test it * docs/buildbot.texinfo (SVNCommitEmailMaildirSource): docs 2007-07-27 Brian Warner * buildbot/changes/mail.py: refactor: move parsing into a method named 'parse', each parser type gets a separate subclass. Remove the old 'sep' argument (hardcode it to '/'), fix usage of prefix= to be a simple leading-substring match. Add a warning to the logs if prefix= does not end in a slash, since that's probably a mistake. Fix both places where a prefix= mismatch would skip all subsequent files in the same email, instead of just skipping the one that didn't match. * buildbot/test/test_mailparse.py: match changes * docs/buildbot.texinfo (Getting Source Code Changes): refactor docs on email-based ChangeSources * docs/buildbot.texinfo (Getting Source Code Changes): move all the changesource types up a level, removing the one-entry menu in the process * buildbot/master.py (BuildMaster.loadConfig_Slaves): rename c['bots'] to c['slaves'], and use buildbot.slave.BuildSlave instances instead of tuples. Closes #63. * buildbot/slave/__init__.py (BuildSlave): define marker class * buildbot/scripts/sample.cfg: use c['slaves'] in sample config * docs/buildbot.texinfo (Buildslave Specifiers): document c['slaves'] * buildbot/test/test_*.py: update tests to match * buildbot/test/test_config.py (ConfigTest.testBots): verify that the c['bots'] backwards-compatibility handler works 2007-07-26 Brian Warner * buildbot/changes/hgbuildbot.py (hook): add an in-process Mercurial change-sending hook, contributed by Frederic Leroy. Thanks! Addresses #50. * docs/buildbot.texinfo (MercurialHook): document it * buildbot/changes/hgbuildbot.py: change the way imports are done, to make it compatible with at least hg-0.9.1 and the current 0.9.4 2007-07-17 Brian Warner * buildbot/process/buildstep.py (BuildStep.getLog): add a convenience method to retrieve a log added with addLog()/addCompleteLog()/etc or logfiles= . I should have added this two years ago.. 2007-07-03 Brian Warner * buildbot/slave/commands.py (ShellCommand._startCommand): when logging the environment variables, put one variable on each line, rather than having one really long line with all of them. This should make them a bit more readable and not trigger the horizontal scrollbar (when viewing it in a web browser) quite so much. Thanks to Albert Hofkamp for the patch. * buildbot/steps/shell.py (ShellCommand): allow workdir= to be a WithProperties instance. Thanks to Axel Hecht for the patch. Closes #43. * buildbot/test/test_properties.py (Interpolate.testWorkdir): test it * docs/buildbot.texinfo (Build Properties): document it * buildbot/slave/commands.py (ShellCommand._startCommand): create the workdir if it didn't already exist. * docs/buildbot.texinfo (Scheduler Types): correct the description of Dependent schedulers. Thanks to Greg Ward for the patch. * buildbot/steps/source.py (SVN.startVC): when applying a patch, add "[patch]" to the step's display. Thanks to Dustin Mitchell for the patch. Closes #49. * buildbot/slave/commands.py (rmdirRecursive): chmod everything to 0700 before removing it, to deal with situations where a build will leave files around without write permissions and the 'copy' or 'clobber' VC checkout modes need to blow away the tree first. Thanks to Steve Milner for the suggestion and the patch (which I mangled horribly). Closes #29. * buildbot/test/test_slavecommand.py (Utilities.test_rmdirRecursive): test for it * buildbot/test/runutils.py (RunMixin.disappearSlave): oops, add an allowReconnect= argument to let callers control whether they want this don't-let-it-reconnect behavior, since some tests care. Factor out the .continueTrying=False lines from those callers. * buildbot/test/test_slaves.py: same * buildbot/test/test_run.py: same 2007-07-02 Brian Warner * buildbot/test/runutils.py (RunMixin.disappearSlave): once a slave has been disappeared, don't let it reconnect. This was causing an intermittent failure in test_slaves.py, when the slave that was supposed to be gone managed to come back by the end of the test and affect the count. * buildbot/test/test_slaves.py (Slave.testFallback2): minor refactorings * buildbot/process/builder.py (SlaveBuilder.__repr__): added some diagnostic messages to track down this problem (Builder.startBuild): same * buildbot/process/builder.py (Builder.maybeStartBuild): choose the slave randomly rather than always taking the first one. Thanks to Pike for the patch. Closes #36. * buildbot/test/test_slaves.py: match the change. to avoid huge changes to the tests, I added a CHOOSE_SLAVES_RANDOMLY flag which enables/disables the round-robin-ness (enabled by default), and some unit tests disable it. * docs/buildbot.texinfo (Buildmaster Architecture): document it * buildbot/slave/bot.py (BuildSlave.__init__): rename the 'host' argument to 'buildmaster_host', to make it more obvious that this points to the buildmaster. Thanks to Bob Proulx for the suggestion. * buildbot/scripts/runner.py (slaveTAC): same * buildbot/test/test_runner.py (Create.testSlave): match the change * contrib/hg_buildbot.py: patch from Frederic Leroy to make this work better. 2007-06-17 Brian Warner * buildbot/test/test_config.py (Factories.testAllSteps): make sure we can round-trip all of our current step classes by calling getStepFactory() and using the results to make a clone of the original step. * buildbot/steps/maxq.py: fix import errors. Guess this hasn't been used in a while.. * buildbot/process/factory.py (BuildFactory.addStep): To simplify the config file, we're moving to using actual instances instead of the (class, kwargs) 'step specification' tuples. BuildFactory still keeps a list of tuples internally, but when real instances are passed in to addStep(), they are asked for their class and kwargs using the new BuildStep.getStepFactory method. BuildFactory accepts both instances and the tuple form, and converts instances to the tuple form, but the instance form is preferred because it gives the Steps a chance to do argument validation. Closes: #11. * buildbot/process/buildstep.py (BuildStep.__init__): record the factory information necessary to implement getStepFactory. The addFactoryArguments() method can be used to include arguments that aren't passed to the BuildStep base class constructor. (BuildStep.setBuild): (BuildStep.setDefaultWorkdir): new methods to take parameters that are needed for live BuildSteps but not to construct the specification tuples that are stashed in the factory. * buildbot/process/base.py (Build.setupBuild): pass 'build' and 'workdir' into new BuildSteps by using methods instead of arguments. This makes the constructor for BuildSteps a lot simpler. * buildbot/steps/*.py: update to match this change. Basically this means adding calls to addFactoryArguments() in the __init__ methods to capture the arguments that aren't passed through to the base class. * buildbot/steps/shell.py (ShellCommand.setDefaultWorkdir): copy the new workdir (if any) into the RemoteShellCommands arguments. * buildbot/steps/source.py: allow workdir= to be optional, implement setDefaultWorkdir() since we don't inherit from (CVS): finally remove old clobber=/export=/copydir= arguments, in favor of the mode= argument that's been around forever now. * buildbot/steps/transfer.py: remove build= argument * buildbot/test/*.py: update to match, generally by turning all build= arguments into subsequent calls to s.setBuild() * buildbot/test/test_config.py (Factories): verify that we can use either BuildStep instances or class/kwarg tuples in both BuildFactory.addStep and BuildFactory.__init__ * docs/buildbot.texinfo (Build Steps): document the new approach, mention compatibility with the old approach, update all examples to use the new style. 2007-06-16 Brian Warner * buildbot/changes/svnpoller.py: when the poll fails, don't kill the LoopingCall, just eat the failure so that we'll poll again next time. This should allow us to tolerate (e.g.) sf.net SVN failures more gracefully. Many thanks to Dustin Mitchell for the patch. Closes #34. 2007-05-17 Brian Warner * buildbot/status/words.py: refactor the IRC status bot into separate 'Contact' and 'Channel' classes. The base Contact class contains the interaction code: commands and responses. The IRCContact subclass (and IrcStatusBot 'Channel') handle the IRC-specific aspects. The plan is to write other subclasses for other IM protocols like AIM and Jabber. 2007-04-16 Brian Warner * CREDITS: update list of contributors. Thank you all! 2007-04-13 Brian Warner * buildbot/status/mail.py (MailNotifier): add the project name to the subject line and message body, to make it easier to distinguish email coming from different buildmasters. Thanks to Benoit Sigoure for the patch. * buildbot/test/test_status.py (Mail): update to match 2007-03-24 Brian Warner * buildbot/steps/transfer.py: open all files in 'b' binary mode to avoid text-conversion problems between DOS/windows and unix. Thanks to Phil Thompson for the patch. Fixes SF#1674927. 2007-03-03 Brian Warner * buildbot/status/html.py (_hush_pyflakes): hush a pyflakes warning about the use of Waterfall here 2007-03-01 Brian Warner * buildbot/interfaces.py (IStatus.getBuilder): mention exceptions 2007-02-28 Brian Warner * buildbot/changes/p4poller.py (P4Source): apply change from Scott Lamb to use a more optimal form of 'p4 changes', to reduce server load. He reports that this optimization was added to p4d release 2005.2, but it should work in all versions. Closes #27. * buildbot/test/test_p4poller.py: match it 2007-02-27 Brian Warner * buildbot/status/web/*.py: move all web status stuff into a separate directory, in anticipation of splitting it into smaller pieces and adding more files. html.py was getting way too big. * buildbot/status/classic.css: move it too * setup.py: add the new sub-package * buildbot/test/test_web.py: match the changes * MANIFEST.in: handle the move of classic.css 2007-02-07 Brian Warner * contrib/OS-X/*: add some launchd .plist files for automatically starting a buildmaster or buildslave under OS-X (10.4 or later). Thanks to Mark Pauley for these. 2007-02-06 Brian Warner * docs/buildbot.texinfo (Requirements): update requirements to stop claiming compatibility with python-2.2 or twisted-1.3.x . Closes #13. * README: mention python-2.5 and twisted-2.5 compatibility * buildbot/clients/debug.py: hush pyflakes warnings by removing a redundant import * buildbot/scripts/startup.py: same, by removing twisted-1.3.0 compatibility * buildbot/status/builder.py: same, by requiring cStringIO * buildbot/status/words.py: same, remove twisted-1.3.0 compat * buildbot/test/test_vc.py: same * buildbot/test/test_web.py: * buildbot/test/test_steps.py: same, remove unused import * buildbot/status/html.py (StatusResourceBuild.body): oops, close the FORM tag * buildbot/master.py (BuildMaster.loadConfig): warn the user if we see any Builders that don't have Schedulers to drive them * buildbot/master.py (BotPerspective.__repr__): fit bitrot, the attributes this uses went away * contrib/bb_applet.py (MyApplet.filled): add a small prefs dialog, to allow you to reset the buildmaster and force connect/disconnect. Correctly handle losing the buildmaster connection (by switching the display to the hexnut, at which point you can use the 'Connect' menu item to reconnect). * buildbot/steps/source.py (Source.start): if we're using a patch, add it as a LogFile to the checkout/update step. This will turn into a link on the waterfall page. * buildbot/scripts/tryclient.py (Try.createJob): implement --diff option, to take the patch from a pre-made file rather than generating it from the local tree. Thanks to Robert Helmer for the idea. Closes #15 (the Trac ticket on the new http://buildbot.net/). * buildbot/scripts/runner.py (TryOptions): add --diff, --patchlevel, and --baserev options * buildbot/scheduler.py (Try_Jobdir.parseJob): treat a baserev of "" as None, meaning HEAD. * buildbot/sourcestamp.py (SourceStamp): update docstring to indicate that baserev=None means HEAD * docs/buildbot.texinfo (try --diff): document it * buildbot/test/test_steps.py (BuildStep): remove use of reactor.iterate(), although the technique I replaced it with is a gross polling hack that must be cleaned up properly some day. This was the last use of reactor.iterate in the entire tree, yay. 2007-02-05 Brian Warner * buildbot/status/mail.py (MailNotifier.__init__): assert that 'mode' is one of the three known values, otherwise we emit some confusing messages later on. Thanks to Grig Gheorghiu for the catch. 2007-01-30 Brian Warner * buildbot/changes/bonsaipoller.py (BonsaiParser.__init__): Ben Hearsum contributed a patch to let BonsaiPoller work with results that contain non-ascii characters. Closes SF#1632641. 2007-01-27 Brian Warner * Makefile (release): produce both .tar.gz and .zip source bundles. Closes SF#1222216. 2007-01-23 Brian Warner * buildbot/changes/freshcvsmail.py: hush a pyflakes warning * buildbot/changes/monotone.py: same * buildbot/changes/maildir.py: combine several files into one, clean up maildir.py to use Services properly. * buildbot/changes/maildirtwisted.py: remove * buildbot/changes/maildirgtk.py: remove, it wasn't used by buildbot anyways. * buildbot/changes/mail.py: match the change * buildbot/scheduler.py: same, since Try_Jobdir uses a maildir * buildbot/test/test_maildir.py: remove use of reactor.iterate() * buildbot/slave/commands.py (command_version): bump to "2.3" to indicate that the buildslave knows about the 'bzr' command 2007-01-22 Brian Warner * buildbot/process/builder.py (Builder.fireTestEvent): don't use 'with', it will become a reserved work in python2.6 . * contrib/bb_applet.py: add a simple gnome-panel applet, to display a terse summary of buildbot status. * docs/hexnut32.png, docs/hexnut48.png, docs/hexnut64.png: add some icons, small versions of the Blender object that lives in docs/images/icon.blend * buildbot/steps/source.py (Bzr): add Bazaar-ng support * buildbot/slave/commands.py (Bzr): same * buildbot/scripts/tryclient.py (BzrExtractor): same (SourceStampExtractor.dovc): modify this to allow non-zero exit status, since 'bzr diff' does that * buildbot/test/test_vc.py (Bzr): same * docs/buildbot.texinfo (How Different VC Systems Specify Sources): add notes on bzr (Bzr): document it 2007-01-20 Brian Warner * contrib/darcs_buildbot.py: tidy up the progress messages 2007-01-17 Brian Warner * contrib/darcs_buildbot.py: enhance to handle multiple patches being pushed at a time. This keeps state in the repository in a small file named .darcs_buildbot-lastchange, but that shouldn't interfere with normal repository operations. Fixes SF#1534049. * buildbot/clients/sendchange.py (Sender): make a single Sender capable of sending Changes with different usernames. 2006-12-11 Brian Warner * buildbot/changes/freshcvsmail.py: mark this file as deprecated, scheduled for removal in 0.7.7 . The FCMaildirSource was moved to buildbot.changes.mail a long time ago, but I forgot to add the DeprecationWarning until now. * buildbot/process/maxq.py: remove this, the functionality now lives in buildbot.steps.maxq * buildbot/clients/debug.py: remove an unnecessary gnome.ui import * buildbot/clients/gtkPanes.py: remove unused import * buildbot/slave/trial.py: remove an unused/incomplete/buggy file, that was meant to provide a special Reporter to run inside trial and give the buildslave lots of machine-readable status. We never finished this project. * buildbot/changes/bonsaipoller.py: remove unused imports * buildbot/changes/svnpoller.py: same * buildbot/process/builder.py: same * buildbot/process/buildstep.py: same * buildbot/slave/bot.py: hush pyflakes warning * buildbot/status/tests.py: remove unused imports * buildbot/status/tinderbox.py: same * buildbot/steps/python_twisted.py: same * buildbot/process/step.py: remove this file, these names were deprecated in 0.7.5 and are now being removed. * buildbot/process/step_twisted.py: same * buildbot/test/test_steps.py (ReorgCompatibility): remove test * buildbot/twcompat.py: remove the empty file, yay it is gone * buildbot/twcompat.py (waitForDeferred): remove the monkeypatch, tw-2.0.0 and newer have the method we need. (getProcessOutputAndValue): same (which): remove this method, tw-2.0.0 t.p.procutils has it * buildbot/twcompat.py (implements): remove this method * buildbot/*: import implements() from zope.interface directly * buildbot/status/mail.py: stop falling back to importing twisted.protocols.smtp.sendmail, now that we don't need tw-1.3.0 support * buildbot/twcompat.py (Interface): remove this import * buildbot/*: import Interface from zope.interface directly * buildbot/twcompat.py (providedBy): remove this method * buildbot/*: turn all uses of providedBy(obj, iface) into iface.providedBy(obj) * buildbot/twcompat.py (maybeWait): remove this method, now that we no longer maintain compatibility with Twisted<=1.3.0 . Twisted-2.0.0 and later allow trial methods to return Deferreds directly. * buildbot/test/*: remove all uses of maybeWait * buildbot/__init__.py (version): bump to 0.7.5+ while between releases * docs/buildbot.texinfo: same 2006-12-10 Brian Warner * buildbot/__init__.py (version): Releasing buildbot-0.7.5 * docs/buildbot.texinfo: set version to match 2006-12-10 Brian Warner * README (REQUIREMENTS): update for release * NEWS: update for release * buildbot/slave/commands.py (command_version): mention that this version (2.2) was released with buildbot-0.7.5 * buildbot/test/test_config.py (StartService.testStartService): inhibit the usual read-config-on-startup behavior, since otherwise the log.err that gets recorded causes the test to fail * buildbot/status/builder.py (LogFile.finish): forget about all subscribers once the log has finished, since after that point we're never going to use them again. This might help free up some memory somewhere. * buildbot/clients/debug.py: update to use gtk.main_quit() rather than the old/deprecated gtk.mainquit() 2006-12-09 Brian Warner * buildbot/steps/transfer.py (_FileWriter.__del__): handle errors better when we can't open the masterdst file * buildbot/scripts/startup.py (Follower._failure): add missing import statement for BuildSlaveDetectedError * buildbot/steps/transfer.py (FileUpload): cleanup (FileDownload): same. Add tests for slave version, add mode=. * buildbot/slave/commands.py (SlaveFileUploadCommand): same (SlaveFileDownloadCommand): same * buildbot/test/test_transfer.py: enhance tests * buildbot/test/runutils.py (makeBuildStep): create a fake form of step.slaveVersion 2006-12-08 Brian Warner * buildbot/scripts/runner.py (sendchange): halt the reactor on both success *and* failure. Without this, the 'buildbot sendchange' command would hang if it could not contact the buildmaster or deliver the Change, which would generally cause the user's commit/record/checkin command to hang too. Thanks to Christian Unger for the catch. 2006-12-06 Brian Warner * NEWS: update with items for the next release * docs/buildbot.texinfo (Adding LogObservers): add a somewhat whimsical example pulled from a punch-drunk email I sent out late one night. (Transferring Files): document some of the other parameters (Adding LogObservers): update to 0.7.5 buildbot.steps.* (SVNPoller): rename svnpoller.SvnSource to SVNPoller * buildbot/test/test_svnpoller.py: same * buildbot/changes/svnpoller.py (SVNPoller): same 2006-11-26 Brian Warner * docs/buildbot.texinfo (Build Properties): remind users that WithProperties must appear in a command= list, not as a top-level instance. * buildbot/steps/shell.py (ShellCommand.start): and assert that we're sending a list or a single string to the RemoteShellCommand * buildbot/scheduler.py (Nightly): Improve docs slightly. * buildbot/scripts/startup.py (start): skip the whole watch-the-logfile thing under windows, since it needs os.fork() * buildbot/scripts/runner.py (restart): remove the old message that got printed after the buildbot was restarted.. it most cases it didn't get printed at the right time anyways 2006-11-25 Brian Warner * buildbot/scripts/runner.py: enhance 'start' and 'restart' to follow twistd.log and print lines until the process has started started properly. For the buildmaster, this means until the config file has been parsed and accepted. For the buildslave, this means until we've connected to the master. We give up after 5 seconds in any case. Helpful error messages and troubleshooting suggestions are printed when we don't see a successful startup. This closes the remainder of SF#1517975. * buildbot/scripts/startup.py: moved app startup code to here * buildbot/scripts/logwatcher.py: utility class to follow log * buildbot/scripts/reconfig.py: rewrite to use LogWatcher * buildbot/slave/bot.py: announce our BuildSlaveness to the log so the LogWatcher can tell the difference between a buildmaster and a buildslave 2006-11-24 Brian Warner * docs/examples/twisted_master.cfg: update to match the version in use on twistedmatrix.com (IRC): re-enable IRC bot. The 'irc.us.freenode.net' hostname I was using before stopped working, but 'irc.freenode.net' works just fine. * buildbot/scripts/runner.py (run): oops, forgot to enable the new 'reconfig' command * buildbot/clients/base.py (TextClient.not_connected): upon UnauthorizedLogin failures, remind the user to connect to the PBListener port instead of the slaveport. (TextClient.disconnected): shut down more quietly * docs/buildbot.texinfo (statuslog): add another reminder * buildbot/scripts/runner.py (Options.subCommands): rename 'buildbot sighup DIR' to 'buildbot reconfig DIR', but keep 'sighup' as an alias. * buildbot/scripts/reconfig.py (Reconfigurator): enhance the reconfig command to follow twistd.log and print all of the lines from the start of the config-file reload to its completion. This should make it a lot easier to discover bugs in the config file. Use --quiet to disable this behavior. This addresses half of SF#1517975, the other half will be to add this same utility to 'buildbot start' and 'buildbot restart'. * docs/buildbot.texinfo (Loading the Config File): same (Shutdown): same * buildbot/interfaces.py (IBuilderControl.forceBuild): remove this method, it has been deprecated for a long time. Use IBuilderControl.requestBuild instead. * buildbot/process/builder.py (BuilderControl.forceBuild): remove * buildbot/master.py (BotPerspective.perspective_forceBuild): same * buildbot/slave/bot.py (Bot.debug_forceBuild): same * buildbot/test/test_control.py (Force.testForce): same * buildbot/test/test_run.py: use requestBuild instead * buildbot/clients/debug.py: replace 'Force Build' button with 'Request Build' (which just adds one to the queue), add Ping Builder, add branch/revision fields to Request Build. * buildbot/clients/debug.glade: same * buildbot/master.py: update interface to match. This creates an incompatibility between new debugclients and old buildmasters. * buildbot/process/builder.py (Builder._attached): delay the call to maybeStartBuild for a reactor turn, to avoid starting a build in the middle of a reconfig (say, if the new Builder uses a new slave which is already connected). 2006-11-23 Brian Warner * buildbot/test/test_transfer.py: appease pyflakes * buildbot/test/test_steps.py: same * buildbot/test/test_bonsaipoller.py: remove the 'import *' that keeps pyflakes from finding undefined names * buildbot/master.py (BuildMaster.loadConfig_Builders): changing a Builder no longer induces a disconnect/reconnect cycle. This means that any builds currently in progress will not be interrupted, and any builds which are queued in the Builder will not be lost. This is implemented by having the new Builder extract the state (i.e. all pending Builds and any desired SlaveBuilders) from the old Builder. (BotPerspective): refactor. The BotPerspective no longer keeps track of all the Builders that want to use this slave; instead, it asks the BotMaster each time it needs this list. This removes addBuilder and removeBuilder. Clean up attached() to acquire all the slave's information in a more atomic fashion. updateSlave() is now the way to make sure the slave is using the right set of Builders: just call it after everything else has been reconfigured. (BotMaster): refactor, removing addBuilder/removeBuilder and replacing them with an all-at-once setBuilders() call. * buildbot/test/test_slaves.py (Reconfig): new test case to exercise this functionality * buildbot/steps/dummy.py (Wait): new dummy BuildStep for the test * buildbot/slave/commands.py (WaitCommand): same * docs/buildbot.texinfo (Loading the Config File): document the changes * buildbot/process/builder.py (SlaveBuilder): refactor. Allow the SlaveBuilder to have its parent Builder changed. (SlaveBuilder.isAvailable): new method to give access to state, which is now a private attribute (SlaveBuilder.buildStarted,buildFinished): new methods to inform the SlaveBuilder about how it is being used. These methods update its internal state. buildFinished() is now the place that invokes maybeStartBuild() on its parent Builder. (Builder.consumeTheSoulOfYourPredecessor): new method to allow a new Builder to take over for an old one, transferring state from the old one. (Buider): refactor the way that SlaveBuilders are used to match, giving them a bit more autonomy. (Builder.buildFinished): this no longer calls maybeStartBuild(): instead the SlaveBuilder calls it on whoever its parent Builder is at the time. This way, when an old Builder is replaced by a new one, and there was a build in progress during the transition, when that build finishes, it will be the new Builder that is told about the newly available slave so it can start a new build. * buildbot/process/base.py (Build.startBuild._release_slave): when the Build finishes, tell the SlaveBuilder that they've been released. * buildbot/status/builder.py (SlaveStatus): add some new setter methods for use by BotPerspective, to keep some attributes more private * buildbot/slave/bot.py (Bot.remote_getDirs): this is no longer called by the buildmaster. (Bot.setBuilderList): instead, we locally announce any leftover directories based upon which Builders we were told about. The master doesn't really care; it's the local admin who may or may not wish to delete them. * contrib/svn_buildbot.py: use /usr/bin/python, not /usr/bin/env, to allow use of python2.4 or whatever. This tool still requires python2.3 or newer. 2006-11-19 Brian Warner * NEWS (IStatusLog.readlines): more news items 2006-11-18 Brian Warner * NEWS: start collecting items for the next release. 2006-11-04 Brian Warner * buildbot/changes/bonsaipoller.py: apply updates from Ben Hearsum. Closes SF#1590310. * buildbot/test/test_bonsaipoller.py: and tests * buildbot/status/tinderbox.py (TinderboxMailNotifier.buildMessage): send out a "testfailed" status when the build results in WARNINGS. Patch from Dave Liebreich. Closes SF#1587352. * buildbot/slave/commands.py (LogFileWatcher.poll): overcome a linux-vs-osx behavior difference w.r.t. reading from files that have reached EOF. This should fix LogFileWatcher on OS-X. Thanks to Mark Rowe for the patch. 2006-10-15 Brian Warner * buildbot/interfaces.py (IStatus.getURLForThing): oops, the method name was misspelled in the interface definition. Thanks to Roy Rapoport for the catch. 2006-10-13 Brian Warner * docs/buildbot.texinfo (Adding LogObservers): update sample code to match the great Steps renaming * buildbot/steps/transfer.py (FileUpload.start): Fix stupid error. Maybe I should run my own unit tests before recording a big change. Good thing I've got a buildbot to remind me. 2006-10-12 Brian Warner * buildbot/steps/transfer.py: rework __init__ and args setup * buildbot/slave/commands.py (SlaveFileDownloadCommand): minor docs improvements * buildbot/slave/commands.py (SlaveFileDownloadCommand.setup): when opening the target file, only catch IOError (to report via stderr/rc!=0), let the others be reported as normal exceptions 2006-10-08 Brian Warner * contrib/svn_watcher.py: fix security holes by using proper argv arrays and subprocess.Popen() rather than commands.getoutput(). Thanks to Nick Mathewson for the patch. Note that svn_watcher.py is deprecated in favor of buildbot/changes/svnpoller.py, and will probably be removed by the next release. * CREDITS: add Nick 2006-10-04 Brian Warner * buildbot/steps/python.py (PyFlakes.createSummary): skip any initial lines that weren't emitted by pyflakes. When the pyflakes command is run under a Makefile, 'make' will echo the command it runs to stdio, and that was getting logged as a "misc" warning. * buildbot/test/test_steps.py (Python.testPyFlakes2): test it * buildbot/test/test_steps.py (testPyFlakes3): another test 2006-10-01 Brian Warner * buildbot/status/html.py (HtmlResource.render): if we get a unicode object from our content() method, encode it into utf-8 like we've been claiming to all along. This allows the comments and author names from svnpoller.py to be delivered properly. * buildbot/changes/svnpoller.py (SvnSource.create_changes): de-unicodify filenames before creating the Change, because the rest of buildbot is unlikely to handle them well. Leave the 'who' field as a unicode object.. I don't think there's anything that will break very soon, and it will probably nudge us towards accepting unicode everywhere sooner or later. Stop using the "date" field that comes out of SVN, since it is using the repository's clock (and timezone) and what we care about is the buildmaster's (otherwise Changes from the future show up later than the builds they triggered). * buildbot/test/test_svnpoller.py (Everything.test1): match the change to .when * buildbot/changes/svnpoller.py (SvnSource): added Niklaus Giger's Suvbersion repository polling ChangeSource. I've hacked it up considerably: any bugs are entirely my own fault. Thank you Niklaus! * buildbot/test/test_svnpoller.py: tests for it * docs/buildbot.texinfo (SvnSource): document it 2006-09-30 Brian Warner * buildbot/scheduler.py (Periodic): submit a reason= to the BuildSet to indicate which Scheduler triggered the build. Thanks to Mateusz Loskot for the suggestion. (Nightly): same * buildbot/test/test_scheduler.py (Scheduling.testPeriodic1): test it * buildbot/changes/p4poller.py (P4Source): some minor stylistic changes: set self.loop in __init__, remove unused volatile= * docs/buildbot.texinfo (.buildbot config directory): add more docs on the .buildbot/options keys used by "buildbot try" * buildbot/scripts/tryclient.py (Try.createJob): remove dead code (Try.deliverJob): same * buildbot/changes/bonsaipoller.py (BonsaiParser): more updates from Robert Helmer (BonsaiPoller): same * buildbot/slave/commands.py (LogFileWatcher.stop): explicitly close the filehandle when we stop watching the file. Before, the filehandle was only closed when the LogFileWatcher was garbage-collected, which could be quite a while in the future. If it was still open by the time the next build started, windows will refuse to let the new build delete the old build/ directory. Fixes SF#1568415, thanks to , , and on #twisted for the catch. 2006-09-29 Brian Warner * buildbot/status/tinderbox.py (TinderboxMailNotifier): updates from Robert Helmer 2006-09-25 Brian Warner * setup.py: the new buildbot.steps module wasn't being installed. Thanks to Jose Dapena Paz for the catch, fixes SF#1560631. (testmsgs): add the extra stuff from buildbot/test/* so you can run unit tests on an installed copy of buildbot, not just from the source tree. * contrib/svn_buildbot.py (ChangeSender.getChanges): the first *4* columns of 'svnlook changed' output contain status information, so strip [:4] instead of [:6]. Depending upon what the status flags were, this would sometimes lead to mangled filenames. Thanks to Riccardo Magliocchetti for the patch. Closes SF#1545146. * buildbot/steps/source.py (Monotone): initial Monotone support, contributed by Nathaniel Smith. Still needs docs and tests, but this code has been in use on the Monotone buildbot for a long time now. * buildbot/slave/commands.py (Monotone): slave-side support * buildbot/changes/monotone.py (MonotoneSource): polling change source * buildbot/changes/bonsaipoller.py (BonsaiPoller): Ben also contributed a Change Source that polls a Bonsai server (a kind of web-based viewcvs CGI script). * buildbot/status/tinderbox.py (TinderboxMailNotifier): Ben Hearsum contributed a status plugin which sends email in the same format that Tinderbox does: this allows a number of tinderbox tools to be driven by Buildbot instead. Thanks Ben! 2006-09-24 Brian Warner * buildbot/changes/mail.py (parseBonsaiMail): fix the parser. Thanks to Robert Helmer for the patch. * buildbot/process/base.py (Build.setupSlaveBuilder): tell our BuildStatus about the buildslave name at the *beginning* of the build, rather than at the end. Thanks to Alexander Lorenz for the patch. * buildbot/status/html.py (StatusResourceBuild.body): always include the slavename in the build page, not just when the build has finished. * buildbot/status/mail.py (MailNotifier.buildMessage): include the slavename in the email message 2006-09-21 Brian Warner * buildbot/scripts/sample.cfg: update to use new BuildStep classes from buildbot.steps * docs/examples/glib_master.cfg: same * docs/examples/hello.cfg: same * docs/examples/twisted_master.cfg: same, update to current usage 2006-09-19 Brian Warner * buildbot/steps/python.py (PyFlakes): refactor, add summary logs (PyFlakes.createSummary): make it compatible with python-2.2 * buildbot/test/test_steps.py (Python.testPyFlakes): add a test for at least the output-parsing parts of PyFlakes 2006-09-18 Brian Warner * buildbot/steps/python.py: oops, fix import of StringIO 2006-09-17 Brian Warner * buildbot/test/test_vc.py (VCBase._do_vctest_update_retry_1): it turns out that SVN-1.4.0 doesn't fail to update once you've replaced a file with a directory, unlike older versions of SVN and pretty much every other VC tool we support. Since what we really care about is that the update succeeds anyway, stop checking that the tree got clobbered and just assert that the build succeeded. (VCBase.printLogs): add a utility function for debugging * buildbot/process/step.py: oops, added extra imports by mistake * buildbot/changes/p4poller.py (P4Source._process_describe): do an rstrip() on the first line coming out of the 'p4 describe' process, to remove the stray ^M that Wade Brainerd reports seeing in the 'when' field. Fixes SF#1555985. * buildbot/master.py (BuildMaster.loadConfig): improve the error message logged when c['schedulers'] is not right * buildbot/scheduler.py (Scheduler.__init__): improve error message when a Scheduler() is created with the wrong arguments * buildbot/test/test_config.py (ConfigTest.testSchedulerErrors): verify that these error messages are emitted * buildbot/process/buildstep.py: rename step.py to buildstep.py . The idea is that all the base classes (like BuildStep and RemoteCommand and LogObserver) live in b.p.buildstep, and b.p.step will be a leftover backwards-compatibility file that only contains aliases for the steps that were moved out to buildbot.steps.* * lots: change imports to match * buildbot/process/step.py: add a DeprecationWarning if it ever gets imported 2006-09-12 Brian Warner * buildbot/scheduler.py (Scheduler.__init__): make sure that builderNames= is actually a sequence, since if you happen to give it a single builder-specification dictionary instead, it won't get caught by the existing assert. Thanks to Brett Neely for the catch. * buildbot/steps/python.py (BuildEPYDoc, PyFlakes): add new steps. No tests yet, alas. * docs/buildbot.texinfo (Python BuildSteps): document them (sendchange): include a link to PBChangeSource, since you need one * buildbot/steps/shell.py: clean up test-case-name line, remove some unnecessary imports * buildbot/steps/dummy.py: same 2006-09-08 Brian Warner * buildbot/steps/transfer.py (FileUpload,FileDownload): new BuildStep which lets you transfer files from the master to the slave or vice versa. Thanks to Albert Hofkamp for the original patch. Fixes SF#1504631. * buildbot/slave/commands.py (SlaveFileUploadCommand): slave-side support for it (SlaveFileDownloadCommand): same * docs/buildbot.texinfo (Transferring Files): document it * buildbot/test/test_transfer.py: test it * buildbot/test/runutils.py (StepTester): new utility class for testing BuildSteps and RemoteCommands without Builds or Bots or PB * buildbot/test/test_steps.py (CheckStepTester): validate that the utility class works * buildbot/interfaces.py (IStatusLog.readlines): make it easier to walk through StatusLogs one line at a time, mostly for the benefit of ShellCommand.createSummary methods. You can either walk through STDOUT or STDERR, but the default is STDOUT. * buildbot/status/builder.py (LogFile.readlines): implement it. Note that this is not yet memory-efficient, it just pulls the whole file into RAM and then splits it up with a StringIO. Eventually this should be a generator that only pulls chunks from disk as necessary. * buildbot/test/test_status.py (Log.testReadlines): test it * docs/buildbot.texinfo: update to match changes * buildbot/process/factory.py: stop using old definitions * buildbot/process/process_twisted.py: same * buildbot/test/test_*.py: same * buildbot/process/step_twisted.py: move definition to.. * buildbot/steps/python_twisted.py: .. here, unfortunately python's relative-import mechanisms prevent this from being named 'twisted' or 'python/twisted' as I would have preferred. * buildbot/process/maxq.py: move definition to.. * buildbot/steps/maxq.py: .. here, leave a compatibility import * buildbot/process/step.py: split the user-visible BuildSteps into separate files, all under buildbot/steps/ * buildbot/steps/source.py: this holds VC-checkout steps like SVN * buildbot/steps/shell.py: this holds ShellCommand and friends * buildbot/steps/dummy.py: this holds the testing steps like Dummy 2006-09-05 Brian Warner * lots: run pyflakes, removed a lot of unused imports, changed the form of some conditional imports to remove false pyflakes warnings. There are still a number of warnings left, mostly from imports that are done for their side-effects. * buildbot/test/test_vc.py: import twisted.python.failure, since it was missing 2006-08-26 Brian Warner * buildbot/test/test_locks.py (Unit.testLater): make the tests compatible with twisted-1.3.0, for some reason I just can't seem to let go of the past. 2006-08-25 Brian Warner * buildbot/status/mail.py (MailNotifier.__init__): fix typo in docs * buildbot/process/step.py (LoggingBuildStep.startCommand): set up all logfiles= in startCommand(), rather than in start() . This makes it easier to have the 'stdio' log come before any secondary logfiles, which I feel makes the waterfall display more understandable. (LoggingBuildStep.setupLogfiles): move the addLog/cmd.useLog code from ShellCommand up into LoggingBuildStep (LoggingBuildStep.__init__): move the handling of logfiles= from ShellCommand up to LoggingBuildStep, because startCommand is provided by LoggingBuildStep, whereas start() was specific to subclasses like ShellCommand and Source. This removes code duplication in those subclasses. (ShellCommand.__init__): same (ShellCommand.checkForOldSlaveAndLogfiles): split out the check for a slave that's too old to understand logfiles= into a separate method, so it can live in ShellCommand. The rest of setupLogfiles() can live in LoggingBuildStep. 2006-08-24 Brian Warner * buildbot/locks.py (BaseLock): you can now configure Locks to allow multiple simultaneous owners. They still default to maxCount=1. Fixes SF#1434997. Thanks to James Knight (foom) for the patch. * docs/buildbot.texinfo (Interlocks): document the new options * buildbot/test/test_locks.py: add a bunch of new unit tests * buildbot/process/base.py (Build.acquireLocks): locks now offer waitUntilMaybeAvailable, not waitUntilAvailable * buildbot/process/step.py (BuildStep.acquireLocks): same * buildbot/master.py (BotMaster.getLockByID): real locks now use the whole lockid in their constructor, not just the name. Also, keep track of which real locks we've handed out by the full lockid, not just class+name, otherwise changing just the maxCount= in the master.cfg file would not actually cause a behavioral change 2006-08-23 Brian Warner * buildbot/__init__.py (version): bump to 0.7.4+ while between releases * docs/buildbot.texinfo: same 2006-08-23 Brian Warner * buildbot/__init__.py (version): Releasing buildbot-0.7.4 * docs/buildbot.texinfo: set version to match * NEWS: update for 0.7.4 * buildbot/slave/commands.py (command_version): mention that this version (2.1) was released with buildbot-0.7.4 2006-08-22 Brian Warner * README: update * CREDITS: new file, list of people who have helped. Thanks! * MANIFEST.in: ship it * MANIFEST.in: stop shipping the old PyCon-2003 paper.. with the new diagrams, the user's manual is more informative than it was. Start shipping the .html user's manual (and generated .png images). * Makefile: update 'release' target to match * buildbot/test/test_web.py (GetURL.testBrokenStuff): delete this test.. I think the web-parts effort will render it pointless well before it ever actually starts to work. 2006-08-20 Brian Warner * buildbot/changes/pb.py (PBChangeSource): fix and simplify meaning of the prefix= argument. It is now just a string which is stripped from the beginning of the filename. If prefix= is set but not found on any given filename, that filename is ignored. If all filenames in a Change are ignored, the Change is dropped. This is much simpler than the previous sep= nonsense, and I should have implemented it this way from the beginning. Effectively resolves SF#1217699 and SF#1381867. Thanks to Gary Granger and Marius Gedminas for the catch and suggested fixes. (ChangePerspective.perspective_addChange): implement the actual prefix comparison * buildbot/test/test_changes.py (TestChangePerspective): test it * docs/buildbot.texinfo (PBChangeSource): document it, explain how to properly use prefix= * docs/examples/twisted_master.cfg (source): update prefix= by adding the trailing slash * docs/examples/twisted_master.cfg: update to actual practice * buildbot/test/test_web.py (WaterfallSteps.test_urls): oops, update test case to match new link text.. the HREF has both a class= setting and an enclosing [] pair that I didn't match in the test. * docs/buildbot.texinfo (ShellCommand.command=): explain why a list of strings is preferred over a single string with embedded spaces (ShellCommand.description=): explain that either single strings or a list of strings is acceptable, and why you might prefer one over the other. Add an example. Fixes SF#1524659, thanks to Paul Winkler for the catch. (Build Steps): update to use f.addStep() rather than using s() and the constructor list * buildbot/process/step.py (ShellCommand): accept either a single string or a list of strings in both description= and descriptionDone= * buildbot/test/test_steps.py (Steps.test_description): test it * buildbot/test/runutils.py (makeBuildStep): support for that test * contrib/CSS/*.css: add some contributed CSS stylesheets, to make the Waterfall display a bit less ugly. Thanks to John O'Duinn for collecting the files and creating the patch. * docs/buildbot.texinfo (BuildStep URLs): document new feature: per-step URLs that will be displayed on the waterfall display, for things like the HTML output of code-coverage tools, when the results are hosted elsewhere. * buildbot/interfaces.py (IBuildStepStatus.getURLs): document the way to retrieve these URLs * buildbot/status/builder.py (BuildStepStatus.getURLs): implement the method to retrieve these URLs. Also provide backwards compatibility for saved BuildStepStatus instances that didn't have the .urls attribute * buildbot/process/step.py (BuildStep.addURL): method to set these URLs from within a BuildStep * buildbot/status/html.py (StepBox.getBox): emit links to the URLs (StepBox.getBox): give these external links a distinct CSS class named "BuildStep external" so a .css file can display them differently * buildbot/test/test_web.py (WaterfallSteps): test that we really do emit those links * buildbot/test/test_steps.py (Steps): test that we can all the URLs. Also add a bunch of other tests on methods that can be called from within BuildSteps * buildbot/test/runutils.py (makeBuildStep): add utility function 2006-08-13 Brian Warner * docs/buildbot.texinfo (BuildStep LogFiles): document them 2006-08-10 Brian Warner * docs/buildbot.texinfo (Index of master.cfg keys): add another index, this one of things like c['sources'] and c['schedulers'] (indices): it looks like my clever idea of putting the @fooindex commands in between the @node and the @subsection (to make the HREF anchor jump to slightly above the section title, which works much better in html) confused texinfo horribly, so I'm moving the index tags back to be just after the @subsection marker. I also added extra lines between the @node/@section paragraph and the index tags, since I think maybe texinfo wants to see these be separate paragraphs. * docs/Makefile (images): make sure images get built when rendering the manual * docs/images/Makefile: same * buildbot/scripts/runner.py: rename 'buildbot master' to 'buildbot create-master', and 'buildbot slave' to 'buildbot create-slave' * docs/buildbot.texinfo: same * README: same * docs/buildbot.texinfo: reimplement the "useful classes" index with actual texinfo indices. The .info rendering is a bit weird-looking but it works well, and the HTML rendering is quite nice. This also puts the index targets in the regular flow of the text, which is easier to maintain. 2006-08-06 Brian Warner * buildbot/slave/commands.py (ShellCommand.__init__): patch from Kevin Turner to prefer the environ= argument be a list rather than a string. If it is a list, it will be joined with a platform-local os.pathsep delimiter, and then prepended to any existing $PYTHONPATH value. This works better in cross-platform (i.e. windows buildslaves) environments when you need to push multiple directories onto the front of the path. (SlaveShellCommand): documented the new magic * docs/buildbot.texinfo (ShellCommand): documented the new magic in a user-visible form * buildbot/test/test_vc.py (BaseHelper.dovc): patch from Kevin Turner to prefer lists over strings when creating/running VC commands during unit tests. This is clearly necessary to survive vcexe containing spaces, like "C:\Program Files\darcs.exe". I renamed the wq() function to qw() though, since that's how it's spelled in perl. Eventually I'd prefer all commands to be specified with lists. * buildbot/slave/commands.py (LogFileWatcher): handle logfiles which are deleted (or not yet created) correctly. Also add failsafe code to not explode if the file-watching poller doesn't get started. Thanks to JP Calderone for the catch and the poller patch. * buildbot/test/test_shell.py (SlaveSide._testLogFiles): add test for that case * buildbot/test/emitlogs.py: same * NEWS: summarize recent changes * docs/buildbot.texinfo (Debug options): suggest an .ssh/options clause to avoid the "host key mismatch" warning * buildbot/process/step_twisted.py (Trial.start): if the buildslave is too old to understand logfiles=, fall back to running 'cat _trial_temp/test.log' like before. (Trial.commandComplete): same. this takes advantage of the LoggingBuildStep refactoring to stall commandComplete long enough to run a second RemoteShellCommand. * buildbot/process/step.py (LoggingBuildStep.startCommand): refactor command-completion handling, to allow methods like commandComplete/createSummary/evaluateCommand to return Deferreds. (LoggingBuildStep._commandComplete): delete the refactored method (ShellCommand.setupLogfiles): if the buildslave is too old to understand logfiles=, put a warning message both into twistd.log and into the otherwise empty user-visible LogFiles. * buildbot/process/step.py (LoggedRemoteCommand.useLog): allow callers to provide the slave-side logfile name, rather than forcing it to come from the local name of the LogFile. (BuildStep.getSlaveName): new method * buildbot/process/base.py (Build.getSlaveName): new method, so steps can find out which buildslave they're running on * buildbot/test/test_steps.py (Version.checkCompare): oops, update to match the s/cvs_ver/command_version/ change 2006-08-05 Brian Warner * buildbot/slave/commands.py (command_version): replace the CVS auto-updated cvs_ver keyword with a manually-updated variable, since CVS is no longer the master repository. Add a description of the remote API change starting in this version (2.1), specifically the fact that SlaveShellCommand now accepts 'initial_stdin', 'keep_stdin_open', and 'logfiles'. 2006-07-31 Brian Warner * docs/buildbot.texinfo (System Architecture): Finally add lots of diagrams to describe how the whole system fits together. The images themselves are kept in SVG files, with ascii-art versions in corresponding .txt files. Texinfo knows how to interpolate the text version into .info files, reference the .png versions from .html files, and include .eps versions in the .ps format. * docs/images/Makefile: tools to create .png and .eps * docs/images/*.svg: created pictures with Inkscape. * .darcs-boring: ignore the generated .eps and .png files 2006-07-24 Brian Warner * buildbot/master.py (BuildMaster.loadConfig): check for duplicate Scheduler names, since they cause setServiceParent to explode later. * buildbot/test/test_config.py (ConfigTest._testSchedulers_7): test it 2006-07-20 Brian Warner * buildbot/scripts/sample.cfg: simplify the sample BuildFactory, which runs the buildbot unit tests * docs/buildbot.texinfo (Index of Useful Classes): add a table of classes that are useful in master.cfg 2006-07-15 Brian Warner * Makefile (some-apidocs): new target to build only some epydocs * setup.py: minor comment.. does the classifiers= argument prevent the setup.py script from working on python2.2/2.3? * buildbot/scripts/sample.cfg: update manhole example, arrange into major sections * buildbot/twcompat.py: fix minor typo in comments * buildbot/manhole.py: move all Manhole-related code out to this module. Implement SSH-based manholes (with TwistedConch), and move to conch's nifty line-editing syntax-coloring REPL shell instead of the boring non-editing monochromatic (and deprecated) old 'telnet' protocol. * buildbot/master.py: remove all Manhole-related code (BuildMaster.loadConfig._add): make sure the old manhole is removed before we add the new one * docs/buildbot.texinfo (Debug options): document new Manhole options * buildbot/twcompat.py (_which): fix some epydoc issues * buildbot/status/html.py (Waterfall.__init__): same 2006-06-29 Brian Warner * buildbot/interfaces.py: get Interface from b.twcompat to hush deprecation warnings under newer Twisteds (by using zope.interface.Interface instead of old twisted.python.components stuff) * buildbot/slave/interfaces.py: same 2006-06-28 Brian Warner * buildbot/slave/commands.py (SVN): add --non-interactive to all svn commands, so it will fail immediately instead of hanging while it waits for a username/password to be typed in. * buildbot/slave/bot.py (SlaveBuilder.commandComplete): add minor log message if the step was shut down * buildbot/scripts/runner.py (SlaveOptions.longdesc): remove obsolete reference to mktap. 2006-06-20 Brian Warner * buildbot/test/test_steps.py (BuildStep.testShellCommand1): update test to include new 'logfiles' argument sent from master to slave 2006-06-19 Brian Warner * buildbot/process/step_twisted.py (Trial): track Progress from _trial_temp/test.log too * buildbot/process/step.py (OutputProgressObserver): generalize the earlier StdioProgressObserver into an OutputProgressObserver that can track LogFiles other than stdio. (LoggingBuildStep.__init__): same * buildbot/process/step_twisted.py (Trial): use logfiles= to track _trial_temp/test.log, not a separate 'cat' command. TODO: this will fail under windows because of os.sep issues. It might have worked before if 'cat' was doing cygwin path conversion. * buildbot/slave/commands.py (LogFileWatcher.__init__): note the creation of LogFileWatchers (ShellCommand._startCommand): and record the files that were watched in the 'headers' section of the ShellCommand output * buildbot/process/step.py (RemoteShellCommand.__init__): duh, you need to actually pass it to the slave if you want it to work. (ShellCommand): document it a bit * buildbot/test/test_shell.py: new test to validate LogFiles * buildbot/test/runutils.py (SlaveCommandTestBase): updates to test LogFiles * buildbot/test/emitlogs.py: enhance to wait for a line on stdin before printing the last batch of lines, to test that the polling logic is working properly * buildbot/process/step.py (LoggedRemoteCommand): improve LogFile handling, making it possible to track multiple logs for a single RemoteCommand. The previous single logfile is now known as the 'stdio' log. (LoggedRemoteCommand.remoteUpdate): accept key='log' updates (RemoteShellCommand.__init__): accept logfiles= (LoggingBuildStep.startCommand): stdio_log is now one of many (ShellCommand): added logfiles= argument, as well as a class-level .logfiles attribute, which will be merged together to figure out which logfiles should be tracked. The latter maybe be useful for subclasses of ShellCommand which know they will aways produce secondary logfiles in the same location. * buildbot/slave/commands.py (ShellCommandPP): add writeStdin() and closeStdin() methods, preparing to make it possible to write to a ShellCommand's stdin at any time, not just at startup. These writes are buffered if the child process hasn't started yet. (LogFileWatcher): new helper class to watch arbitrary logfiles while a ShellCommand runs. This class polls the file every two seconds, and sends back 10k chunks to the buildmaster. (ShellCommand): rename stdin= to initialStdin=, and add keepStdinOpen= and logfiles= to arguments. Set up LogFileWatchers at startup. (ShellCommand.addLogfile): LogFile text is sent in updates with a key of "log" and a value of (logname, data). (SlaveShellCommand): add 'initial_stdin', 'keep_stdin_open', and 'logfiles' to the master-visible args dictionary. (SourceBase.doPatch): match s/stdin/initialStdin/ change (CVS.start): same (P4.doVCFull): same * buildbot/test/test_vc.py (Patch.testPatch): same * buildbot/test/emit.py: write to a logfile in the current directory. We use this to figure out what was used as a basedir rather than looking to see which copy of emit.py gets run, so that we can run the commands from inside _trial_temp rather than inside buildbot/test * buildbot/test/subdir/emit.py: same * buildbot/test/runutils.py (FakeSlaveBuilder): take a 'basedir' argument rather than running from buildbot/test/ (SlaveCommandTestBase.setUpBuilder): explicitly set up the Builder rather than using an implicit setUp() * buildbot/test/test_slavecommand.py (ShellBase.setUp): same (ShellBase.testShell1, etc): use explicit path to emit.py instead of assuming that we're running in buildbot/test/ (and that '.' is on our $PATH) * buildbot/slave/commands.py (Command.doStart): refactor Command startup/completion a bit: now the SlaveBuilder calls doStart(), which is not meant for overridding by subclasses, and doStart() calls start(), which is. Likewise the SlaveBuilder calls doInterrupt(), and subclasses override interrupt(). This also puts responsibility for maintaining .running in Command rather than in SlaveBuilder. (Command.doInterrupt): same (Command.commandComplete): same, this is called when the deferred returned by start() completes. * buildbot/slave/bot.py (SlaveBuilder.remote_startCommand): same (SlaveBuilder.remote_interruptCommand): same (SlaveBuilder.stopCommand): same 2006-06-16 Brian Warner * buildbot/test/test_shell.py: new test file to contain everything relating to ShellCommand (SlaveSide.testLogFiles): (todo) test for the upcoming "watch multiple logfiles in realtime" feature, not yet implemented * buildbot/test/emitlogs.py: support file for testLogFiles * docs/buildbot.texinfo (ShellCommand): document the feature * buildbot/test/test_steps.py (BuildStep.setUp): rmtree refactoring * buildbot/test/runutils.py (SlaveCommandTestBase): utility class for tests which exercise SlaveCommands in isolation. * buildbot/test/test_slavecommand.py: Move some utilities like SignalMixin and FakeSlaveBuilder from here .. * buildbot/test/runutils.py: .. to here, so they can be used by other test classes too * buildbot/test/test_vc.py: more SignalMixin refactoring * buildbot/test/test_control.py: same * buildbot/test/test_run.py: and some rmtree refactoring 2006-06-15 Brian Warner * buildbot/test/test_vc.py (P4.testCheckoutBranch): rename from 'testBranch' to match other VC tests and have the tests run in roughly increasing order of dependency * buildbot/test/test_steps.py (LogObserver): new test to verify LogObservers can be created at various times and still get connected up properly * buildbot/test/runutils.py (setupBuildStepStatus): utility method to create BuildStepStatus instances that actually work. * buildbot/process/step.py (LogObserver): add outReceived and errReceived base methods, to be overridden * buildbot/status/builder.py (BuildStatus.addStepWithName): change API to take a name instead of a step, reducing the coupling somewhat. This returns the BuildStepStatus object so it can be passed to the new Step, instead of jamming it directly into the Step. * buildbot/process/step.py (BuildStep.setStepStatus): add a setter method * buildbot/process/base.py (Build.setupBuild): use both methods * buildbot/test/test_web.py (Logfile.setUp): rearrange the setup process a bit to match 2006-06-14 Brian Warner * docs/buildbot.texinfo (Adding LogObservers): add some limited docs on writing new LogObserver classes (Writing New Status Plugins): brief docs on how Status Plugins fit together * buildbot/process/step_twisted.py (TrialTestCaseCounter): implement a LogObserver that counts how many unit tests have been run so far (Trial.__init__): wire it in * buildbot/test/test_twisted.py (Counter): unit test for it * buildbot/process/step_twisted.py (HLint.commandComplete): update to new cmd.logs['stdio'] scheme (Trial.commandComplete): same (BuildDebs.commandComplete): same * buildbot/process/step.py (LoggedRemoteCommand): use a dict of LogFiles, instead of just a single one. The old single logfile is now called "stdio". LoggedRemoteCommand no longer creates a LogFile for you (the code to do that was broken anyway). If you don't create a "stdio" LogFile, then stdout/stderr will be discarded. (LogObserver): implement "LogObservers", which a BuildStep can add to parse the output of a command in real-time. The primary use is to provide more useful information to the Progress code, allowing better ETA estimates. (LogLineObserver): utility subclass which feeds complete lines to the parser instead of bytes. (BuildStep.progressMetrics): this is safer as a tuple (BuildStep.setProgress): utility method, meant to be called by LogObservers (BuildStep.addLogObserver): new method, to be called at any time during the BuildStep (even before any LogFiles have been created), to attach (or schedule for eventual attachment) a LogObserver to a LogFile. (StdioProgressObserver): new LogObserver which replaces the old "output" progress gatherer (LoggingBuildStep.__init__): same (LoggingBuildStep.startCommand): set up the "stdio" LogFile (LoggingBuildStep._commandComplete): must use logs['stdio'] instead of the old single ".log" attribute. * buildbot/status/builder.py (LogFile): remove old logProgressTo functionality, now subsumed into StdioProgressObserver * buildbot/test/test_status.py (Subscription._testSlave_2): the log name changed from "output" to "stdio". * buildbot/interfaces.py (ILogFile): add the Interface used from the BuildStep towards the LogFile (ILogObserver): and the one provided by a LogObserver * buildbot/status/builder.py (LogFile): implement it * buildbot/interfaces.py (LOG_CHANNEL_*): move STDOUT / STDERR / HEADER constants here .. * buildbot/status/builder.py (STDOUT): .. from here 2006-06-13 Brian Warner * buildbot/test/test_p4poller.py (TestP4Poller.failUnlessIn): fix compatibility with python2.2, which doesn't have the 'substr in str' feature. (TestP4Poller.makeTime): utility function to construct the timestamp using the same strptime() approach as p4poller does. It turns out that time.mktime() behaves slightly differently under python2.2, probably something to do with the DST flag, and that causes the test to fail under python2.2. (changing the mktime() arguments to have dst=0 instead of -1 caused it to fail under python2.3. Go figure.) (TestP4Poller._testCheck3): use our makeTime() instead of mktime() 2006-06-12 Brian Warner * buildbot/process/step.py (P4): merge in patch SF#1473939, adding proper Perforce (P4) support. Many many thanks to Scott Lamb for contributing such an excellent patch, including docs and unit tests! This makes it *so* much easier to apply. I had to update test_vc.py to handle some recent refactorings, but everything else applied smoothly. The only remaining thing I'd like to fix would be to remove the hard-wired port 1666 used by p4d, and allow it to claim any unused port. This would allow two copies of the test suite to run on the same host at the same time, as well as allowing the test suite to run while a real (production) p4d was running on the same host. Oh, and maybe we should add a warning to step.P4 that gets emitted if the slave is too old to provide the 'p4' SlaveCommand. Otherwise it looks great. (closes: SF#1473939). * buildbot/slave/commands.py (P4): same (P4Sync): same, some minor updates * buildbot/changes/p4poller.py: same * docs/buildbot.texinfo: same * buildbot/test/test_p4poller.py: same * buildbot/test/test_vc.py (P4): same * setup.py: add Trove classifiers for PyPI 2006-06-08 Brian Warner * buildbot/status/client.py (RemoteBuilder.remote_getCurrentBuilds): oops, I screwed up when changing this from getCurrentBuild() to getCurrentBuilds(). Each build needs to be IRemote'd separately, rather than IRemote'ing the whole list at once. I can't wait until newpb's serialization adapters make this unnecessary. 2006-06-06 Brian Warner * buildbot/process/step.py (WithProperties): make this inherit from ComparableMixin, so that reloading an unchanged config file doesn't cause us to spuriously reload any Builders which use them. * buildbot/test/test_config.py (ConfigTest.testWithProperties): add a test for it 2006-06-03 Brian Warner * contrib/windows/{setup.py, buildbot_service.py}: add support for running py2exe on windows, contributed by Mark Hammond. Addresses SF#1401121, but I think we still need to include buildbot/scripts/sample.cfg * setup.py: include buildbot_service.py as a script under windows * buildbot/status/html.py: when sys.frozen (i.e. we're running in a py2exe application), get the icon/css datafiles from a different place than usual. * buildbot/status/mail.py (MailNotifier.buildMessage): don't double-escape the build URL. Thanks to Olivier Bonnet for the patch. Fixes SF#1452801. 2006-06-02 Brian Warner * contrib/svn_buildbot.py (ChangeSender.getChanges): ignore the first six columns of 'svnlook' output, not just the first column, since property changes appear in the other five. Thanks to Olivier Bonnet for the patch. Fixes SF#1398174. 2006-06-01 Brian Warner * buildbot/test/test_web.py (Logfile.setUp): set the .reason on the fake build, so that title= has something to be set to * buildbot/status/html.py (BuildBox.getBox): set the 'title=' attribute of the "Build #NN" link in the yellow start-the-build box to the build's reason. This means that you get a little tooltip explaining why the build was done when you hover over the yellow box. Thanks to Zandr Milewski for the suggestion. * buildbot/clients/gtkPanes.py (Box.setColor): ignore color=None (Box.setETA): handle ETA=None (by stopping the timer) (Box.update): make the [soon] text less different than the usual text, so the rest of the text doesn't flop around so much. It would be awfully nice to figure out how to center this stuff. (ThreeRowBuilder.stepETAUpdate): more debugging printouts * buildbot/process/step.py (ShellCommand): set flunkOnFailure=True by default, so that any ShellCommand which fails marks the overall build as a failure. I should have done this from the beginning. Add flunkOnFailure=False to the arguments if you want to turn off this behavior. 2006-05-30 Brian Warner * buildbot/clients/gtkPanes.py: add a third row: now it shows last-build/current-build/current-step. Show what step is currently running. Show ETA for both the overall build and the current step. Update GTK calls to modern non-deprecated forms. There's still a lot of dead code and debug noise to remove. * buildbot/process/step_twisted.py (Trial): set the step name, so it shows up properly in status displays 2006-05-28 Brian Warner * buildbot/test/test_properties.py (Run.testInterpolate): on the build we use to verify that WithProperties works: ** set flunkOnFailure=True so that build failures get noticed ** set workdir='.' so that the build succeeds, otherwise it is trying to touch 'build/something', and 'build/' doesn't exist because usually that's created by a Source step ** set timeout=10, because Twisted-1.3.0 has a race condition that this test somehow triggers, in which the 'touch' process becomes a zombie and we wait for th etimeout before giving up on it. * buildbot/test/runutils.py (RunMixin.logBuildResults): utility method to log the Build results and step logs to the twisted log. (RunMixin.failUnlessBuildSucceeded): use logBuildResults to record what went wrong if a build was expected to succeed but didn't. * buildbot/process/step_twisted.py (Trial): set the default trialMode to '--reporter=bwverbose', which specifies verbose black-and-white text. Back in twisted-1.3/2.0 days we had to use '-to', but those are completely missing in modern Twisteds. * buildbot/scripts/sample.cfg: make the sample Manhole config use a localhost-only port, to encourage better security * docs/buildbot.texinfo (Change Sources): mention darcs_buildbot.py * .darcs-boring: add a Darcs boringfile * README (REQUIREMENTS): stop claiming compatibility with Twisted-1.3.0 * contrib/darcs_buildbot.py: write a darcs-commit-hook change sender 2006-05-27 Brian Warner * buildbot/__init__.py: bump to 0.7.3+ while between releases * docs/buildbot.texinfo: same 2006-05-23 Brian Warner * buildbot/__init__.py (version): Releasing buildbot-0.7.3 * docs/buildbot.texinfo: set version to match * NEWS: update for 0.7.3 * docs/buildbot.texinfo (Change Sources): mention hg_buildbot.py, give a quick mapping from VC system to possible ChangeSources (Build Properties): add 'buildername' * buildbot/process/base.py (Build.setupStatus): oops, set 'buildername' and 'buildnumber' properties * buildbot/test/test_properties.py (Interpolate.testBuildNumber): test them 2006-05-22 Brian Warner * docs/buildbot.texinfo (Build Properties): explain the syntax of property interpolation better * README (INSTALLATION): remove old '-v' argument from recommended trial command line * docs/buildbot.texinfo (ShellCommand): add docs for description= and descriptionDone= arguments. Thanks to Niklaus Giger for the patch. SF#1475494. * buildbot/slave/commands.py (SVN.parseGotRevision._parse): use 'svnversion' instead of grepping the output of 'svn info', much simpler and avoids CR/LF problems on windows. Thanks to Olivier Bonnet for the suggestion. (SVN.parseGotRevision): oops, older verisons of 'svnversion' require the WC_PATH argument, so run 'svnversion .' instead. * buildbot/interfaces.py (IChangeSource): methods in Interfaces aren't supposed to have 'self' in their argument list 2006-05-21 Brian Warner * buildbot/process/step.py (ShellCommand.start): make testInterpolate pass. I was passing the uninterpolated command to the RemoteShellCommand constructor (ShellCommand._interpolateProperties): oops, handle non-list commands (i.e. strings with multiple words separated by spaces in them) properly, instead of forgetting about them. * buildbot/test/test_properties.py (Run.testInterpolate): new test to actually try to use build properties in a real build. This test fails. * buildbot/test/runutils.py (RunMixin.requestBuild): utility methods to start and evaluate builds * buildbot/test/test__versions.py: add a pseudo-test to record what version of Twisted/Python/Buildbot are running. This should show up at the beginning of _trial_tmp/test.log, and exists to help debug other problems. * buildbot/status/html.py (Waterfall): add 'robots_txt=' argument, a filename to be served as 'robots.txt' to discourage web spiders. Adapted from a patch by Tobi Vollebregt, thanks! * buildbot/test/test_web.py (Waterfall._test_waterfall_5): test it (Waterfall.test_waterfall): tweak the way that filenames are put into the config file, to accomodate windows pathnames better. * docs/buildbot.texinfo (HTML Waterfall): document it * buildbot/process/process_twisted.py (QuickTwistedBuildFactory.__init__): recent versions of Twisted changed the build process. The new setup.py no longer takes the 'all' argument. (FullTwistedBuildFactory.__init__): same (TwistedReactorsBuildFactory.__init__): same * contrib/hg_buildbot.py: wrote a commit script for mercurial, to be placed in the [hooks] section of the central repository (the one that everybody pushes changes to). 2006-05-20 Brian Warner * buildbot/slave/commands.py (Darcs.doVCFull): when writing the .darcs-context file, use binary mode. I think this was causing a Darcs failure under windows. 2006-05-19 Brian Warner * buildbot/scripts/tryclient.py (CVSExtractor.getBaseRevision): use a timezone string of +0000 and gmtime, since this timestamp is sent to a buildmaster and %z is broken. * buildbot/test/test_vc.py (CVSHelper.getdate): use no timezone string and localtime, since this timestamp will only be consumed locally, and %z is broken. * buildbot/slave/commands.py (CVS.parseGotRevision): use +0000 and gmtime, since this timestamp is returned to the buildmaster, and %z is broken. 2006-05-18 Brian Warner * NEWS: update in preparation for next release * buildbot/test/test_vc.py (VCS_Helper): factor out all the setup-repository and do-we-have-the-vc-tools code into a separate "helper" class, which sticks around in a single module-level object. This seems more likely to continue to work in the future than having it hide in the TestCase and hope that TestCases stick around for a long time. * buildbot/test/test_vc.py (MercurialSupport.vc_create): 'hg addremove' has been deprecated in recent versions of mercurial, so use 'hg add' instead 2006-05-07 Brian Warner * buildbot/scheduler.py (Try_Jobdir.messageReceived): when operating under windows, move the file before opening it, since you can't rename a file that somebody has open. * buildbot/process/base.py (Build.setupBuild): if something goes wrong while creating a Step, log the name and arguments, since the error message when you get the number of arguments wrong is really opaque. 2006-05-06 Brian Warner * buildbot/process/step_twisted.py (Trial.setupEnvironment): more bugs in twisted-specific code not covered by my unit tests, this time use 'cmd' argument instead of self.cmd * buildbot/process/process_twisted.py (TwistedBuild.isFileImportant): fix stupid braino: either use startwith or find()==0, not both. (TwistedReactorsBuildFactory.__init__): another dumb typo * buildbot/test/test_slavecommand.py (ShellBase.testInterrupt1): mark this test as TODO under windows, since process-killing seems dodgy there. We'll come back to this later and try to fix it properly. * buildbot/test/test_vc.py (CVSSupport.getdate): use localtime, and don't include a timezone (CVSSupport.vc_try_checkout): stop trying to strip the timezone. This should avoid the windows-with-verbose-timezone-name problem altogether. (Patch.testPatch): add a test which runs 'patch' with less overhead than the full VCBase.do_patch sequence, to try to isolate a windows test failure. This one uses slave.commands.ShellCommand and 'patch', but none of the VC code. * buildbot/slave/commands.py (getCommand): use which() to find the executables for 'cvs', 'svn', etc. This ought to help under windows. * buildbot/test/test_vc.py (VCBase.do_getpatch): Delete the working directory before starting. If an earlier test failed, the leftover directory would mistakenly flunk a later test. (ArchCommon.registerRepository): fix some tla-vs-baz problems. Make sure that we use the right commandlines if which("tla") picks up "tla.exe" (as it does under windows). (TlaSupport.do_get): factor out this tla-vs-baz difference (TlaSupport.vc_create): more tla-vs-baz differences * buildbot/test/test_slavecommand.py (ShellBase.testShellMissingCommand): stop trying to assert anything about the error message: different shells on different OSes with different languages makes it hard, and it really isn't that interesting of a thing to test anyway. * buildbot/test/test_vc.py (CVSSupport.capable): skip CVS tests if we detect cvs-1.10 (which is the version shipped with OS-X 10.3 "Panther"), because it has a bug which flunks a couple tests in weird ways. I've checked that cvs-1.12.9 (as shipped with debian) is ok. OS-X 10.4 "Tiger" ships with cvs-1.11, but I haven't been able to test that yet. 2006-04-30 Brian Warner * buildbot/test/test_vc.py (VCBase.runCommand): set $LC_ALL="C" to make sure child commands emit messages in english, so our regexps will match. Thanks to Nikaus Giger for identifying the problems. (VCBase._do_vctest_export_1): mode="export" is not responsible for setting the "got_revision" property, since in many cases it is not convenient to determine. (SVNSupport.capable): when running 'svn --version' to check for ra_local, we want error messages in english * buildbot/test/test_slavecommand.py (ShellBase.testShellMissingCommand): set $LC_ALL="C" to get bash to emit the error message in english * buildbot/slave/commands.py (SourceBase.setup): stash a copy of the environment with $LC_ALL="C" so that Commands which need to parse the output of their child processes can obtain it in english. (SVN.parseGotRevision): call "svn info" afterwards instead of watching the output of the "svn update" or "svn checkout". (Darcs.parseGotRevision): use $LC_ALL="C" when running the command (Arch.parseGotRevision): same (Bazaar.parseGotRevision): same (Mercurial.parseGotRevision): same * buildbot/scripts/tryclient.py (SourceStampExtractor.dovc): set $LC_ALL="C" when running commands under 'buildbot try', too * buildbot/test/__init__.py: remove the global os.environ() setting, instead we do it just for the tests that run commands and need to parse their output. * buildbot/test/test_scheduler.py (Scheduling.testTryJobdir): remove the overly-short .timeout on this test, because non-DNotify platforms must fall back to polling which happens at 10 second intervals, so a 5 second timeout would never succeed. 2006-04-24 Brian Warner * docs/buildbot.texinfo (Installing the code): update trial invocation, SF#1469116 by Niklaus Giger. (Attributes of Changes): updated branch-name examples to be a bit more realistic, SF#1475240 by Stephen Davis. * contrib/windows/buildbot2.bat: utility wrapper for windows developers, contributed by Nick Trout (after a year of neglect.. sorry!). SF#1194231. * buildbot/test/test_vc.py (*.capable): store the actual VC binary's pathname in VCS[vcname], so it can be retrieved later (CVSSupport.vc_try_checkout): incorporate Niklaus Giger's patch to strip out non-numeric timezone information, specifically the funky German string that his system produced that confuses CVS. (DarcsSupport.vc_create): use dovc() instead of vc(), this should allow Darcs tests to work on windows * buildbot/scripts/tryclient.py (SourceStampExtractor): use procutils.which() everywhere, to allow tryclient to work under windows. Also from Niklaus Giger, SF#1463394. * buildbot/twcompat.py (which): move the replacement for a missing twisted.python.procutils.which from test_vc.py to here, so it can be used in other places too (specifically tryclient