Add a Source step that get a tar.gz archive from an FTP server and uncompress it, ready to be build.

FtpTar? source step patch

I think there's some good discussion taking place on this one in the mailing list, but I'll ask the question here too: why make this a new step? Why not invoke ftp or wget or curl or some other convenient command-line tool, then invoke 'tar xf' on the file?

I can see how this would make life easier for a windows buildslave that doesn't have those tools already installed, but I'm not sure this is worth having this extra single-purpose code inside buildbot. FTP (versus http, or scp) and gzip (versus bz2 or zip) and tar (versus zip or some other aggregation format) is not very general. It might be reasonable to make it more general (i.e. have a trio of 'get file from server onto the slave' step, ftp/http/scp, and a 'unpack file' step that knows about .zip or .tar.gz or whatever), but of course then we're adding 4 or 5 buildsteps instead of just one.

It is possible to run ShellCommands outside the normal build/ subdirectory, by providing an alternate workdir= argument. For example:

 f.addStep(ShellCommand(workdir=".", command=["wget", "ftp://blah/foo.tar.gz"]))
 f.addStep(ShellCommand(workdir=".", command=["tar", "xf", "foo.tar.gz"]))
 f.addStep(ShellCommand(workdir=".", command=["mv", "foo", "build"]))
 f.addStep(Compile)

Tell me more about the use case for this feature.. let's see if adding a new buildstep is the right way to go, or maybe improving the docs or adding some convenience features to other steps might be better.

As I explained on the mailing list, all of the Source steps provide management of the build directory. You solved it in a single command (mv) for UNIX, but you would require at least 2 more on Windows. And frankly speaking, using a ShellCommand to issue a mv is overkill. Just think about the waterfall page having a (hopefully green) box for mv. And think also to all that lies behind for it, such as the log files.

I am currently preparing another version. I finally split the FtpTar command into two. There's the FtpDownload step, which is a Source step, and takes care of the build directory. It is now implemented using twisted. As such it is self sufficient for FTP and can be used standalone. But it is meant to be used with ...

The Uncompress step, that is a ShellCommand with some name, description definitions, and a special parameter filename that generates automatically the command.

I'd have liked to keep these two in a single step. I mean having the FTP download, plus the uncompress step integrated, the latter kept in the form of a ShellCommand step (managing all of the uncompression possibilities would be really too much work).

Separate FtpDownload? and Uncompress steps (from pristine 0.7.8, i.e. not from FtpTar?)