twistd.log -following doesn't work on OS-X

[warner]

Something is broken about the way that 'buildbot reconfig' tries to follow twistd.log on OS-X, it never sees the restart or reconfig finish and eventually gives up.

Benoit did a bit of tracing, the message is attached.

[smsm1986]

this problem occurs with buildbot start too.

[warner]

More analysis: it appears that reading off the end-of-file on OS-X results in a sticky EOF bit being set, and no further reads will ever give you any data, even if somebody else adds more data to that file. This is in contrast to, say, linux, on which you can read() and get an empty string, then later do another read() and get more data.

To avoid this, GNU tail (from coreutils) does a stat() check to look for a change in filesize, then carefully reads only as many new bytes as stat() suggests are present.

I think we've got two options: either do the same thing (with os.fstat(self.f)[stat.ST_SIZE]), or run /usr/bin/tail -f in a subprocess and watch *it*'s output. It would feel a little bit silly to shell out to a program as simple as tail, but it appears that tail is not as simple as I assumed it was.

Spawning a tail process seems unlikely to work on windows, but on the other hand the existing hand-coded logwatcher doesn't work on windows either (it's completely bypassed, since I think 'buildbot start' doesn't daemonize anyways). So this would be windows-neutral but should make OS-X (and any other platform that has this sort of EOF behavior) work.

[warner]

I went ahead and implemented the option that runs /usr/bin/tail -f using spawnProcess(), and it seems to work on both linux and OS-X. Yay!.

[exarkun]

As I mentioned on #316, the new tail-based code fails on Solaris.

[exarkun]

Also, another approach might be to use os.read instead of file.read. In this case, the lower level API has more consistent behavior across different POSIXes.