Ticket #141 (closed defect: fixed)

Opened 1 year ago

Last modified 4 weeks ago

twistd.log -following doesn't work on OS-X

Reported by: warner Assigned to:
Priority: major Milestone: 0.7.7
Component: configuration Version: 0.7.6
Keywords: Cc: smsm1986

Description

Something is broken about the way that 'buildbot reconfig' tries to follow twistd.log on OS-X, it never sees the restart or reconfig finish and eventually gives up.

Benoit did a bit of tracing, the message is attached.

Attachments

tsuna-notes (3.8 kB) - added by warner on 11/21/07 02:09:09.

Change History

11/21/07 02:09:09 changed by warner

  • attachment tsuna-notes added.

11/21/07 02:15:15 changed by warner

  • milestone changed from undecided to 0.7.7.

11/21/07 11:16:15 changed by smsm1986

  • cc set to smsm1986.

this problem occurs with buildbot start too.

12/21/07 21:58:26 changed by warner

More analysis: it appears that reading off the end-of-file on OS-X results in a sticky EOF bit being set, and no further reads will ever give you any data, even if somebody else adds more data to that file. This is in contrast to, say, linux, on which you can read() and get an empty string, then later do another read() and get more data.

To avoid this, GNU tail (from coreutils) does a stat() check to look for a change in filesize, then carefully reads only as many new bytes as stat() suggests are present.

I think we've got two options: either do the same thing (with os.fstat(self.f)[stat.ST_SIZE]), or run /usr/bin/tail -f in a subprocess and watch *it*'s output. It would feel a little bit silly to shell out to a program as simple as tail, but it appears that tail is not as simple as I assumed it was.

Spawning a tail process seems unlikely to work on windows, but on the other hand the existing hand-coded logwatcher doesn't work on windows either (it's completely bypassed, since I think 'buildbot start' doesn't daemonize anyways). So this would be windows-neutral but should make OS-X (and any other platform that has this sort of EOF behavior) work.

12/21/07 22:43:14 changed by warner

  • status changed from new to closed.
  • resolution set to fixed.

I went ahead and implemented the option that runs /usr/bin/tail -f using spawnProcess(), and it seems to work on both linux and OS-X. Yay!.

12/14/08 08:44:34 changed by exarkun

As I mentioned on #316, the new tail-based code fails on Solaris.

12/14/08 08:51:44 changed by exarkun

Also, another approach might be to use os.read instead of file.read. In this case, the lower level API has more consistent behavior across different POSIXes.