Comment by 📡 byte

Re: "help debugging likely disk write issue?"

In: s/FreeBSD

just strip everything from the server part, and write some logging calls in a loop there, this way you can reproduce it without waiting for a week and try to debug it.

📡 byte

2023-11-09 · 7 months ago

1 Later Comment

🌊 tjp [OP] · 2023-11-13 at 18:21:

I finally figured this out.

Based on the frequency of the problem arising I thought it might be connected with svlogd's log rotation, and the svlogd manpage refers to a possibility of it locking up when an external log file processor fails. So to induce it more quickly I reduced the max log file size and generated a bunch of traffic to my server with a shell script.

I was using the !processor functionality of svlogd to run rotated log files through a script (`!gzip`), but also starting svlogd with `chpst -p 1`, which sets a soft limit of 1 process. I removed that limit and it works fine now.

Original Post

🌒 s/FreeBSD

help debugging likely disk write issue? — I have a server service running in a freebsd server I manage, which is connected via a pipe to a logging service which writes to disk. Periodically (generally after a few weeks), the server entirely stops doing anything. It doesn't respond to any requests, and even after a restart it doesn't bind to any ports and none of the usual startup log messages come through. The only way I've found to get it un-stuck is to restart the logging service in the...

💬 tjp · 8 comments · 2023-11-07 · 7 months ago