Case study: After some changes I've noticed logrotate flushes the main log (as it should) but all gzipped archived copies it creates are empty (20b). Logrotate destroys logs instead of preserving them.
You had one job, logrotate!
Turns out my logrotate file looked like this:
/var/log/httpd/*_log /custompath/log/*_log {
..
create ...
}
And the latter had been symlinked to the former.
It made sense when I wrote that. Cover all the bases. But the way logrotate works, it first scans all paths for files that need rotating and then applies rotation to them all.
So if I have /custompath/log/error_log reflected as /var/log/httpd/error_log, logrotate notices two files in need of rotation. It then compresses the first one, replaces it with empty log (create), then compresses the second one (now empty) and replaces the first .gz with an empty archive.
Nice job, logrotate. Make sure the collected filenames resolve to unique files? Nope. Maybe at least don't overwrite already existing gzips? Nope.