On OS X, /var/log/system.log is rotated by the newsyslog
command, which is run at half-past each hour (see /System/Library/LaunchDaemons/com.apple.newsyslog.plist) whenever the Mac is running and awake. The rotation rules are in /etc/newsyslog.conf, from which the relevant lines are:
# logfilename [owner:group] mode count size when flags [/pid_file] [sig_num]
/var/log/system.log 640 7 * @T00 J
If I understand this right (see the newsyslog.conf man page), the "@T00" in the "when" field means that the log will be rotated if newsyslog runs between midnight and 1am -- i.e. at the 12:30am run. But if the Mac is off or asleep at 12:30, this run won't happen and the log won't get rotated that day, which is probably why you see such variable results.
If you want to change the rotation criteria, feel free to edit /etc/newsyslog.conf; most of the other logs get rotated based on size, and I'm not sure why system.log is different.
UPDATE: Starting in 10.9, the rotation control for system.log moved from /etc/newsyslog.conf to /etc/asl.conf (the config for the Apple System Log facility, which does the writing to system.log). It'll have a section like this:
# Rules for /var/log/system.log
> system.log mode=0640 format=bsd rotate=seq compress file_max=5M all_max=50M
? [= Sender kernel] file system.log
? [<= Level notice] file system.log
? [= Facility auth] [<= Level info] file system.log
? [= Facility authpriv] [<= Level info] file system.log
The "rotate=seq compress file_max=5M all_max=50M" section controls archiving and retention. See man asl.conf for more info and options.
The logging system got a pretty thorough rewrite in 10.12, but this doesn't appear to have changed.