Apple's Unified Logging for

Sysadmins
MacSysAdmin 2017

Tim Sutton

Sauce Labs
http://www.deploystudio.com/Forums/viewtopic.php?
id=1952
MacSysAdmin 2010 (!)
 syslog
• syslog -w
• -w waits for syslog information
• sudo syslog -c 0 -d
• log everything: Emergency to Debug
• syslog -c 0
• check filter mask
Penn State MacAdmins 2012

Ed Marczak: Logs, Damn Logs, and Statistics

➜ syslog -w

NOTE: Most system logs have moved to a new logging

system. See log(1) for more information.
Curiosity
I told Tycho I would give a talk
about Apple Unified Logging at
MacSysAdmin
Frustration
Penn State MacAdmins 2017

peeling back the bark

> @chilcote
macops.ca/logging
(https://developer.apple.com/videos/play/wwdc2016/721)
 Efficiency

• Logging now to a database, no plaintext - only `log` and Console can read
these

• Persistence of log data varies depending on importance

• Message formatting happens at display / review time, not during writing

Log message types, and scopes
• "Standard" types

• Default

• Info

• Debug

• "Special" types

• Error

• Fault

• Events can belong to a subsystem, and categories within

 Privacy
{
"category" : "Registration",
"processImageUUID" : "E60289AD-3395-3F98-84AD-A6CDDF651631",
"processUniqueID" : 334,
"threadID" : 2895,
"timestamp" : "2017-10-01 13:09:35.139280-0400",
"traceID" : 28876680072396804,
"messageType" : "Default",
"senderProgramCounter" : 960056,
"processID" : 334,
"machTimestamp" : 31131983679,
"timezoneName" : "",
"subsystem" : "com.apple.IDS",
"eventMessage" : " ** Found signed in iCloud account: <private>",
"senderImageUUID" : "E60289AD-3395-3F98-84AD-A6CDDF651631",
"processImagePath" : "\/System\/Library\/PrivateFrameworks\/IDS.framework\/identityservicesd.app\/Contents\/MacOS\/iden
"senderImagePath" : "\/System\/Library\/PrivateFrameworks\/IDS.framework\/identityservicesd.app\/Contents\/MacOS\/ident
}
show vs. stream
Console.app 1.0
 Configuration

• sudo log config --mode 'level:debug' --subsystem com.apple.WebDriver

• logging configuration profile property list files

• (Apple's words, not mine)

 Logging Preferences
➜ ls /System/Library/Preferences/Logging/Subsystems

com.apple.Accessibility.plist com.apple.TCC.plist com.apple.icloudpreferences.plist com.apple.persona.plist

com.apple.AppKit.plist com.apple.TimeMachine.plist com.apple.internetAccounts.plist com.apple.pf.plist
com.apple.AppleIR.plist com.apple.accounts.plist com.apple.libsqlite3.plist com.apple.photoanalysisd.graph.p
com.apple.AssetCache.plist com.apple.amp.MediaServices.plist com.apple.locationd.Core.plist com.apple.photoanalysisd.job.pli
com.apple.AssetCacheServices.plist com.apple.authkit.plist com.apple.locationd.Legacy.plist com.apple.photoanalysisd.plist
com.apple.BezelServices.plist com.apple.avfaudio.plist com.apple.locationd.Motion.plist com.apple.pluginkit.plist
com.apple.DesktopServices.plist com.apple.awd.awdd.plist com.apple.locationd.Position.plist com.apple.sandbox.reporting.plis
com.apple.ExchangeWebServices.plist com.apple.awd.framework.plist com.apple.locationd.Utility.plist com.apple.sbd.plist
com.apple.Finder.plist com.apple.bluetooth.plist com.apple.mDNSResponder.plist com.apple.securityd.plist
com.apple.HTTPServer.plist com.apple.calendar.plist com.apple.mac.install.plist com.apple.sharing.plist
com.apple.IPConfiguration.plist com.apple.captive.plist com.apple.mail.plist com.apple.siri.plist
com.apple.ManagedClient.plist com.apple.catalyst.plist com.apple.mediaremote.plist com.apple.social.plist
com.apple.NetworkSharing.plist com.apple.cdp.plist com.apple.multipeerconnectivity.plist com.apple.socialpushagent.plist
com.apple.ProtectedCloudStorage.plist com.apple.clouddocs.plist com.apple.network.plist com.apple.symptomsd.plist
com.apple.Safari.plist com.apple.coreanimation.plist com.apple.networkextension.plist com.apple.syncdefaults.plist
com.apple.SafariShared.plist com.apple.coreaudio.plist com.apple.networkserviceproxy.plist com.apple.useractivity.plist
com.apple.SkyLight.plist com.apple.coredata.plist com.apple.nlcd.plist
com.apple.StandaloneHIDFudPlugins.plist com.apple.duetactivityscheduler.plist com.apple.notes.plist
com.apple.SystemConfiguration.plist com.apple.eapol.plist com.apple.passkit.plist

➜ ls /System/Library/Preferences/Logging/Resources
com.apple.WebDriver.plist

➜ ls /Library/Preferences/Logging/Subsystems

com.apple.WebDriver.plist com.sentinelone.sentineld-helper.plist com.sentinelone.sentineld.plist

com.sentinelone.sentineld-guard.plist com.sentinelone.sentineld-updater.plist com.sentinelone.sentinelctl.plist
https://developer.apple.com/documentation/os/logging?language=objc
 76 <key>com.great-company.my-great-app</key>
77 <dict>
78 <key>DEFAULT-OPTIONS</key>
Search… All gists GitHub Sign up for a GitHub account Sign in
79 <dict>
80 <key>Level</key>
81 <dict>
Instantly share code, notes, and snippets. Create a gist now
82 <!-- The lowest-level log that's enabled.
83 All log messages sent through NSLog are at
zwaldowski / Extra Logging for My Great App.mobileconfig Star 39 Fork 7 84 the Default level.
85
Last active 25 days ago
86 Accepted values:
87 - Inherit: The default. Categories inherit
Code Revisions 6 Stars 39 Forks 7 Embed Download ZIP
<script src="https://gist.github.com/zwaldowski/f9c82dfe1595506a36bd71f55cbb7538.js"></script>
88 subsystem behavior, subsystems inherit the
89 system (OS) behavior.
90 - Default: Only default-level messages.
Apple Configuration Profile for Logging in iOS 10 and macOS Sierra 91 - Info: Default-level and info-level messages.
92 - Debug: All messages. -->
93 <key>Enable</key>
Extra Logging for My Great App.mobileconfig Raw
94 <string>Debug</string>
1 <?xml version="1.0" encoding="UTF-8"?> 95 <!-- The lowest-level log that's written to
2 <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> 96 disk.
3 <plist version="1.0"> 97
4 <!-- iOS 10, macOS Sierra, and friends bring a new logging subsystem that's 98 Accepted values:
5 supposed to scale from the kernel, up to frameworks, and up to apps. It defaults 99 - Inherit: The default. Categories inherit
6 to a more regimented, privacy-focused approach that large apps and complex 100 subsystem behavior, subsystems inherit the
7 systems need. 101 system (OS) behavior.
8 102 - Default: Only default-level messages.
9 It, along with Activity Tracing introduced in iOS 8 and macOS Yosemite and the 103 - Info: Default-level and info-level messages.
10 Console app in macOS Sierra, hope to help you graduate from caveman debugging to 104 - Debug: All messages. -->
11 quickly identify issues across time, events, user input, and even process 105 <key>Persist</key>
12 boundaries. 106 <string>Debug</string>
13 107 </dict>
14 This configuration profile template allows you to change logging behaviors 108 <!-- Changes how `%@` et. al. are formatted into
15 for your app, your frameworks, third party code, and even Apple code without 109 the log.
16 access to the source. The idea is to leave all your logging code in your app — 110
17 even when shipping – and change the information level at runtime. 111 To ensure the privacy of users, it is recommended
18 112 that log messages consist strictly of static strings
19 Read more about this: 113 and numbers.
20 - https://developer.apple.com/reference/os/1891852-logging 114
21 - https://developer.apple.com/videos/play/wwdc2016/721/ 115 Accepted values:
22 116 - Inherit. The default. Categories inherit
23 You can modify and install this payload by double-clicking it in macOS, or on 117 subsystem behavior, subsystems inherit the
24 an iOS-based device using Apple Configurator. You can also use Apple 118 system (OS) behavior.
25 Configurator to sign and send this profile to others. 119 - Public: Dynamic strings and complex dynamic
26 120 objects are collected for the log.
27 This, too, is all documented: 121 - Private: Static strings and numbers are collected
28 - https://developer.apple.com/library/ios/documentation/NetworkingInternet/Conceptual/iPhoneOTAConfiguration/Introduction/Introduction.html#//apple_ref/doc/uid/TP40009505
122 for the log. -->
29 - https://developer.apple.com/library/ios/featuredarticles/iPhoneConfigurationProfileRef/Introduction/Introduction.html 123 <key>Default-Privacy-Setting</key>
30 - http://nshipster.com/configuration-profiles/ 124 <string>Public</string>
31 125 <!-- Changes how long non-critical messages stay
32 And, always remember: assume positive intent. --> 126 in the log, in days (I think).
33 <dict> 127
34 <!-- Change this UUID using `uuidgen`. --> 128 Keys for this dictionary are the non-critical
35 <key>PayloadUUID</key> 129 log levels: Default, Debug, and Info. -->
36 <string>00000000-0000-0000-0000-000000000000</string> 130 <key>TTL</key>
37
38
39
https://gist.github.com/zwaldowski/f9c82dfe1595506a36bd71f55cbb7538
<!-- Increment when changing the contents of ANY payload. -->
<key>PayloadVersion</key>
<integer>1</integer>
131
132
133
<dict>
<key>Default</key>
<integer>14</integer>
40 <!-- Change this identifier and other info based on your credentials. --> 134 </dict>
/usr/bin/log
Windows: APIs vs. Files
Tips
 process/senderImagePath
• --predicate processImagePath and senderImagePath really are full paths,
even though `log` and console show only the process name in typical output
 process/senderImagePath
2017-09-30 02:46:10.603459-0400 0x840 Error 0x0 43 eoshostd: (EmbeddedOSSupportHost) connect() failed, reason: <private>, giving up

--style syslog
2017-09-30 02:46:10.603459-0400 localhost eoshostd[243]: (EmbeddedOSSupportHost) connect() failed, reason: <private>, giving up

--style json
{
"processImageUUID" : "469A00D6-3C4F-33EE-95F1-EDE17BCCB531",
"processUniqueID" : 243,
"threadID" : 2112,
"timestamp" : "2017-09-30 02:46:10.603459-0400",
"traceID" : 1260856231845957636,
"messageType" : "Error",
"senderProgramCounter" : 293550405,
"processID" : 243,
"machTimestamp" : 159540275634612,
"timezoneName" : "",
"eventMessage" : "connect() failed, reason: <private>, giving up",
"senderImageUUID" : "16538049-0951-3460-ABCD-8E6C06D631D6",
"processImagePath" : "\/usr\/libexec\/eoshostd",
"senderImagePath" : "\/System\/Library\/PrivateFrameworks\/EmbeddedOSSupportHost.framework\/Versions\/A\/EmbeddedOSSupportH
}
Metadata naming in Console.app, `log`
Console.app 'log' command

Message --predicate 'eventMessage contains "fail"'

Process (both path and name) --predicate 'processImagePath == "/usr/sbin/cfprefsd"'

Library (both path and name) --predicate 'senderImagePath CONTAINS[cd] "cloud"'

Type (log level) --predicate 'messageType == info'

Subsystem --predicate 'subsystem == "com.apple.securityd"'

Category --predicate 'category == "User Defaults Daemon"'

PID --process
 `log` and predicates
• Quoting really matters for --predicate option!

• 'eventMessage contains "fail"'

• 'messageType == info', 'eventType == logEvent'

• Operator capitalization doesn't matter:

• 'processImagePath CONTAINS[cd] "cfprefs"'

• See Apple's Predicate Programming Guide (or Munki documentation..)

• https://developer.apple.com/documentation/foundation/nspredicate

• https://github.com/munki/munki/wiki/Conditional-Items
Apple Developer Docs
 `log config`
➜ sudo log config
System mode = INFO STREAM_LIVE

➜ sudo log config --mode 'private_data:on'

➜ sudo log config

System mode = INFO STREAM_LIVE PRIVATE_DATA

➜ sudo log config --reset

➜ sudo log config

System mode = INFO
 `log config`
➜ sudo log config --status --subsystem com.apple.notes
Mode for 'com.apple.notes' DEBUG PERSIST_DEBUG

➜ /usr/libexec/PlistBuddy -c 'Print' \
/System/Library/Preferences/Logging/Subsystems/com.apple.notes.plist
Dict {
DEFAULT-OPTIONS = Dict {
Default-Privacy-Setting = Public
Enable-Oversize-Messages = true
Level = Dict {
Persist = Debug
Enable = Debug
}
TTL = Dict {
Error = 30
}
}
}
 `log stats` (new in High Sierra)
stats Shows a breakdown of the events contained within a log datastore or archive. The following options can
be supplied to all modes of log stats:

--archive archive Display statistics for events stored in the given archive. The archive must be
a valid log archive bundle with the suffix .logarchive.

--sort events | bytes Sort tabulated data output by number of events, or number of bytes.

--count count | all Limit tabulated data to the given number of lines, or all displays all entries
in tables.

--style human | json Control the format style of the requested output mode.

In addition, one of the following output modes can be supplied:

--overview Displays statistics for the entire archive.

--per-book Displays statistics per log book, the subsections of a log archive.

--per-file Displays statistics per file in the archive.

--sender sender Displays statistics for a given sender image name.

--process process Displays statistics for a given originating process.

--predicate predicate Displays statistics for all events matching the given predicate.
 `log stats` (new in High Sierra)
➜ log stats --predicate 'subsystem == "com.apple.notes"'
== predicate ===========================================================
size: 1,847 bytes (uncompressed)
start: Sat Sep 9 16:14:05 2017
end: Sat Sep 9 16:14:08 2017

events: [ total log trace signpost ]

[ 26 26 0 0 ]

activity: [ create transition action ]

[ 0 0 0 ]

log messages: [ default info debug error fault ]

[ 0 0 26 0 0 ]

processes:
[ events (%total), decomp. bytes (%total), image UUID, image ]
[ 20 ( 76.9%), 1,567 ( 84.8%), 65768508-7764-396D-9DC6-C3393641B613, com.apple.Notes.SpotlightIndexExt
[ 6 ( 23.1%), 280 ( 15.2%), A77F5314-1B8B-35B5-8376-62677A2DAC6F, com.apple.Notes.datastore ]

senders:
[ events (%total), decomp. bytes (%total), image UUID, image ]
[ 17 ( 65.4%), 1,519 ( 82.2%), D2373F71-3179-3516-9234-DCA97ABD4FD7, NotesShared ]
[ 6 ( 23.1%), 280 ( 15.2%), A77F5314-1B8B-35B5-8376-62677A2DAC6F, com.apple.Notes.datastore ]
[ 2 ( 7.7%), 32 ( 1.7%), 65768508-7764-396D-9DC6-C3393641B613, com.apple.Notes.SpotlightIndexExt
[ 1 ( 3.8%), 16 ( 0.9%), 4FB06CCB-58C3-3860-BC6C-455E0B36E1FD, Notes ]

categories:
[ events (%total), decomp. bytes (%total), category ]
[ 15 ( 57.7%), 1,251 ( 67.7%), Indexer ]
[ 8 ( 30.8%), 572 ( 31.0%), ]
eclecticlight.co/category/macs (Howard Oakley)
Consolation
Other uncertainties
 "Offline" logs?

show Shows contents of the system log datastore, archive or a specific

tracev3 file. If a file or archive is not specified, the system
datastore will be shown. If it is from a future system version that
log cannot understand, it exists with EX_DATAERR (65) and an error
message. The output contains only default level messages unless
--info and/or --debug are specified.

➜ ~ find /var/db/diagnostics -name '*.tracev3' | wc -l

143
eclecticlight.co/category/macs (Howard Oakley)
MakeLogarchive
No public API for reading logs
(ConsoleKit and LoggingSupport private frameworks)
Updates for apps to use 10.12 SDK

/var/log, /Library/Logs, etc.

syslog
 Thank you!

macops.ca/logging