FortiAnalyzer

Logging

FortiAnalyzer 6.4
© Copyright Fortinet Inc. All rights reserved.
Last Modified:
Wednesday, February 16, 2022
Lesson Overview

Log Overview

Protecting Log Information

Viewing and Searching Logs

Troubleshooting and Managing Logs

Monitoring Events
Log Overview
Objectives
• Describe the purpose of collecting and storing logs
• Describe the log file workflow (archive and analytics)
Purpose of Logs
• Log messages record information containing specific details about what is
occurring on the network

• Must examine multiple logs to discover the chain of activity that led to the breach
• Determine load on network devices
• Track service usage
• Support incident response and forensic analysis

4
Log Storage Regulations
• Regulatory requirements are mandating the use of logs
• Levels and analysis requirements are often defined by legislation
• Ensure logging enabled and recording data at correct level to satisfy regulations
• Logs can provide evidence to deal with offending parties when unauthorized activity
detected
• Logging data must be able to stand up in court
• Monitoring of logs hampered by extensive amounts of data being captured and lack
of means to manage, correlate, and analyze data
• Logging levels should be set high enough to satisfy any regulation and allow you to do your job
• Too much data is as bad as too little

5
Log Data Management Best Practices
• Document what is being logged and why
• Ensure data for all device and applications is being captured and not filtered
(ongoing!)
• Centralize log storage in common format
• Synchronize time on all devices
• Maintain backups of logs and implement a policy that specifies log retention periods
• Define procedures to preserve data integrity
• Test incident response plan (ongoing)

6
Log Types by Device
Device Log type
FortiGate • Traffic [forward, local, sniffer]
• Event [Endpoint, High Availability, System, User, Router, VPN, WAD, Wireless]
• Security [Application Control, AntiVirus, Data Leak Prevention (DLP), Anti-Spam,
Web Filter, Intrusion Prevention System (IPS), Anomaly (D0S-policy), WAF]

FortiCarrier Traffic, Event

FortiCache Traffic, Event, AntiVirus, Web Filter
FortiClient Traffic, Event
FortiMail History, Event, AntiVirus, Email Filter
FortiManager Event
FortiSandbox Malware, Network Alerts
FortiWeb Event, Intrusion Prevention System (IPS), Traffic
Syslog Generic

Must enable ADOMs to collect logs from non-FortiGate devices

7
 Log File Workflow
FortiAnalyzer
ADOM data policy dictates
Raw logs Logs indexed how long logs are kept in
compressed analytics
Logs in database
and saved to
log file

FortiGate 1 SQL
2

Log Phase FortiAnalyzer Immediate Analytic

Log file rolls
Logs purged Location Support
ADOM data policy over and
from database
dictates how long archived Compressed Compressed No. Considered
logs are kept in (Archive logs) in log file “offline”
archive Indexed Compressed Yes. Considered
(Analytics logs) in log file and “online” (can view in
indexed in Log View, FortiView,
Logs deleted SQL database and Reports)

Archive Analytics
8
Knowledge Check
1. Logs in the compressed phase are known as <fill in the blank> logs.
A. Archive
B. Analytics

2. What happens when a log file saved on FortiAnalyzer disks reaches the size
specified in the device log settings?
A. The log file rolls over and is archived.
B. The log file is stored as a raw log and is available for analytic support.

9
Lesson Progress

Log Overview

Protecting Log Data

Viewing and Searching Logs

Troubleshooting and Managing Logs

Monitoring Events
Protecting Log Data
Objectives
• Understand high performance log storage (RAID)
• Perform log backups
• Configure log redundancy
• Configure log encryption
Log Storage Using RAID
• RAID allows you to have a copy of your logs, should a critical event on your
FortiAnalyzer occur
• RAID is not supported on all FortiAnalyzer models

12
 Log Backup
• Protect log data from disk failure, deletion, or corruption Log View > <log type>

• Backup mechanisms include:

• Backing up log files using the GUI or CLI
• GUI (Log View) provides control to download a specific filtered view
• GUI (Log Browse) provides rolled log download (can also schedule log upload
of rolled logs by clicking System Settings > Advanced > Device Log Settings)
• CLI more suitable for bulk data dumps
• Uploading logs to an FTP, SFTP, or SCP server

Log View > Log Browse

Text or CSV format

To restore logs, use execute Includes logs, archives, and quarantine (use logs-only if
restore logs instead only log files needed)

# execute backup logs <device name|all> <ftp|sftp|scp> <IP of server> <user name> <password> <location on server>

13
 Log Redundancy
1. Configure FortiAnalyzer HA cluster 3. Set up log forwarding in Aggregation
• Provides real-time redundancy when primary fails mode
• Provides log and configuration synchronization • Collector sends delta (incremental changes) of its
logs, quarantined files, and archives to an
2. Configure FortiGate to send an aggregation server
identical set of logs to a second • Sends only what analyzer doesn’t have
logging server • If catastrophic failover of analyzer, collector sends all
• FortiAnalyzer or syslog of the data and repopulates analyzer automatically
• CPU, RAM load will be higher on FortiGate • Aggregation mode only supported between two
(more so if SSL enabled) FortiAnalyzers
• Log daemon must handle an additional TCP
connection to a second log device
• If system is sized properly, the extra load won’t be FortiAnalyzer FortiAnalyzer
a factor (Collector) (Analyzer)

FortiAnalyzer FortiAnalyzer
(main) (second)
Aggregation client Aggregation server
FortiGate
14
FortiGate
Log Forwarding
• Forward to FortiAnalyzer, syslog, or Common Event Format (CEF)
• Supports two forwarding modes: aggregation and forwarding
1. Set log forwarding mode • aggregation: Logs and content
files stored and uploaded at
scheduled time
# config system log-forward
• forwarding: Realtime or near
edit <log aggregation ID>
realtime forwarding logs to
set mode <aggregation, forwarding, disable>
servers
end

2. Configure the server (FortiAnalyzer or syslog/CEF that receives logs)

• FortiAnalyzer:
# config system log-forward-service
set accept-aggregation enable
end

3. Configure the client (FortiAnalyzer forwarding the logs) Can specify which device’s logs to
• forward and set log filters to only send
System Settings > Log Forwarding
logs that match filter criteria
15
 Encrypted Log Communication: OFTPS
• The Optimized Fabric Transfer Protocol (OFTP) is used #config log disk setting
over SSL when information is synchronized between set status enable
FortiAnalyzer and FortiGate
• OFTP listens on port TCP/514
• Default setting
• Auto-negotiated, so the oftp server will use the OFTPS protocol only if being used by the
connecting FortiGate

# config log fortianalyzer setting FortiGate’s default encryption level is high

set enc-algorithm {high-medium | high* | low} (low encryption models can do only the low
end level)

# config system global FortiAnalyzer’s default encryption level is

set enc-algorithm {high* | medium | low} high. This encryption level must be equal
end to, or less than, the FortiGate device

16
Preventing Log Modification
• To prevent log modification, you can add a log checksum
• Can configure FortiAnalyzer to record log file hash value, timestamp, and
authentication code at transmission or rolling. Options include:
• md5: Record the log file’s MD5 hash value only
• md5-auth: Record the log file’s MD5 hash value and authentication code
• none: Do not record the log file checksum

# config system global

set log-checksum md5-auth {md5 | md5-auth | none}
end

• You can also

# config change
system the OFTP certificate
certificate oftp to a custom one
set custom enable
set certificate <your PEM formatted certificate>
set private-key <your PEM formatted private key>
end
17
Knowledge Check
1. Which log forwarding mode stores logs and content files and uploads to another
FortiAnalyzer server at a scheduled time?
A. Forwarding mode
B. Aggregation mode

2. FortiAnalyzer uses the Optimized Fabric Transfer Protocol (OFTP) over SSL for
what purpose?
A. To encrypt log communication between devices
B. To prevent log modification

18
Lesson Progress

Log Overview

Protecting Log Data

Viewing and Searching Logs

Troubleshooting and Managing Logs

Monitoring Events
Viewing and Searching Logs
Objectives
• View and search for logs in Log View
• View summary data in FortiView
• View dashboards and widgets features
Log View
• View traffic logs, event logs, or security logs for each ADOM
• Can restrict view to one or more devices or a log group (a group of devices that you can place
together into a single logical object)

21
 Searching for Logs
Set device and time frame

Set filters Custom View

Specify columns to display

(with associated data) in table

View in real time or historical

View raw or formatted
Select device and
log type Log details
22
Saving Frequent Log Searches
• Save frequent searches as a custom view
Log View

23
 Search—Troubleshooting and Tips
• Confirm that Case Sensitive Search • Add column(s) from Column Settings to add
enabled proper filter name to filter list

Enable column
• Right-click data in the log table to set a
filter for that data

Existing filters in
drop-down list
Column now
appears as filter in
drop-down list
24
FortiAnalyzer Application Logs
• FortiAnalyzer application logs:
• Include audit logs for SIEM and SOAR applications
• Each ADOM has its own audit logs
• Accessible in Log View

Use Log Browse to see

log details

25
FortiView
• FortiView includes two panes:
• FortiView
• Monitors
• Each ADOM has its own data analysis in FortiView

26
FortiView Pane
• Integrates real-time and historical data into single, summary views
• Analytics logs only (archive logs not displayed)

27
FortiView Summaries for FortiGate, FortiCarrier, and FortiClient EMS

• View summaries of log data in both

tabular and graphical formats
• Threats
• Top Threats/Threat Map/IOC/FortiSandbox
Detection
• Traffic
• Top Sources/Destinations/Countries/ Policy
Hits/DNS Logs
• Applications and Websites
• Top Application/Cloud Hover and click data
Applications/Websites/Browsing Users for more details
• VPN
• SSL & Dialup IPsec/Site-to-Site IPsec
• System
• Admin Logins/System Events/Resource
Usage/Failed Authentication Attempts

28
Monitors Pane
• Displays both real-time monitoring and historical trends

29
Monitors Dashboard and Widgets Features
Use Blank or Template
• Monitors dashboards and
widgets features:
• Create predefined or custom
dashboards
• For both predefined and custom
dashboards, you can add, delete,
move, or resize widgets
• You can add the same widget or
dashboard multiple times and apply
different settings to each widget
• Each widget monitors one activity
• You can resize widgets or display a
widget in full screen

Change Settings

30
 Dashboards and Monitors
• Compromised Hosts • Threat

• Local System Performance

31
Indicator of Compromise (Compromised Hosts)
• IOC engine detects end users with suspicious web usage compromises by checking
new and historical logs against IOC signatures (based on a FortiGuard subscription)
• Uses today’s FortiGuard threat intelligence to provide visibility of today’s threats
• Flow:
• FortiAnalyzer downloads threat intelligence FortiGuard package (TDS) every day
• FortiGate sends security logs to FortiAnalyzer
• FortiAnalyzer runs real-time threat detection when it receives the logs (FortiGate Web Filter logs)
• Customers can see consolidated view of compromised devices in FortiAnalyzer’s FortiView
• Breach detection engine parses logs into two main categories:
• Infected (real breach)
• Highly Suspicious (possible breach)

For reporting and a more historical audit of

detections of malware, botnet, and intrusions, you
can look at the threat report.
32
IOC Dependencies for Implementation
• One-year subscription to IOC
• FortiAnalyzer includes an evaluation license, but it is restricted
• Web Filter services subscription on FortiGate(s)
• Web Filter policies on FortiGate(s) that send traffic to FortiAnalyzer
• Breach detection or analytic engine runs against the FortiGate web filter logs to identify breaches
related to web traffic

33
IOC/Compromised Host Example

34
Resolve IP Address to Hostname
1. Configure local DNS servers on FortiAnalyzer

2. Enter the following CLI command:

# config system fortiview settings
May induce delay on FortiView
    set resolve-ip enable
end while IPs resolve

• Best practice is to resolve IP addresses on FortiGate end

• Get both source and destination (FortiAnalyzer IP resolve does destination IPs only)
• Offloads the work from FortiAnalyzer
35
Searching Archived Logs Through Log Fetching
• Fetch archive logs from another
FortiAnalyzer and then run queries or Fetch Client: System Settings > Fetcher Management
reports on those archived logs
• Can select devices and time period to be 1. On fetch client, create a profile for the fetch server:
indexed
• Customize log retention settings for
generating reports on older logs
• Avoid log duplication Must have Super_User or
Standard_User profile
• FortiAnalyzers must run the same
firmware version
• FortiAnalyzer fetch client queries the
remote FortiAnalyzer fetch server to
2. On fetch client, send fetch request:
retrieve data 
• Ensure:
• ADOM has enough space for incoming logs!
Fetch server then
• Data policy supports fetching logs of the reviews the request
specified time period and approves or
rejects it
36
FortiSOC
• FortiSOC enables:
• SOAR
• SIEM
• It is a subscription service

37
FortiSoC—SOAR
• FortiSoC valid license allows administrator to access SOAR features
• Enables the ability to automate SOC tasks through the use of playbooks
• Playbooks:
• You can create, add (predefined), edit, delete and run
• Include FortiSOC connectors, which display actions that can be performed
• Local (FortiAnalyzer)
• FortiOS
• FortiClient EMS
• Include triggers that determine when playbook is to be executed
• Each playbook can only include one trigger
• Include tasks that include automated actions that take place on FortiAnalyzer or devices with
configured FortiSoC connectors
• Playbook monitor shows the status of the playbook job

38
FortiSoC—Dashboard
• FortiSoC includes multiple dashboards for playbooks, incidents, and events
FortiSOC > Dashboards > Playbooks FortiSOC > Dashboards > Events

FortiSOC > Dashboards > Incidents

39
Fabric View
• Fabric View module enables:
• To create fabric connectors
• View the list of endpoints
• You can create the following connectors:
• ITSM
• Storage
• Security Fabric

40
FabricView (Contd)
• Identity Center pane:
• Displays a list of users and endpoints
• Correlate them with FortiAnalyzer modules
• Map user and endpoint

• Assets pane:
• To view endpoint and user information to make sure they are compliant
• Useful for incident response and compliance

41
Knowledge Check
1. Which FortiAnalyzer feature allows you to obtain the archived logs of specified
devices from another FortiAnalyzer device?
A. Log forwarding in Aggregation mode
B. Log fetching

2. Which FortiAnalyzer feature becomes available when you subscribe to FortiSoC

service?
A. SOAR
B. Security Ratings

42
Lesson Progress

Log Overview

Protecting Log Information

Viewing and Searching Logs

Troubleshooting and Managing Logs

Monitoring Events
Troubleshooting and Managing Logs
Objectives
• Gather log volume statistics
• Manage disk quota
 Gathering Log Rate and Device Usage Statistics
• Use the following FortiAnalyzer CLI commands to troubleshoot logging issues

Difference between log rate and message rate: One log

message can consist of multiple logs in LZ4 format.

What to Investigate… CLI Command to Use…

What is the log receive rate for each second? # diagnose fortilogd lograte
What are the log receive rate totals? # diagnose fortilogd lograte-total
What is the device log rate? # diagnose fortilogd lograte-device
What is the log rate for each log type? # diagnose fortilogd lograte-type

What is the message receive rate for each second? # diagnose fortilogd msgrate

What is the SQL insertion status? # diagnose sql status sqlplugind

What is the device log usage for all logging devices? # diagnose log device

45
 Gathering Log Rate and Log Volume Per ADOM
• Use the following FortiAnalyzer CLI commands to calculate log rate and log volume
per ADOM
What to Investigate… CLI Command to Use…
What is the log receive rate for all ADOMs? # diagnose fortilogd lograte-adom all

What is the log receive rate for a specific # diagnose fortilogd lograte-adom {adom-name}
ADOM?

What is the log volume for all ADOMs? # diagnose fortilogd logvol-adom all

What is the log volume for a specific # diagnose fortilogd logvol-adom {adom-name}
ADOM?

46
Insert Rate vs. Receive Rate and Log Insert Lag
• Insert Rate vs. Receive Rate System Settings > Dashboard
• Insert Rate = SQL Insertion Rate
• sqlplugind
• Receive Rate = Raw Receiving Rate
• fortilogd

• Log Insert Lag Time

• Difference between log received and log
inserted in the database

47
Increase ADOM Disk Quota
• Monitor log rate coming from each device # diagnose log device
• If high volume of logs, consider increasing ADOM log quota so the oldest logs are
not lost

• Allocating insufficient quota If ADOMs are enabled If ADOMs are disabled

to an ADOM can: (and only root ADOM)

• Prevent you from reaching your log

retention objective
• Cause unnecessary CPU resources System Settings > All ADOMs Log View > System Storage
enforcing quota with log deletion
and database trims
• Adversely affect reporting if the
quota enforcement acts on
analytical data before a report is
complete

48
Rolling Logs and Auto-Deleting Old Logs
• How can you better manage your logs
on disk?
• Roll log files when the size System Settings > Advanced > Device Log Settings
exceeds a set threshold

System Settings > Advanced > File Management

• Automatically delete logs
of a specified age

49
Knowledge Check
1. What data does the CLI command # diagnose fortilogd lograte
provide?
A. The log receive rate per second
B. The message receive rate per second

2. Your ADOM data policy is set to keep logs in archive for 365 days, but the logs are
being deleted prematurely from that ADOM and CPU resources are also high.
What is the most likely problem?
A. The ADOM disk quota is set too low, based on log rates.
B. A global automatic deletion policy is set to delete device logs every six months.

50
Lesson Progress

Log Overview

Protecting Log Information

Viewing and Searching Logs

Troubleshooting and Managing Logs

Monitoring Events
Monitoring Events
Objectives
• Configure event handlers
 Monitoring Events
• Event Monitor displays events based on configured event handlers

FortiSoC > Event Monitor > All Events

Right click for

more details and to
comment or
acknowledge
53
Configuring Event Handlers
• Event handlers are specific matched conditions in the raw logs
• Configured for each ADOM and can apply to a single device or multiple devices
• Can use predefined event handlers or create your own
• Can send alert notifications (must configure back end):
• Alert email
• SNMP trap FortiSoC > Handlers > Event Handler List
• Syslog server

54
 Generic Text Filters
• Generic text filters allow more precise and flexible control over which logs trigger an event
• Multiple operators and logic are supported
• Supported operators:

Operator Meaning
== Equal (Exact match)

!= Not equal  (Not matching)

< Smaller than

<= Smaller than or equal

> Greater than

>= Greater than or equal

~ Contained (Included somewhere in the string)

!~ Not contained (Not included)

Tip: Search your raw logs for the log file on which you
Tokens: '(', ')', '&', '|', 'and', 'or',  'not' want to add an event handler and copy the string you
want to match
55
 Customizing Event Handlers
• Additional Info field can be FortiSoC > Handlers > Event Handler List
used to
o Change the default event
handler queries
o Create custom
messages/notifications

Customized event
handler

FortiSoC > Event Monitor > All Events

Default message

Customized event
handler message

56
Knowledge Check
1. What are event handlers?
A. Threats identified by FortiGuard
B. Specific matched conditions in the raw logs

57
Logging Best Practices
• Upload FortiAnalyzer local logs to a remote server
• Increase local event logging level to debug
• Configure SNMP traps for critical system events
• Configure log upload for rolled logs on a daily basis

58
Lesson Progress

Log Overview

Protecting Log Information

Viewing and Searching Logs

Troubleshooting and Managing Logs

Monitoring Events
Review
 Describe the purpose of collecting and storing logs
 Describe the log file workflow (archive and analytics)
 Understand high performance log storage (RAID)
 Perform log backups
 Configure log redundancy
 Configure log encryption
 View and search logs in Log View
 View summary data in FortiView
 Gather log volume statistics
 Manage disk quota
 Configure event handlers