Intrusion Detection System (IDS) and Its Detailed

Working Function – SOC/SIEM

An intrusion detection system (IDS) is a type of security software designed to

automatically alert administrators when someone or something is trying to
compromise information system through malicious activities such as DDOS
Attacks or security policy violations.

An IDS works by monitoring system activity through examining vulnerabilities in

the system, the integrity of files and analyzing patterns based on already known
attacks. It also automatically monitors the Internet to search for any of the
latest threats which could result in a future attack.

Detection Methods
An IDS can only detect an attack. It cannot prevent attacks. In contrast, an IPS
prevents attacks by detecting them and stopping them before they reach the target.

An attack is an attempt to compromise confidentiality, integrity, or availability.

The two primary methods of detection are signature-based and anomaly-based. Any
type of IDS (HIDS or NIDS) can detect attacks based on signatures, anomalies, or
both.

The HIDS monitors the network traffic reaching its NIC, and the NIDS monitors the
traffic on the network.

Host Based intrusion detection system (HIDS)

 A host-based intrusion detection system (HIDS) is additional software installed on a
system such as a workstation or a server.

It provides protection to the individual host and can detect potential attacks and
protect critical operating system files. The primary goal of any IDS is to monitor
traffic.

The role of a host Intrusion Detection System is passive, only gathering, identifying,
logging, and alerting. Examples of HIDS:

 OSSEC – Open Source Host-based Intrusion Detection System

 Tripwire
 AIDE – Advanced Intrusion Detection Environment
 Prelude Hybrid IDS

The primary goal of any IDS is to monitor traffic. For a HIDS, this traffic passes
through the network interface card (NIC). Many host-based IDSs have expanded to
monitor application activity on the system.

As one example, you can install a HIDS on different Internet-facing servers, such
as web servers, mail servers, and database servers. In addition to monitoring the
network traffic reaching the servers, the HIDS can also monitor the server
applications.

It’s worth stressing that a HIDS can help detect malicious software (malware) that
traditional anti-virus software might miss.

Because of this, many organizations install a HIDS on every workstation as an extra

layer of protection, in addition to traditional anti-virus software. Just as the HIDS on a
server is used primarily to monitor network traffic, a workstation HIDS is mainly used
to monitor network traffic reaching the workstation. However, a HIDS can also
monitor some applications and can protect local resources such as operating system
files. In other organizations, administrators only install a HIDS when there’s a
perceived need.

For example, if an administrator is concerned that a specific server with proprietary

data is at increased risk of an attack, the administrator might choose to install a HIDS
on this system as an extra layer of protection.

Each uncompleted session consumes resources on the server, and if the SYN flood
attack continues, it can crash the server.
Some servers reserve a certain number of resources for connections, and once the
attack consumes these resources, the system blocks additional connections. Instead of
crashing the server, the attack prevents legitimate users from connecting to the server.

IDSs and IPSs can detect an SYN flood attack and respond to block the attack.
Additionally, many firewalls include a flood guard that can detect SYN flood attacks
and take steps to close the open sessions.

Network-Based Intrusion Detection System (NIDS)

A network-based intrusion detection system (NIDS) monitors activity on the network.

An administrator installs NIDSs sensors on network devices such as routers and
firewalls.

These sensors gather information and report to a central monitoring server hosting a
NIDS console.A NIDS is not able to detect anomalies on individual systems or
workstations unless the anomaly causes a significant difference in network traffic.

Additionally, a NIDS is unable to decrypt encrypted traffic. In other words, it can only
monitor and assess threats on the network from traffic sent in plaintext or non-
encrypted traffic.
Important tools for NIDS

Examples of Network IDS:

SNORT

The decision on where you want to place the sensors depends on what you want to
measure. For example, the sensor on the Internet side of the firewall will see all the
traffic.

However, the sensor on the internal side of the firewall will only see traffic that passes
through the firewall. In other words, the firewall will filter some attacks, and the
internal sensor won’t see them.

If you want to see all attacks on your network, put a sensor on the Internet side. If you
only want to see what gets through, put sensors internally only. If you want to see
both, put sensors in both places.

Signature-Based Detection
Signature-based IDSs (also called definition-based) use a database of known
vulnerabilities or known attack patterns. For example, tools are available for an
attacker to launch a SYN flood attack on a server by simply entering the IP address of
the system to attack.

The attack tool then floods the target system with synchronize (SYN) packets, but
never completes the three-way Transmission Control Protocol (TCP) handshake with
the final acknowledge (ACK) packet. If the attack isn’t blocked, it can consume
resources on a system and ultimately cause it to crash.

If the attack isn’t blocked, it can consume resources on a system and ultimately cause
it to crash. However, this is a known attack with a specific pattern of successive SYN
packets from one IP to another IP.

The Intrusion Detection System can detect these patterns when the signature database
includes the attack definitions. The process is very similar to what antivirus software
uses to detect malware. You need to update both Intrusion Detection System
signatures and antivirus definitions from the vendor on a regular basis to protect
against current threats.
 Anomaly-Based Detection
Anomaly-based (also called heuristic-based or behavior-based) detection first
identifies normal operation or normal behavior. It does this by creating a performance
baseline under normal operating conditions.

The IDS provides continuous monitoring by constantly comparing current network

behavior against the baseline. When the Intrusion Detection System detects abnormal
activity (outside normal boundaries as identified the baseline), it gives an alert
indicating a potential attack.

Anomaly-based detection is similar to how heuristic-based antivirus software works.

Although the internal methods are different, both examine activity and make decisions
that are outside the scope of a signature or definition database.

This can be effective at discovering zero-day exploits. A zero-day vulnerability is

usually defined as one that is unknown to the vendor. However, in some usage,
administrators define a zero-day exploit as one where the vendor has not released a
patch.

In other words, the vendor may know about the vulnerability but has not written,
tested, and released a patch to close the vulnerability yet. In both cases, the
vulnerability exists and systems are unprotected. If attackers discover the
vulnerabilities, they try to exploit them. However, the attack has the potential to create
abnormal traffic allowing an anomaly-based system to detect it.

Any time administrators make any significant changes to a system or network that
cause normal behavior to change, they should recreate the baseline. Otherwise, the
IDS will constantly alert on what is now normal behavior.

Physical Intrusion Detection System

Physical intrusion detection is the act of identifying threats to physical systems.
Physical intrusion detection is most often seen as physical controls put in place to
ensure CIA. In many cases physical intrusion detection systems act as prevention
systems as well. Examples of Physical intrusion detections are:

 Security Guards
 Security Cameras
 Access Control Systems (Card, Biometric)
 Firewalls
 Man Traps
 Motion Sensors

Wireless Detection
A wireless local area network (WLAN) IDS is similar to NIDS in that it can analyze
network traffic. However, it will also analyze wireless-specific traffic, including
scanning for external users trying to connect to access points (AP), rogue APs, users
outside the physical area of the company, and WLAN IDSs built into APs.

As networks increasingly support wireless technologies at various points of a

topology, WLAN IDS will play larger roles in security. Many previous NIDS tools
will include enhancements to support wireless traffic analysis. Some forms of IDPS
are more mature than others because they have been in use much longer. Network-
based IDPS and some forms of host-based IDPS have been commercially available for
over ten years.

Network behavior analysis software is a somewhat newer form of IDPS that evolved
in part from products created primarily to detect DDoS attacks, and in part from
products developed to monitor traffic flows on internal networks.

Wireless technologies are a relatively new type of IDPS, developed in response to the
popularity of wireless local area networks (WLAN) and the growing threats against
WLANs and WLAN clients.

False Positives Vs False Negatives

IDSs are susceptible to both false positives and false negatives. A false positive is an
alert or alarm on an event that is non-threatening, benign, or harmless.

A false negative is when an attacker is actively attacking the network, but the system
does not detect it. Neither is desirable, but it’s impossible to eliminate both.

Most IDSs trigger an alert or alarm when an event exceeds a threshold. Consider the
classic SYN flood attack, where the attacker withholds the third part of the TCP
handshake. A host will send an SYN packet and a server will respond with an
SYN/ACK packet.

However, instead of completing the handshake with an ACK packet, the attacking
host never sends the ACK, but continues to send more SYN packets. This leaves the
server with open connections that can ultimately disrupt services.

If a system receives one SYN packet without the accompanying ACK packet, is it an
attack?  Probably not. This can happen during normal operations.
If a system receives over 1,000 SYN packets from a single IP address in less than 60
seconds, without the accompanying ACK packet, is it an attack? Absolutely.

With this in mind, administrators set the threshold to a number between 1 and 1,000 to
indicate an attack. If administrators set it too low, they will have too many false
positives and a high workload as they spend their time chasing ghosts. If they set the
threshold too high, actual attacks will get

If they set the threshold too high, actual attacks will get through without
administrators knowing about them. Most administrators want to know if their system
is under attack. That’s the primary purpose of the IDS.

However, an IDS that constantly cries “Wolf!” will be ignored when the real wolf
attacks.

It’s important to set the threshold low enough to reduce the number of false positives,
but high enough to alert on any actual attacks.There is no perfect number for the
threshold. Administrators adjust thresholds in different

Reporting
IDSs report on events of interest based on their settings. All events aren’t attacks or
actual
issues, but instead, they provide a report indicating an event might be an alert or an
alarm. Administrators investigate to determine if it is valid.

Some systems consider an alarm and an alert as the same thing. Other systems use an
alert for a potentially serious issue, and an alarm as a relatively minor issue. The goal
in these latter systems is to encourage administrators to give a higher precedence to
alarms than alerts.

The actual reporting mechanism varies from system to system and in different
organizations. For example, one IDS might write the event into a log as an alarm or
alert, and then send an email to an administrator account.

In a large network operations center (NOC), the IDS might send an alert to a
monitor easily viewable by all personnel in the NOC.
Intrusion Detection System Responses
An IDS will respond after detecting an attack, and the response can be either passive
or active.A passive response primarily consists of logging and notifying personnel,
whereas an active response also changes the environment to block the attack:

Passive IDS.
A passive IDS logs the attack and may also raise an alert to notify someone.
Most IDSs are passive by default. The notification can come in many forms, including
an
email, a text message, a pop-up window, or a notification on a central monitor.

Active IDS.
An active IDS logs and notifies personnel just as a passive IDS does, but it can also
change the environment to thwart or block the attack. For example, it can modify
access control lists (ACLs) on firewalls to block offending traffic, close processes on
a system that were caused by the attack, or divert the attack to a safe environment,
such as a honeypot or honeynet.

Sensor Placement for a Network IDS

If you are deploying a network IDS, you should decide in advance where to place the
monitoring sensors. This will depend significantly on what kind of intrusion or
attempted intrusion you are trying to detect. Start by creating a detailed network
diagram, if you don’t already have one.

A network Start by creating a detailed network diagram, if you don’t already have
one. A network diagram can be invaluable to IDS planning. When looking at the
diagram, evaluate key network choke points or collections of systems that are
sensitive to business operations. A well prepared diagram may provide intrinsic clues
to the right location for IDS sensors.

If the IDS is going to monitor a web server for penetrations, then the most useful
position for the sensor will be on the DMZ segment with the web server. This
assumes, of course, that your web server is in a DMZ segment, rather than outside or
inside the firewall (neither of which is a particularly good idea).

If attackers compromise the server, the IDS has the best chance of detecting either the
original penetration or the resulting activity originating from the compromised host.
If the IDS is going to monitor for intrusions targeting internal servers, such as DNS
servers or mail servers, the best place for a sensor is just inside the firewall on the
segment that connects the firewall to the internal network.

The logic behind this is that The logic behind this is that the firewall will prevent the
vast majority of attacks aimed at the organization, and that regular monitoring of
firewall logs will identify them. The IDS on the internal segment Will detect some of
those attacks that manage to get through the firewall. This is called “defense in depth.

Host integration for Host Intrusion Detection System

If you plan to use a host-based system, you should have an adequate testing and
familiarization phase. This allows the operators and analysts to become familiar with
the operation of that particular piece of software.

The IDS should be installed on a development system well in advance of planned

installation on a production system. Even on a quiescent system, some files will
change regularly (for example, the audit files), so the IDS will report some changes.

Some host-based systems, as an additional example, will report when a user process
alters the system password file. This would happen if an intruder added an account.

It also happens, however, when a user changes his or her password. The IDS analyst
needs time to become familiar with the correct operation of each system so that he or
she can properly diagnose deviations from “normal” alarms.

Keep in mind, when using a host-based system, that it should be monitored frequently.
This means twice a day, at least. If an attacker has superuser access to the system, he
or she can alter the IDS or the IDS database to suppress alarms.

If the IDS writes to If the IDS writes to a file, the attacker can simply edit the output
file. In other words, always be suspicious that something may have altered the
configuration of the IDS.

Alarm Configuration
IDS comes with configurable alarm levels. Some will integrate with network
management stations, some allow paging, some send e-mail, and some can
interoperates with firewalls to shut down all traffic from the network that originated
the attack.
Be very cautious about using these features. In fact, for the first month or two, turn off
all alarms.

Only look at the output from the system to see what it is detecting. All IDSs have, as
discussed above, some level of false positives; that level can be as high as 80 or 90
percent of reported alarms. You need to be familiar with your particular system before
you start turning on alarms.

Alarm misconfiguration, or over-aggressive response to alarms, can lead to an

organizational decision to turn off the IDS. Richard Marcinko, a former US Navy
SEAL, tells about throwing rabbits over fences into areas protected by motion sensors.

When the guard force got tired of responding to alarms (only to find rabbits munching
on the lawn), Marcinko’s team was free to pass through that area, knowing the alarms
would be ignored.

Hackers know that IDS installations are monitored by humans and that humans have
human failings. They know that if they trigger alarm after alarm after alarm, the
people monitoring the system will stop paying attention.

Likewise, if the IDS is configured to instruct the firewall to deny all traffic from
“attacking” networks, the hackers can easily exploit this.

Someone sufficiently motivated or malicious could use this against the organization
by spoofing attacks from the organization’s business partners or well-known sites
(Yahoo, AltaVista, Amazon, CNN, Microsoft, etc) so that the firewall will deny
inbound traffic from those sites, including e-mail and website traffic.

Remember, the IDS is not securities saving grace, it is only a tool (and a fairly
unintelligent one at that).

Integration Schedule
Install one sensor at a time. Don’t rush the installation in order to roll out the IDS
capability in a short time span. It takes a certain amount of time for the administrators

It takes a certain amount of time for the administrators and analysts to gain familiarity
with the peculiarities of a given system or network point, and the peculiarities may not
be the same from point to point.

A sensor in a DMZ may see a given set of behaviors, while a sensor on the internal
network may see another set of behaviors, with a very small intersection.
It is crucially important that the staff assigned to monitor the IDS be adequately
familiar with each device in the configuration.

Preparing the system with Intrusion Detection

System
Deciding on the placement of the Intrusion Detection System within the network is
critical. The IDS machine must connect to a port that can see all traffic between the
LAN and the Internet.

This means either connecting to a mirrored switch port or a hub located between the
Internet connection and the LAN. If a firewall and only one IDS sensor is used, the
sensor should be placed between the firewall and the LAN, for reasons that will be
discussed later.

Choosing the type of machine to use is dependent on the environment and the data
desired. A Snort IDS setup can involve one or several independent machines, or many
that report to a central database server. The faster the connection being monitored and
the level of logging dictate the machine capabilities.

For brevity, this article focuses on installing a single stand-alone IDS at the network
edge. For a Linux install, a desktop computer that is several years old should suffice.
Figure on a minimum of 256MB of RAM, a 20GB hard drive, a 600-MHz processor
and a CD drive, all features of desktop machines made within the past few years.

For installing a base Linux operating system, a machine to create the installation CD
is needed. A Windows box running Burn4Free (a freeware ISO burner) will work
fine. In addition, the network parameters (IP address and such) and a network
connection for the IDS machine should be determined prior to the Linux installation.

The needed applications

Snort essentially works on pattern matching by comparing packets to signatures of
known attacks. There are literally thousands of such signatures available.

Think of Snort as an intelligent sniffer: It takes a continuous trace of inbound and

outbound Internet traffic and analyzes the trace by comparing against the signature
database in real time. To do this manually would be impossible.

If a packet matches a pattern in a selected signature, an alert is generated. Analyzing

the alerts for meaningful data is no easy task, given the amount of data and its raw
format presentation.
Therefore, a method is needed to collect and provide for group analysis of the data.
This example uses MySQL as the database application, but Microsoft SQL Server or
Oracle may be used for the alert database as well. While populating a well-formatted
database with Snort information is necessary for categorizing information, as with
sniffer analysis, the process of analyzing such a database is labor-intensive.

This is where BASE comes into play. It’s a Web front end to the database that
presents the Snort alert data. This provides the information a network or security
administrator needs to identify threats and enact controls to reduce the threats.

Other support applications needed include the Apache Web server, the GCC compiler

and the PHP HTML scripting language.

Administering the Intrusion Detection System

After a successful installation, pointing a Web browser to the IDS will produce a
summary alert window.

From here, intrusion-detection data may be analyzed efficiently. BASE offers many
data aggregation and presentation tools.Each alert can be analyzed individually or as a
group.

The majority of the alerts generated constituted false positives because the alerts were
on regular traffic that may have had abnormal but perfectly harmless characteristics.

The IDS sensor should always be placed between the firewall and the LAN. Suppose
the alert in Figure 3 was indicative of a valid attack. The firewall could then be
configured to deny all traffic from that source address. No new alerts should be
logged after the firewall configuration, thereby effectively eliminating the threat.

Going forward
Building a functional IDS sensor is only the first step. Once installed, the IDS
administrator should spend a significant amount of time exploring the alerts and
capabilities of the system.

One doesn’t begin a major building project after setting up and operating a table saw
for the first time, and such is the case with Snort/BASE.

As threats emerge, rules must be added to the system to match the signatures of those
threats.
Snort offers a subscription service for access to emerging rules for a minimal fee or
free access to the same rules to registered users for 30 days after they are released to
the subscription service. Oinkmaster is an excellent tool for updating rules regularly.

In addition, signatures may be created manually, or pass options may be added to

signatures that are determined to produce an abundance of false positives.

Determining if alerts are in fact normal network traffic or an actual threat is obviously
necessary, as it would be foolish to disable a signature simply because it’s producing
many alerts.

Other open-source tools such as MRTG, ntop and tcpdump, in conjunction with server

and network equipment log analysis, can provide the data needed to streamline the
IDS configuration.

Snort can be deployed in a centrally managed distributed environment in which

multiple sensors report back to a single database server. In large enterprise networks,
this can be useful in correlating events as well as simply parsing information from
multiple points on the network.

It isn’t uncommon to deploy Snort sensors at borders between security zones in a

LAN, such as between administrative servers and local users.

A signature-based network IDS is simply a tool to enforce your company’s security

policy. Expecting that installing an IDS (or any single security solution, for that
matter) will eliminate all threats is flirting with a false sense of security. However,
delving into the world of open-source IDS is a path that can produce immediate and
significant returns.

Summary
An IDS is an essential part of a good network security architecture. IDS solutions have
their strengths and weaknesses, which must be measured and evaluated before you
decide to deploy one on your network. When viewed and implemented as part of a
network security fallback mechanism, an IDS is usually well worth the investment.