Security Assessment and

Testing
Components
• There are three main components

• Security Test

• Security Assessment

• Security Audit
 Security Testing Security Assessment Security Audit

• Verifies that a control is • It is comprehensive security review • Systematic Evaluations performed

functioning properly of a system, application or network with the purpose of demonstrating
• Predominantly are automated • Risk assessment is performed, and the effectiveness of controls to a
tests, while some tests may mitigation strategies are third-party
required human analysis recommended • It is performed by independent
• Automated scans, tool-assisted • Assessment includes Security auditors
Penetration Testing, manual Testing • Auditors provide an impartial,
attempts to break the controls • The output of the program is a unbiased view of the state of
• Performed by organization’s assessment report to management security controls.
security staff outlining the outcomes and the • The reports are intended for BoD,
• Results are meant for internal use recommendations government regulators, or other
only • Predominantly performed by third-parties.
• Designed to evaluate controls with organization’s security staff
an eye towards finding potential • Results are meant for internal use
improvements only
• Designed to evaluate controls with
an eye towards finding potential
improvements
Security Audit Types
• Two Types of Audits
Internal Audit External Audit

• Performed by an organizations internal staff • Performed by third-party auditors

• The reports are typically intended for • Reports are intended for third-party stake
Internal Audience holders
• Disadvantage is conflict of Interest and • They are unaware of internal dynamic and
hidden agenda politics hence they may not have any hidden
agendas
• Major disadvantage is the cost
• Sometimes lack of internal working
knowledge may translate to longer time to
get oriented and be able to perform the test
• Signing a NDA is a pre-requiste
Audit Strategy
• Establishing a clear set of goals is the most important step for
planning a security audit.
• Audit can be driven by the following factors
• Compliance requirements
• Significant changes to the architecture
• New developments in threat facing the organization
• The scope of the audit should be determined in coordination
with business unit managers
• The business unit managers should be included early in the
audit planning process and should be engaged throughout the
Aduit lifecycle
Audit process
Goal Conduct the Audit Documentation
• Determine the goal of the audit • Stick to the plan and document • Document the results
deviations • Key requirement that should start at
the planning process and continue all
the way to the results

Involve Stake holders Plan the Audit Communicate

• Bring in Business unit managers at • Ensure are all goals are met on time • Communicate to the right leaders in
the earliest stage possible and budget order to achieve and sustain a strong
• It ensures the needs of business are security posture
identified and addressed

Scope Audit Team

• Determine the scope of the • Choose the right audit team
assessment • Choose whether the team will consist
of internal or external personnel,
depending on the goals, scope,
budget and available expertise
Service Organization Controls (SOC)
• SAS 70 has got transformed in SSAE 16/
• Developed by American Institute of Certified Public Accountants
(AICPA)
• European equivalent is ISAE 3402
• Three kinds of SOC reports are there
• SOC 1 : Pertains to Financial controls
• SOC 2: Pertains to security, availability, confidentiality, Integrity, Process and
Privacy (collectively called Trust Services)
• SOC 3: Pertains to security, availability, confidentiality, Integrity, Process and
Privacy (collectively called Trust Services)
• SOC 2 provides detailed data related to the controls and is not
intended for public use
• SOC 3 is just a “Seal of approval” and does not contain the detailed
results, predominantly placed in service provider websites and
marketing collateral
Vulnerability Assessment / Testing
• Vulnerability Assessment must be done by professionals with deep security
experience and highest level of trust
• Before carrying VA, written authorization from management is necessary. This
protects the tester against prosecution
• Goals of VA are
• Evaluate the true security posture
• Identify as many vulnerabilities as possible
• Test how systems react to certain circumstances and attacks
• Vulnerability assessment results are “snapshot in time”
• Because they are point-in-time snapshot, these assessments should be done
regularly
• Low-priority, better-protected and less-at-risk environments can be scanned once
or twice an year
• High-priority, more vulnerable targets, should be scanned nearly continuously
• If automatic scans are used, it is recommended to use more than one tool.
• Scans should be run by different experts (time to time)
Vulnerability Assessment Types
• Personnel Testing
• Identifying vulnerabilities in standard employee practices,
demonstrating social engineering attacks
• Physical Testing
• Reviewing facility and perimeter protection mechanisms
• Performing physical security vulnerability assessment
• System and network testing
• 3 main categories are there
• Network discovery scan
• Network vulnerability assessment
• Web Application vulnerability scan
Network Discovery Scan
• Searching for systems with open ports
• They do not actually probe systems for vulnerabilities
• Some techniques
• TCP SYN Scanning:
• Sends a single packet to each scanned port with the SYN packet set
• If it receives a response back with SYN and ACK flags set, this indicates the port is open at the
sender end.
• This is also called as “half-open” scanning
• TCP Connect Scanning:
• Opens a full connection to remote system on the specified port
• Used when the user running the scan does not have necessary permission to run a half-open
scan
• TCP ACK Scanning
• Sends a packet with ACK flag set, indicating that it is part of an open connection
• Xmas Scanning
• Sends a packet with the FIN, PSH, URG flags set.
• The most common tool used for network discovery scan is nmap
Network Vulnerability Scan
• It goes deeper than the discovery scan
• They continue to probe the network for presence of known
vulnerabilities
• These tools contain a database of known vulnerabilities along with
the tests they can perform to identify these vulnerabilities
• Two common problems associated
• False-positive: reporting a vulnerability without have substantial evidence to
prove or reporting mistakenly. It is a nuisance
• False-negative: not identifying a vulnerability and failing to report it as part of
the results. It is a dangerous situation
• Authenticated scans help reduce the False positive/negative results
• Authenticated scans are performed with read-only access to the servers being
scanned.
Web Vulnerability Scan
• Special purpose scanners that analyze web applications for
known vulnerabilities
• They can discover vulnerabilities not visible to network
vulnerability scanners
• Its good to run these scans in the following circumstances
• Scan all applications for the first time
• Scan any new application before moving to production
• Scan any modified application before the code changes move to
production
• Scan all applications on a scheduled recurring basis
• PCI recommends web application scans are performed at least
once annually
Commonly exploited Vulnerabilities
• Problems that occur at the core of OS; attacker exploiting the vulnerability will have the
Kernel Flaw most powerful level of control
• Countermeasure: Ensure security patches are tested deployed and verified

• Buffer overrun due to improper bounds verification

Buffer Overflow • Countermeasure: Good programming language, developer education; automated source
code scanners

• Symbloic link is a stub file that redirects the access to another place; if attacker can
compromise the symbolic link, they may be able to gain unauthorized access
Symbolic Links • Countermeasure: programs/scripts must be written to full path to file cannot be
circumvented

• Numbers many OS use to represent open files in a process; certain file descriptor numbers
are universal, meaning same thing to all programs
File Descriptor attack • Countermeasure: Good programming language, developer education; automated source
code scanners and application security testing

• Exists when a design of a program puts it in a vulnerable condition before ensuring that
those vulnerabilities are mitigated
Race Condition • Counter measure: Good programming language, developer education; automated source
code scanners and application security testing

• Attacks rely on inappropriate access control of some part of the system on which a more
File and Directory permissions secure part of the system depends
• Countermeasure: File Integrity checkers
Penetration Testing
• Goes beyond vulnerability testing and actually tries to exploit the
system
• They require focused attention from trained security professionals
• Its goal is the measure an organizations’ level of resistance to an
attack and to uncover any weaknesses within the environment
• It emulates the same methods attackers would use
• The type of penetration testing should depend upon
• The organization, its security objectives, and the managements’ goals
• The result is a report given to management that describes the
vulnerabilities identified and the severity of those vulnerabilities. IT
also may provide mitigating strategies
• Its critical Senior Management is aware of this and have given
authority to do so
Penetration Testing Process

Discovery Enumeration Vulnerability Exploitation Reporting

Mapping

• Foot printing • Performing • Identifying • Attempting to • Reporting the

and gathering port scans vulnerabilities gain findings to
information and resource in the unauthorized Management
about target identification systems and access by
methods resources exploiting
vulnerabilities
Penetration/ Vulnerability testing Types
Black box testing White box testing Gray box testing
[Zero Knowledge] [Full Knowledge] [Partial knowledge]
• The tester has no prior • The tester has complete • Some Information about
knowledge of the internal knowledge of the internal internal working is given to
design or features of the system the tester.
system • Allows test team to target • It helps guide their tactics
• It simulates the external specific internal controls and towards areas we want to
attacker best features have thoroughly tested
• Disadvantage is that it will • It may yield a more complete • This approach mitigates the
probably not detect all result risks of the other two
vulnerabilities • It may not be representative models
• Another disadvantage is that of an external hacker
the testing team may
inadvertently impact
another system
Penetration tests
Blind Tests Double Blind Types Targeted

• The tester only has • Also known as stealth • Involves external and
publicly available data to assessment internal parties carrying
work with • It is a blind test to both the out a focused test on
• The network security team tester as well as the specific areas of interest
has prior knowledge of security team
this test to defend • It is used to evaluate the
security levels and
responses of the security
team
• It is a realistic
demonstration of the likely
success or failure of an
attack
Log Review
• Examination of system logs to detect security events or verify effectiveness
of security controls
• The most key requirement for effective log review is the synchronization
across all the log sources
• NTP is the protocol for time synchronization (UDP 123)
• NTP:
• Oldest protocol used in internet
• Time value is sent in a UDP datagram that carries a 64-bit timestamp on port 123
• It’s a client/server architecture, with hierarchical time sources organized into strata
• Stratum 0 is the most authoritative and consists of highly accurate time sources such
as atomic clocks, GPS
• Stratum 1 consists of primary time sources that are directly connected to stratum 0
• Stratum 2 are local network servers that an organization’s NTP server will connect
• Stratum 3 are other local servers and clients
• Nodes on the same stratum can communicate with each other to improve efficiency of
their times
Log Tampering Prevention
• Remote Logging:
• Putting a log file into another device will protect from tampering it in a
compromised system
• Simplex Communication:
• Using one way communication between the reporting devices and the central
log repository. Accomplished by severing the “receive” pairs on an Ethernet
cable.
• Data diode ~ physically ensuring one-way path
• Replication:
• Making multiple copies and keeping them in different locations
• Write-once media:
• Using write-once media to prevent unauthorized modifications to log files
• Cryptographic Hash:
• Powerful technique for ensuring unauthorized modifications are easily
noticed.
Synthetic Transactions
• Transactions that are initiated by an end-user is called real
transactions
• Automatic script based transaction with expected output is
called Synthetic Transaction
• They allow to systematically test the behavior and performance of
critical services
• They can help test new service mimicking end-user behaviors to
ensure systems work as it ought to
• This is an effective way of testing software from the outside
Synthetic Transactions Vs Real User Monitoring

Real User Monitoring Synthetic Transaction

Passive way to monitor real user interactions They help in ensuring the user does not get
with a web application or system dissatisfied or encounter a problem
It uses real users instead of scripted It is based on custom scripts mimicking user
commands behaviour
It more accurately captures the actual user They can detect rare occurrences more
experience reliably than waiting for user actions
It tends to produce noisy data and thus may Its very predictable and can be regular
require more back-end analysis because their behaviour is scripted
It lacks the elements of predictability and Synthetic transactions are run against test
regularity, which could mean that a problem code and the output is compared against
wont be detected during low utilization periods expected results, clearly showing mismatches
Use Case Testing
• Use case describes the sequence of actions between the user
and the system that result in an expected output
• Use cases are textual but are graphically represented using
Unified Modeling Language (UML)
• Use cases are related to one another in a variety of ways called
associations
• Including another use cases ~ the use case will be always executed
• Extending a use case ~ second use case may or may not be executed
depending on the decision point in the main use case
• Use cases are mainly helpful in determining the normal or
expected behavior of a system rather than in assessing its
security
Misuse Case Testing
• Misuse case is a use case that includes threat actors and the actions they want to
perform on a system
• Under UML, threat actors are represented as stick figures with shaded heads and their
actions are depicted as shaded ovals
• The misuse case is meant to threaten a specific portion or legitimate use case of our
system
• Misuse case testing helps to ensure we have effectively addressed each of the risks we
identified and decided to mitigate during risk assessment phase
• Misuse case doesn’t require to include all the possible threats to the system, but it
should include the ones that was decided to be addressed
• It is also referred to as abuse case testing
• They are used by software developers to evaluate the vulnerability of their software to
known risks
Code Reviews
• A systematic examination of instructions that comprise a piece of software,
performed by someone other than the author of that code
• It is the foundation of software assessment programs
• It is often also known as “peer reviews”
• It starts with the organization setting the coding standards to be followed
• The preliminary step to code review is to ensure the developer followed the
defined coding standard
• After this step, the reviewer shall check for unneeded functions or procedures
that may lead to “code bloat” ~ which makes it harder to maintain and secure
the application
• Defense programing is a best practice to be adopted by all software
development operations ~ constantly look for opportunities for things to go bad
Fagan Code Review Process
• Fagan inspection is the most formal code review process with 6 steps
• Planning
• Overview
• Preparation
• Inspection
• Rework
• Follow-up

• This level of formality is normally found only in highly restrictive environments

where code flaws may have catastrophic impact.
Testing Methods
• Static Testing
• Evaluates the security of software without running it
• Usually involves the use of automated tools designed to detect common software flaws, such as
Buffer overflows
• In mature development environments, developers are given access to static analysis tools and
use them throughout the design, build and test process
• Helps developers identify programming flaws and vulnerabilities.
• Static analysis can never reveal logical errors and design flaws
• Dynamic Testing
• Evaluates security of software in a runtime environment and is often the only option for
organizations deploying applications by someone else
• Testers do not often have access to source code
• Dynamic testing can involve the use of synthetic testing
• It is effective for compatibility testing, detecting memory leakages, and identifying dependencies,
and for analyzing software without having to access the software’s actual source code
Testing Methods
• Fuzz Testing
• Specialized dynamic testing technique that provides many different inputs to
software to stress its limits and find previously unknown flaws
• Two main categories of Fuzz Testing are
• Mutation (dumb) Fuzzing:
• Takes previous input values from actual operation of the software and manipulates it to
create fuzzed input. It might alter the characters of the content, append strings etc.
• ZZUF tool automates the process of mutation fuzzing
• Generational (intelligent) Fuzzing:
• Develops data models and creates new fuzzed input based on an understanding of the
types of data used by the program
Interface Testing
• An interface is an exchange point of data between the system/user
• Interface testing is a systematic evaluation of a given set of exchange points
• The testing should include known good and bad exchanges
• The primary task of interface testing is to build all the test cases ahead of time,
document them, and then insert them into a repeatable and automated test
engine.
• Interface testing is a special case of Integration testing ~ which is the
assessment of how different parts of a system interact with each other
Interface Testing Types
• Application Programming Interface (API)
• Offers a standard way for code modules to interact and may be exposed to outside world.
• Developers must test API to ensure they enforce all security requirements

• User Interface (UIs)

• Graphic User Interface and command-line interfaces that provide end-users with the ability
to interact with the software. The test should include reviews of all user interfaces to verify
that they function properly

• Physical Interfaces
• Exists in some applications that manipulate machinery, logic controllers etc.
• Testers should pay careful attention to physical interfaces because of the potential
consequences if they fail
Test Coverage Analysis
• It is practically impossible to complete test a software
• Testing professionals conduct Test Coverage Analysis to estimate the degree of
testing conducted against the new software.
• It is computed using the formula

Number of use cases tested

• Test coverage =
Total number of use cases

• This is a highly subjective calculation

Account Management

• Compromising privileged users of the system is the

preferred technique for attackers
• Three ways to accomplish:
• Compromise an existing privileged account

• Create a new privileged account

• Elevate the privileges of a regular user account

Adding accounts
• Every organization should, at minimum, have acceptable use policy that
specifies what the organization considers acceptable use of the IT resources
made available to the employees

• The AUP is the useful first line of defense

• Testing that all employees are aware of AUP and other applicable policies can
be the first step in auditing user accounts

• The AUP should also dictate the default expiration date of accounts, the
password policy, and the information to which a user should have access.
Modifying/Suspending accounts
• Accumulation of access privileges over the life time of an employee in the
organization results in privilege accumulation.

• This is a dangerous practice that gives the employee more privileges than
needed for performing their job function

• Another important practice in account management is to suspend accounts that

are no longer needed

• Account reconciliation is an important function that helps determine dormant

accounts.
Business Continuity Plan Testing
• BCP maintenance should be incorporated into change management procedure

• Tests and DR drills should be conducted atleast once a year

• The first exercise should not include all employees rather a small representative
sample of the organization

• People conducting the drills should expect to encounter problems and mistakes
BCP Drills
Checklist Test Structured walk-through Simulation Test Parallel Test Full-Interruption Test

• Copies of BCP/DR plan • Representatives from • This test takes a lot of • Some systems are • Most intrusive to
distributed to the each department come planning and resources moved to alternate site regular operations
different departments together and go over • All employees and processing takes • The original site is shut
for review the plan participating in place down and processing
• This ensures nothing is • The group reviews the operational and • The results are takes place at the
taken for granted or objective, scope, support functions come compared with the alternate site
omitted assumptions of the together to practice a regular processing • Recovery team fulfills
• Planning team plan specific scenario done at original site its obligations in
integrates all changes • The group walks- • It raises the awareness • Ensures specific preparing the systems
to the master plan through different level of the people systems can function and environments for
• It is also called desktop scenarios of the plan involved adequately at alternate the alternate site
or table top test from beginning to end • The drill shall include site during disaster • All processing is done
to make sure nothing is only those materials at alternate site
left out that will be available in • It should be performed
an actual disaster. only after all other tests
• The test continues upto are completed
the point where satisfactorily
physical migration to • Senior mgmt. approval
new facility gets is needed before
initiated performing this test
Security Training and Awareness
• Security training is the process of teaching a skill or set of skills that will allow
people to perform specific tasks better

• Security awareness is the process of exposing people to security issues so

that they may be able to recognize them and better respond to them

• The key measure of security awareness program is the degree to which the
users change their behaviors when presented with a certain situation
Social Engineering
• Process of manipulating individuals so that they perform actions that violate the security
policy

• Phishing – the most popular form of social engineering attack conducted through digital
communication

• Spear Phishing – a type of phishing attack that is targeted to a specific group or

individual

• Whaling – specific phishing attack targeting senior executives or individuals

• Drive by download – invisibly redirect the user to malicious distribution server; it is an

automatic attack that is triggered simply by visiting a malicious website.

• Pretexting – social engineering attack over phone

Key Performance Indicators (KPI)
• Process by which to measure the performance of security controls and processes

• ISO 27004 deals with KPI metrics

• Some key terms associated with KPI

• Factor: An attribute of the ISMS that can be described as a value that can change over time
• Eg: number of AV alerts or number of investigations conducted

• Measurement: the value of a factor at a particular point in time. This is the raw data
• Eg: 20 AV alerts per day or 15 investigations per month

• Baseline: An arbitrary value for a factor that provides a point of reference or denotes that some
condition is met by achieving some threshold value
• Eg: number of AV alerts per month will not be more than 25, or the number of investigations open for more than 48
hrs should not be more than 10
Key Performance Indicators (KPI)
• Some key terms associated with KPI
• Metric: A desired value that is generated by comparing various results with each other or
baseline
• Eg: ratio of false-positives AV alerts to valid alerts per month

• Indicator: An interpretation of one or more metrics that describes an element of the

effectiveness of the ISMS. Indicators are meaningful to management.

• KPIs should be easily understandable to both business and technical audience

and should be aligned with one or more organizational goals
Key Performance Indicators (KPI)
• KPI are driven by organizational goals.

• KPI process include

• Choose the factors that can show the state of our security

• Define baselines for some or all factors under consideration

• Develop a plan for periodically capturing the values of these factors

• Analyze and interpret the data

• Communicate the indicators to all stakeholders

Key Risk Indicators (KRI)
• KRI tells us where we are in relation to our risk appetite

• They measure how risky an activity is so that leadership can make informed
decisions about the activity

• KRIs are selected for their impact on the decisions of the senior leaders in the
organization

• It is useful to relate them to SLE equations

• KRIs alert us when something bad is likely to happen so that we can change our
behavior and defeat the threat
Technical Reporting
• The technical report should be the application of a standard methodology to the
specific system of study

• The raw data and automated reports should be provided in an appendix

• The key elements of a good technical report are

• Threats

• Vulnerabilities

• Probability of exploitation

• Impact

• Recommended actions
Executive Summary
• Translate the key findings and recommendations into language that is
approachable and meaningful to the senior leadership

• Goal is to get their attention and execute the desired change

• Multiple approaches can be used

• The Cost approach ~ looks at the cost or acquiring or replacing the asset
• The Income approach ~ considers the expected contribution of the asset to the company’s
revenue stream.
• The Market approach – Determine how much other firms are paying for a similar asset in the
marketplace. It requires a fair amount of transparency in terms of what other organizations
are doing.
Karthikeyan Dhayalan
MD & Chief Security Partner

www.cyintegriti.com