Nuevo Blueteam
Nuevo Blueteam
Nuevo Blueteam
Don Murdoch^
Blue Team Handbook Vol 2: SOC, SIEM,
and Threat Hunting Use Cases
Notesfrom the Field
Copyright © 2018 by Don Murdoch. AH rights reserved. Except as permitted under the United States
Copyright Act of 1976, no part of this publication may be reproduced or distributed in any form or by
any means, or stored in a database or retrieval system, without the prior written permission of the
publisher and author.
This book is available at special quantity discounts to use as premiums and sales promotions or for
use in academic and corporate training programs. Please contact the author through
www.blueteamhandbook.com.
AH trademarks are trademarks of their respective owners. Rather than put a trademark Symbol after
every occurrence of a trademarked ñame, we use ñames in an editorial fashion only, with no
intention of infringement of the trademark. Where such designations appear in this book, they have
been printed with initial Caps.
TERMS OF USE: This is a copyrighted work and its licensors reserve all rights in and to the work. Use
of this work is subject to these terms. Except as permitted underthe Copyright Act of 1976 and the
right to store and retrieve one copy of the work, you may not decompile, disassemble, reverse
engineer, reproduce, modify, create derivative works based upon, transmit, distribute, dissemínate,
sell, publish or sublicense the work or any part of it without prior consent from the author, secured
via paper letter with a blue ink signature. You may use the work for your own noncommercial and
personal use; any other use of the work is strictly prohibited. Your right to use the work may be
terminated if you fail to comply with these terms.
THE WORK IS PROVIDED "AS IS." The author does not warrant or guarantee that the functions
contained in the work will meet your requirements, that its operation will be uninterrupted or error
free, or that the work will qualify as an expert witness. The author shall not be Hable to you or anyone
else for any inaccuracy, error or omission, regardless of cause, in the work or for any damages
resulting therefrom. Under no circumstances shall the author be Hable for any indirect, incidental,
special, punitive, consequential, or similar damages that resultfrom the use of or inability to use the
work. This limitation of Hability shall apply to any claim or cause whatsoever whether such claim or
cause arises in contract, tort or otherwise.
If you would Hke to get in contact with the author or illustrator, please use the contact form on the
website at www.blueteamhandbook.com.
Art Notes: Art in the BTHb series is designed to be humorous and punny (yes, that IS a wordl). We
hope you enjoy it. If you would like to use the artist for your own books, presentations, or articles,
please use the contact form on www.blueteamhandbook.com.
ii
Table of Contents
Tabie of Conteras
Preface 7
Foreword 11
Introduction 13
Security Operation Center Field Notes 15
SOC Defined 15
SOC Charter 16
Business Valué Chain Tie In 16
Identify SOC Services 17
SOC Project Planning Outline and Field Notes (VI.02) 20
Consider, and be Prepared, forTough Questions 27
Collect the Bread and Butter Data Sources 29
Useful MBA Concepts: SWOT and PESTL 30
Funding the SOC 31
Getting into the Hunt 38
SOC Directly Supports the CSIRT Function 38
Metrics for the SOC 39
SOC Training, Skills, Staffing, and Roles 45
SOC Layered Operating Models 52
SOC Maturity Curve Using the CMMI 55
Example SOC Turnover Shift Check List 59
Security Monitoring Use Cases by Data Source 61
TheScenario 61
Defining the SOC Use Case 65
Organizational Considerations for Use Case Development 69
"Top Ten" Security Operations Use Cases 70
AntiSpam and Email Messaging 71
Email and Web: Interactions with Look a Like or Doppelganger Domains 73
Antivirus (A/V) Systems 74
Application Whitelisting 76
Command and Control 78
Data Loss Prevention (DLP) 78
Domain Ñame Services (DNS) 78
End Point Detection and Response 82
Data Islands or System Snowflakes 83
Windows Account Life Cycle Events (ALCE) 84
Monitoring Jump Boxes 92
NetWork Hardware Devices and Appliances 94
Printing 95
Operating System Security, Change, and Stability 96
Table of Contents
ii
Table of Contents
List of Tables
Table of Figures
v
Preface
Preface
With the ever-advancing adversary, technology advancements, and a critical
need for more skilled security operations practitioners, it is imperative for
organizations to enhance their PDR cycle: Protection, Detection, and Response.
This book attempts to answer that cali by sharing experiences gained
implementing fíve different SIEM technologies for more than a dozen
organizations, running a MSSP división, and building several security operations
centers.
Who this book isfor: IT pros, cyber security pros, security operations staff,
security consultants, SOC staff, SIEM designers and consultants, and line
managers: those responsible for protecting information assets and teaching the
next generation of security professionals.
About the Author: Don Murdoch, GSE #99, MBA, MSISE (GISF, GSEC, GCIH,
GCIA, GPPA, GMON, GCFE, GCFA, GCPM, GPEN, GSNA, GPPA, GCWN, GCUX,
TOGAF Enterprise Architect, SABSA Chartered Architect. CISSP, ISSAP), is a
thirty-year veteran of information technology, with more than half a career
devoted to information, Computer, and network security. Don started his career
with a boutique contracting firm located in eastern Virginia, writing COBOL and
FoxPro code. For the next twelve years, Don took on role in a different aspect of
IT as he grew his career: managing a small network, writing oíd school Perl/CGI
software, developing international billing software for a startup ISP, and
managing IT for an Application Service Provider right at the time of the Dot Com
bubble. Don transitioned into Information security where he started a DRP
practice for an IT commercial spin off from a televisión production company.
Things started cooking when Don entered his "digital cornbat training" phase in
the "Wild, Wild, West" of academic computing for one of Virginia's largest
institutions for higher learning. Don wrangled bots, tangled with well-equipped
adversarles, and discovered what today are described as nation State grade
attackers who were using the University network as a training ground. His
University was the first in Virginia to implement a SIEM, user managed anti-
spam technology, and an active countermeasures network based on Tom
Listons' Labrea TarPit. After that experience, Don took on managing SIEM,
conducting employee investigations, and security architecture for a Fortune 500
healthcare firm. That firm was acquired in 2012. His career took a security
hiatus for a few years when he was an Enterprise Security Architect and then
started running Infrastructure Strategy and Planning team for a Fortune 50
Corporation. In 2016, an opportunity to develop an MSSP practice carne up with
a different boutique Consulting firm. After two years, Don left to run the Cyber
Range at Regent University, where he is today coaching the next generation of
Cyber Defenders.
7
Preface
Don started working with the SANS Institute in 2002, taking courses, earning
certifications, developing Stay Sharp courses during the mid-2000's, and
currently teaches courses at the Community level. He earned the GIAC Security
Expert (GSE #99) certification in 2014, as was later vetted as a Cyber Guardian:
Blue Team in 2016 (#38).
8
Preface
• Martin Tremblay, GSE #80: Martin has 20 years of combined red and blue
team experience. He works for a leading international Consulting firm and is
based in Cañada.
Update Notes
Major updates are indicated with (VI#) in the section heading.
Versión 1.02: Expanded the project plan section beginning on 20, and in
particular the EDIS discussion based on request from a State InfoSec team.
Reviewed and added various Windows Event IDs*, corrected a few errors.
Updated the list of suspicious EXE's.
9
Foreword
Fo reword
The choirs sang and the trumpets blared as a joyous parade marched down the
avenue amid the blinding confetti thrown from the high-rise Windows above.
Sweet smells of cotton candy and funnel cakes permeated the air. The feeling of
triumph flowed through us all...at least it flowed through a much younger me. I
also may have been the only one who heard the marching bands and the angelic
choirs. And, even though my excitement was palatable, my role in it all was
merely tangential. But it was a turning point. Our SOC team's first (somewhat)
successful SIEM deployment. From the ashes of web-based syslog, convoluted
database exports to spreadsheets, and tools with ñames like ACID and BASE,
aróse the Colossus of Logs. From my little comer and my basic use case, I
frequently paid homage to this wonder of the computing world, mostly through
conducting searches that evolved over time, and crafting basic tools to help
analysts.
Those crack analysts were applying some techniques covered in this book, but
they didn't have a copy from which to work. At that time, our team spent
countless hours thinking about our data, analyzing it, and determining ways to
find evil. Our engineers built processes, tools, and dashboards so the SOC could
work more effectively and empower the júnior analysts. Over time, our SOC
innovated, building more focused and custom dashboards for múltiple use
cases, ingesting intel and other content, writing Scripts to pivot, and more, all to
improve the analysis process. That team and toolset continually improved and
never quite lived happily ever after, but l'll always remember how much my
career outlook changed after that first experience with a good team and a
decently implemented SIEM.
But, now that l've seen more SOCs and log management implementations than I
can count, both good and bad, I have since realized that maybe the choirs I
heard were a little fíat and that the trumpets might have been blaring
something other than fanfare. If our group only had a book like this at the time,
we would have used the SIEM with greater success. The most effective SOCs
with the most solid SIEM implementations got there through thoughtful
strategy and skill, with seasoned pros who had spent years in the fight. They
answered the right questions: Which Information is important? Which logs
produce that Information? Who needs it? Why? Encapsulated in this book is a
fifteen-year career building SOCs and implementing SIEM technology for finding
evil every day. Many of the thoughts in here come through in my work today.
11
Foreword
I met Don and many other thought leaders in our community at our annual
Security Onion Conference in 2017, where Don delivered a very well-received
talk on building security operations use cases. Don later asked me to be one of
the technical content reviewers because of my experience as a technician,
consultant, and leader. I was humbled and excited to particípate. As I read
through the draft, I found myself waxing nostalgic on my first SIEM deployment
in the early- to mid-2000s. I later commented to Don that it felt like the words
on the page were things l've been saying for a long time and were topics on
many Client engagements, but never written down in one place. In "BTHb: SOC,
SIEM, and Threat Hunting Use Cases", Don covers how to use security focused
data sources to their fullest, how to write a solid SOC focused Use Case, security
metrics you can actually use, and how to build engagement plans and practices
so you will be successful.
You might not hear choirs singing; but, if you're about to embark on the journey
of building a SOC and/or SIEM, whether to implement in a green field, to
valídate your position, or simply to improve your security posture and
capability, you have the right book in your hands.
12
Introduction
Introduction
This idea for this book actually predates the first book in the series, BTHb
Incident Response Edition. In 2011, our team needed to replace our commercial
SIEM platform. We headed down a path that lead to my fourth major SIEM
implementation. We needed an outline to develop use cases, document all of
the attributes of a use case and SOC procedures to fully use the new platform. I
wanted our chosen vendor to have the best possible chance of bidding on the
work and completing our use cases on time, and on budget. After vendor
selection, we engaged a major firm, and set about replacingthe legacy platform.
The vendor estimated they could achieve 26 to 28 use cases. After 12 weeks, we
exceeded project expectations. They implemented 35 of 37 fully defined use
cases that totaled 497 pages in print once the paperwork was done. The vendor
liked the use case témplate format that they adopted it, and they still use it
today. We added fourteen new right click integrations. We even had a custom
Ul extensión that pulled in a dozen account attributes for every user account
listed in an alert when we opened an alarm. Lastly, we also went from 1.5
people monitoring the prior solution to four full time analysts.
As a result of all that work, the idea for BTHb:SOCTH was born, and I started
collecting notes that eventually became the book you now hold. Along the way I
started a MSSP practice with a good friend working with a Consulting firm. We
won 78% of our POC's in year one, and had a 100% renewal rate during year
two, so several of those life lessons are incorporated herein.
This book will cover many topics related to the Security Operations Team from a
"Field Notes" perspective. It is based on a log career implementing múltiple
SIEM technologies, building SOC's, conducting all manner of cyber investigation,
developing and running an MSSP. The major topics are:
13
Introduction
It is my sincere hope that this self-published book delivers on the Blue Team
Handbook motto: "a zero-fluff reference guide for the security practitioner,
written with the intention of sharing real life experience". I trust that you will
learn something useful as you read it as many readers of BTHb:INRE have
shared with me over the years.
14
Security Operation Center Field Notes
In a nutshell, the SOC is the first Une ofdefense. This definition incorporales
several important strategies for a successful SOC. First, a SOC must be under a
single management and reporting structure so that it has a clear line of
authority, funding, reporting, and accountability. Second, a SOC must have
awareness of all aspects of both the business and the IT environment, from the
smallest workstation to the largest supercluster in the cloud. Third, a SOC need
to understand its area of operation (AO), how they will support the business,
monitor business applications, and infrastructure. These criteria must be
covered in the SOC charter. Fourth, SOC budget needs to be large enough to
continually invest in people and support cross training instead of super
sophisticated software. That concept leads to the fifth strategy: train and
encourage analysts to be calm, correctly interpret alerts and their supporting
data. This requires that SOC analysts are well trained.
One point deserves some elaboration. There are few different ways that the
SOC team can establish its AO. The SOC can use the IT General Controls
program, corporate policy/procedure, guidance from standards like the ISO
2700X series, or follow the Center for Internet Security's 20 critical Controls.
When designing, building, staffing, and operating a SOC, you need to develop a
charter and mission
statement.
15
Security Operation Center Field Notes
and be able to determine if that activity presents a significant enough risk to the
organization that the activity needs to be effectively dealt with. SOC teams
don't solve security issues with complex SIEM software. They solve it with
knowledge, skill, and ability. Complex SIEM tools help - but they are not a
technological panacea.
SOC Charter
Every security operations center needs a "charter". The SOC charter defines
how the SOC serves the business, mandate(s), and define governance and
operational rules, what its areas of operation are, and how the organization
needs to respond to the alarm conditions and monitoring the SOC performs.
Note that the SOC charter is not the same as a project charter. The SOC Project
Implementation Charter is the formal document that authorizes the project to
develop a SOC, possibly implement a SIEM, and empowers the project manager
to apply resources and create the SOC.
The SOC charter is often developed in tándem with the SOC/SIEM project
implementation charter. The SOC charter should be scoped properly, whereas
an implementation charter is a Project Management Institute (PMI) defined
document. The term comes from the Project Management Body of Knowledge
(PMBOK) as a type "project artifact". Don't get the two confused.
In order for the SOC, and IT in general, to be relevant to and communicate with
the business, they must understand how the business speaks, and the
businesses' context and concept of operations1. Formally, a valué chain should
create some form of competitive advantage in the marketplace.
1 These concepts are well defined in the Sherwood Applied Business Security Architecture
(SABSA.)
16
Security Operation Center Field Notes
Monitor Security Posture: This is the primary role of the SOC: monitoring the
environment for security conditions, alarms, health of the security platform, and
responding through the organizations various technical solution(s).
17
Security Operation Center Field Notes
freelance or make up these points on the fly- plan ahead. To start planning,
review the application inventory and determine if IR support can be handled
internally or if a third party needs to be engaged. Once you have planned,
exercise your plan at least twice a year using a tabletop exercise format. Once
that is stabilized, intégrate various real data or activity components into testing
the IR plan. Then gradúate to engaging an externa! pen test team, outline an
engagement structure, and put the blue team to the test.
Malware Analysis: If a SOC analyst can safely recover a malware sample, then
they may be inclined to perform some lightweight malware analysis using
Services like VirusTotal, JoeSandbox, or ThreatExpert. That advice was useful in
ten years ago, and is no longer considered best practice. In 2017, the better
course of action is to run samples through a local malware analysis engine built
on Cuckoo sandbox to prevent informing the attacker, who is Hkely monitoring
online Services, that their malware wasfound. These tools allow a user to
upload a suspect binary and then advise if it is known bad and provide varying
18
Security Operation Center Field Notes
levels of activity analysis such as registry changes, new Services, file system
changes, IP addresses in use, ordomain ñames looked up. If the analysis reveáis
something suspicious, then the SOC analyst would take that operational
intelligence and be able to better search security data. More complex reverse
engineering beyond this cursory level is a very specialized skill and requires
environment setup for this purpose.
19
Security Operation Center Field Notes
encapsulated into reusable code. How quickly does a new exploit appear after a
vulnerability is announced in a technology you depend on? By keeping aware of
attack tools, vendor announcements, and postings from major vendors from the
IR community such as SANS, TrustWave, IANS, FireEye, CrowdStrike, AlienVault,
and EMC/RSA, you can build a very low-cost CTI program and then make a
purchase decisión.
20
Security Operation Center Field Notes
Develop key business focused understanding of the organization and how the
SOC can support its goals and objectives.
1. Understand the organizational need for a SOC, which means that you need
to understand your organization's goals and objectives. By being able to
articúlate how the SOC protects what the organization produces, sells, or
the Services provided to others, the SOC will have more credibility, be
relevant to the business, and support your organization's mission
statement.
2. Understand the business problem(s) the SOC needs to address and valué
chain resources that the SOC needs to monitor. You may need more of a
"compliance" focused SOC, a tactical SOC, an Incident focused SOC, or some
combination of these. The SOC will monitor several components of the
valué chain in addition to general IT resources. The SOC that intelligently
targets the valué chain for monitoring will be more successful and relevant
to the business.
3. Identify the SOC sponsor. The sponsor may have an uphill struggle to
initiate, build, and deploy the SOC. The SOC manager must be sure that the
sponsor relationship is well maintained. The "customer" should want SOC
Services, and not have them dumped in their lap. The other operational
roles will need to be well staffed. Evaluators and regulators are examples of
"external stakeholders". These roles will be staffed by auditors with varying
skill levels who are attempting to measure and report on risk and the
degree of compliance within the organization. Understanding the questions
stakeholders are likely to ask will inform use cases, reporting, and data
sources that should be implemented to report to the SIEM platform.
4. Ensure there is an actual need for a SOC and its supporting logging
infrastructure. Be ready to articúlate that need, and explain how the staff
and technical capabilities meet the need. Here, you should develop a formal
business case. Be prepared to justify the staff, resources, access, and
software needed to build a SOC.
5. Develop key "Security State" understanding (the "as is" versus the "to be"
State). This understanding is technical in nature and corresponds to various
use cases and monitoring needs from the traditional IT perspective.
Wherever possible, connect a security State monitoring capability with a
valué chain component and the IT General Controls program. Refer to the
21
Security Operation Center Field Notes
most applicable standard for your industry, such as the ISO 27002. See page
245 for more information.)
Build your initial business case, charter, project plan, budget request, and
justification to support buildingthe SOC.
This process will likely be two to eight months' worth of effort. Design the
phases, identify the key inputs and outputs per phase per the PMBOK, and who
will support each project phase.
3 Customers and users. Customers are the persons or organizations who will approve and manage
the project's product, Service, or result (from PMBOK V5).
4 Sponsor. A sponsor is the person or group who provides resources and support for the project
and is accountable for enabling success (from the PMBOK V5).
5 Stakeholder: an individual, group, or organization who may affect, be affected by, or perceive
itself to be affected by a decisión, activity, or outcome of a project (PMBOK V5).
6 These are ITIL V3 Change Management terms.
22
Security Operation Center Field Notes
7 The steps are nearly the same done in the Business Impact Analysis (BIA) phase of a traditional
Business Continuity Plan (BCP), and then the Disaster Recovery Plan (DRP). If your organization as
a BCP, DRP, or TOGAF7 style EA team, then consult with them for the application and server
inventory
23
Security Operation Center Field Notes
only" credentials for the systems that house this Information such as a
Configuration Management Data Base (CMDB).
From a Project Management perspective, the major steps for the EDIS process
are outlined below.
For each of the identified data sources, you will need these planning and
implementation elements:
24
Security Operation Center Field Notes
1. Determine how the data source will be collected. Consult SIEM Data
Collection Methods and Considerations on page 207 for more information
on SIEM data collection methods.
2. Review the current auditing and logging configuration for fit.
3. Estímate to the extent possible the volume of data. For this point, try to get
an average daily volume over at least a five-week period, which should
catch any surges that naturally occur across a month boundary.
4. Determine if data can be trimmed, meaning review the data to find out if
there are low to no valué records provided by the data source that can be
pruned or dropped either at the collection point or the arrival point.
5. Inventory the data fields from the data source, and develop an internal SOC
training program so that all SOC staff understand the data source.
6. As needed, implement the necessary change control to configure the data
source to report to the SIEM.
Plan the Technology provisioning process to support the SIEM, and another
identified SOC Services (see p. 17). Plan for twice the data you think you will
need in year one.
25
Security Operation Center Field Notes
Build your log architecture, source data collection delivery, and SIEM and
logging deployment plan.
1. Perform software and vendor selection based on a scoring model built from
use cases that correlate to your business model, unique data sources,
compliance requirements, and InfoSec program.
2. Review the auditing stance and build out the Events Per Second (EPS) rating
for each of the systems in the environment that will provide data. Then plan
for a 50% increase so that your solution can weather an "event storm". Its
critical to determine the EPS after the source system has had its auditing
level configured!
3. Provisión the hardware and storage platform and implement.
4. Monitor your data feeds, reporting, and system response time.
5. Build your data integration plan for commodity sources, and carefully select
customized sources. For example, an ERP application is not likely to be
supported, so you will likely need to develop a database query to pulí data
from the audit table, implement auditing, test, develop a method to archive
current data to a historical table, and monitor to ensure that the query
process has minimal system impact.
1. There is an entire chapter in this book on building out use cases. Review all
of that material and compare it against the use cases in your chosen
platform.
2. Plan how to implement the vendor-defined use cases as these should
provide baseline coverage.
3. Forecast the effort required and data sources to implement your own
custom use case.
4. From there, prioritize the implementation so that you will have project
measurement that supports defining earned valué.
26
Security Operation Center Field Notes
Build your SOC Metrics, as defined in Metrics for the SOC on page 39.
Many technical platforms have reporting and measurement that supports SOC
and SIEM metrics. There are many organizational metrics that need to be
developed and collected. This aspect of developing a SOC and SIEM
implementation plan will evolve over time.
Training is a constant. SOC skills need to advance as the attacker's skill and
determination advance. Ensure that there is budget for at least two tiers of
education. Provide premium education for the more sénior tier, and then
develop OTJ training for the júnior tier. OTJ should consist of knowledge
transfer, short course, and job skills focused education. Investígate your local
community college work forcé education program and capabilities in area.
There are several open source or very low-cost options. Consider ENISA,
SecurityTube, SANS Cyber Aces, local BSides conferences, DerbyCon, and the
annual Security Onion conference as more inexpensive education options.
27
Security Operation Center Field Notes
1. Nothing has happened yet. Why do we need to do this? How can you be
sure that nothing has happened yet? As a possible answer, try these out:
"It's not if, it's when."
2. How will the team detect and respond to security issues, incidents, and data
breaches? How did we do this before? Isn't that what the sysadmins do?
3. Did the organization incur any costs from an incident last year? Virus
outbreaks? What costs incurred from our peers and competitors?
4. How many users at what "level" were negatively affected (as in lost
productivity) from an incident?
5. How are you going to measure yourselves and get on the IT Balanced
Scorecard? As a possible approach, ask if you can be on the business
scorecard during the SOC charter development process.
6. How will the team determine what alarm conditions are prioritized over
something eise - who wins? (Hint: asset valué tied to critica! business
process and revenue stream protection).
7. I thought we spent X on Security last year. Why do you want more?
8. I know We don't have anything worth stealing. Why do we need to do more
and more of this security "stuff"?
9. Don't those things cost millions?
10. We are doing vulnerability assessments. Isn't that enough? If you are not
doing active and timely remediation, then no, it isn't.
11. Those security people keep saying no, so l'm going to say no to them this
time. So there.
12. We can successfully outsource that for 1/8 the cost, right? After all, that's
what the vendors say. Why do you disagree with them? Aren't they
experts?
13. How will this SOC solve business problems for us?
14. What does this SOC thing look like year 1, year 3, and year 5?
15. I don't want to buy more expensíve security people only have them quit.
What are you going to do about staff retention? Burnout? Attrition? Internal
transfer? I recently heard at a security conference when people take a SOC
job they plan to quit in 18 months.
16. How will you know when you have had a success? What does success and
failure look like for a SOC? (or a major security purchase?)
17. Have you been talking to the auditors again? They said something about this
last year. I bought a new firewall.
18. Show me a playbook first - can you do that? Come back when that's done.
19. IT Is outsourced, it is "company X's" responsibility, not ours. We have no
liability because that rests with the outsourced vender and it's in the
cloud/vendor contract8.
8 I would encourage not to blurt out in response to this question that that is "A Guaranteed
Orange Suit Acceptance Posture". It does not go over well.
28
Security Operation Center Field Notes
20. What can you do with a third of that? Because that's all we have.
21. We spent $3.5M on SOX last year. No more!
29
Security Operation Center Field Notes
Once you add your own "must have's" to this list, your next task is to get the
daily data volume for each source. Volume has three factors: events per day,
average event width, and the typical peak or spike times. From there, you can
estímate the capacity you will need for your platform.
SWOT Analysis
SWOT is a strategic planning technique used to help an organization identify the
Strengths, Weaknesses, Opportunities, and Threats that every manager should
understand, and be prepared lo use in strategic planning exercises. Building a
SOC is an internal business ventare, which is affected by both infernal and
external pressures. SWOT analysis will improve your business case for your
SOC, will also help you plan, and ifdone well can help identify adversarles that
will launch attacks against the organization. Below is a very brief example to
give you an ¡dea what a SWOT analysis for a SOC project could look like.
30
Security Operation Center Field Notes
PESTL Analysis
PESTL (Political, Economic, Socio-cultural, Technological, and Legal) analysis is a
framework of the macroeconomic and macro environmental factors pushing
against the organization. It is used by strategic management, marketing, and
business development teams who need a solid understanding of how the
organization will perform given a particular business venture. PESTL analysis will
help SOC planning along two dimensions: the change and pace of technology
that the organization consumes or produces, and the legislativo or regulatory
environment where the organization operates and has a presence that requires
monitoring.
Understand the App Stack. The SOC leadership team needs to understand how
the organization is funded by its valué chain, and in turn how much of the IT
spend that SOC may receive in both Capital Expense (CapEx) and Operational
Expense (OpEx). Practically, this means that your team needs an inventory of
the business applications by their criticality, and then a map of the servers,
database(s), storage, and network connections that enable and support these
applications. With this model in mind, you can then work to align your
monitoring Controls, capabilities, and instrumentation to support the
availability, integrity, and security of those applications. Your Disaster Recovery
Planning / Business Continuity Planning (DRP/BCP) team can be highly valuable
as they have a criticality-based view of the application stack. If you don't have a
DRP/BCP team, the build the app inventory yourself with the intention that the
work can be used for SecOps, as it will assist IT when there be a need for a
recovery event.
31
Security Operation Center Field Notes
Finalize the Reasons to Fund SecOps: There are many, many reasons to fund
security operations and the logging infrastructure that SOCs and SIEM platforms
require.
32
Security Operation Center Field Notes
Indirect Costs:
Base pay, benefits, and the inevitable staff turnover disrupts the cost model.
The US Bureau of Labor Statistics lists Information Security Analyst base pay at
$92,500 in 2016, and $95,510 in 2017. If your internal overhead load is 30% for
administrativo costs, a loaded position costs $124,163/year. At this rate that is
$620,815 for five people per year in 2017 dollars. A 2016 study published on
glassdoor.com can help to understand the hiring climate: Companies spend
$4,000 to fill a position through open recruitment with a 52-day vacancy period,
33
Security Operation Center Field Notes
The staffing cost is further influenced by the coverage model. If the team
operates with 24/7/365, that requires 4.52 people, or 5 FTE's at a bare
mínimum to staff the SOC with a single person staffing the fácility. A lonely job
indeed. This valué is based on 8,760 hours per year, two weeks paid time off and
8 holidays which yields 48.4 work weeks, or 1936 hours of coverage per person.
In reality, any 24-hour operations team should plan on at least nine people to
accommodate vacations, sick time, and staff turnover. Five people will cover the
shifts, and the remaining three will provide additional coverage during high
activity hours, such as 7AM to 7PM and some portion of Saturday. Missing from
this estímate is the percentage of "admin" time. Admin time consists of all other
company required tasking that detracts from conducting actual heads down job
duties.
Incident response has a very high burn out rate compared to other technical
professions. Therefore, to compénsate rotate your SOC front line through
different SOC Services so that they have variance in job duties. Also, look for
analysts that aspire to move up and not stay doing the same thing every day.
SOC managers should always evalúate opportunities to vary the job duties in
orderto retain people.
Facility/Space: Most organizations have a per square foot rate for office space
that should be in the cost structure. As an example, the average 2016 cost per
square foot in Atlanta, Georgia was $20.01 for Class A space and $16.36 for
Class B space10. If you assume 90 square feet for shared workgroup areas with
two workspaces available for a five-person team on rotation, the monthly office
space cost is $2,944.80 for a two-person SOC. There are other single purchase
costs, however. For example, you may want two large format monitors and PC's
to run them with everything mounted on the wall. That could easily be a $4,000
single event cost ítem.
34
Security Operation Center Field Notes
Vendor Neutral Formal Education: Assuming you can find security people, you
will still likely need to train them in SOC operations. As of August 2018, one of
the best courses from SANS Institute is "SEC511:
Continuous Monitoring and Security Operations" with its
corresponding GIAC GMON certification. This course covers
the practical skills needed for every SOC staff member. I can
attest that there is a measurable improvement to the
quality and speed of each analyst who completed this
course and the corresponding verification. The August 2018
cost weighs in at $6,939 USD. Also, ensure thatthe travel and hotel costs are
included when looking at the cost of training, and estímate $180011. Stay for Day
Six and compete for the SEC511 coin12!
Product Training: Every major SIEM vendor has a series of product training
course. These are usually included in the initial proposal and implementation.
There will be a cost for new staff as they come onboard. To defray the cost, l've
had success by asking for a training credit with an upgrade, system
enhancement activity, or adding in new product component.
Desktops: Analyst hardware, monitors (the more, the merrierl), monitor arms,
client-side software, analyst licenses for dozens of applications, furniture, and
lighting. Think Quad 24" or27" displays, and an Ergotron type quad arm. Nice!
Vendor Support: Dedicated vendor support for the security product suite, use
case implementation, and continual upgrade processes. These support
relationships are usually priced on a per hour basis with a minimum number of
hours per week. If your SIEM vendor charges $225/hour and sells this support
arrangement with a minimum of 4 hours, the annual support cost is $46,800.
111 have had good luck shaving a bit by staying in the hotel next door.
12 Coin Image provided by the SEC 511 course authors and is used with permission.
13 https://www.td.org/newsletters/learning-circuits/time-to-develop-one-hour-of-training-2009
35
Security Operation Center Field Notes
at 3-4 months before your server maintenance period to ensure you are not
paying a premium for keeping a server online outside of its maintenance
window. If you need six servers at $8,000 each, that's an initial capital outlay of
$48,000 just for the hardware. Plan for 20% annual maintenance, and
technology refresh atthe four-year mark. Most of the SIEM implementations
l've done were virtualized using VMware 5 and 6. This model can be quite
successful - but you will need to add in the cost for the Hypervisor. When it
comes to actual capacity, spec 2 more CPU's and 4 more GB of memory that you
think you need and virtualize your platform on just that server. The long-run
benefits you will gain are tremendous. These include volume snapshots, copy
over to the new platform during tech refresh, and the ability to more easily mix
and match drive configuration.
Integrating new data sources: To minimize impact as new systems are brought
online, incorpórate the SOC engineering staff in the IT provisioning process. The
objective is to ensure that new systems or major system updates can provide
relevant log data to the SIEM/SOC team. This labor charge should be assigned to
the application or system, not SecOps, and is a recurring cost ítem for the Ufe of
the SIEM.
Content Development: Developing new and refining current use cases within
the SIEM solution. Current use cases will also be updated based on
improvements from the threat hunting team. The better you define the input
data, content needed, analysis, rules, notification, SOC actions, and outcome
desired, the more accurate a cost you will have and the higherthe opportunity
for success.
SIEM Software: SIEM platform licenses are most often driven by a sizing factor.
Typical factors are the "ingest" rate in events per second (EPS), GB per day, or
monitored device counts. You will find there is a class of non-security relevant
events that arrive at the SIEM along with the data that's really needed.
Depending on numerous factors (event cost, Processing horsepower, log
storage, event width) you may want to develop a tiered logging method. Costs
can vary widely here, from $20,000 per year and on up. The better you define
your environment, the better an estímate you will get from a vendor.
36
Security Operation Center Field Notes
Audit Evidence Support: The SOC is often asked to support reporting on security
event data and incidents in direct support of an intemal or external audit.
Staffing this specific role is closely related to the regulatory environment and
how often auditors make requests. To estímate this, determine how many
audits your organization responds to per year and the reporting needed to
support those audits. The SOC team should always record their time to support
audits, as this is a Service to the business. If you have recurring audits tied to a
particular unit, then ask for accounting charge codes to document costs for the
SOC to support operating units.
1. Startup time will have an impact. MSSPs in effect deploy a partial to full
SIEM solution on your network. Each data source needs to be integrated
into the platform, hardware will need to be deployed, and your organization
will still need to define your own incident response process.
2. An MSSP will only be able to go just so far when investigating alerts. If you
are fortúnate, the MSSP can cover 50% to 70% of the alarm conditions well
and will engage your organization on 15% of the observed alerts.
3. MSSPs will never know your network like you do, and you cannot easily
quantify this impact to their quality of Service delivery. MSSPs also are
unlikely to know what changed on the network, as they rarely particípate in
change control.
4. MSSPs work with you through a defined SLA and reporting relationship.
They cannot replace your own staff who can reach out directly to a system
custodian - this is an invaluable benefit to having in-house staff functioning
in a security operations role.
5. Their opinión on alarm sensitivity and configuration /$ not your opinión
because they tend to look at "genuine threat" conditions, and will ignore or
tune out many other conditions. Your use of SOC should inelude policy
issues, threat hunting, audit reporting, and gleaníng operational valué from
the mountain of data the SIEM will consume.
6. There are some tasks that should be outsourced ifthey are infrequent, like
system forensic analysis. However, the battlespace of today tells us that we
need a memory image more than a disk image. This is nigh impossible for a
third party outsource MSSP to collect but may within their ability to analyze.
37
Security Operation Center Field Notes
7. You cannot delegate responsibility to a third party for the security of the
assets under your organizations care, no matter how much someone tries to
convince you that you can. You may be able to delegate authority to
opérate, but not the responsibility of system security.
8. 7/24/365 monitoring by a third party will cost you less for the labor
component than building your own 6-8-person team. There is no getting
around that fact. If you you are being pressured to outsource, realize this
argument and devise ways to respond to the argument.
9. MSSP's may also be able to perform system upgrades and very likely have
done more upgrades than you. Factor in the cost of your deployment a
week or two to perform an annual upgrade.
10. Lastly, you get out of a MSSP relationship what you put into an MSSP
relationship. If you do not invest time, then don't assume that they will give
you stellar resuits.
The "reactive or detective only model or posture" from the 2000's is no longer
effective today. Today's SOC teams need to change their focus, assume that
there is a likely compromiso, become detection oriented, and proactively mine
the vast amounts of data coming into their systems and actively look for
patterns of intrusión and misbehavior. Proactive threat hunting is an ideal
career and skill development path for SOC analysts. Once they understand all of
the organization's data sources, know how to handle alerts, and demónstrate
that they have established research skills, capitalizo on that and get them
involved in threat hunting. Depending on how the SOC team operates, you
couid have a SOC analyst can perform hunting one day a week, one week per
month, or take a particular hunt pathway. Threat hunting is further defined on
page 171, with numerous use cases beginning on page 61.
38
Security Operation Center Field Notes
doesn't it? Today's cybercriminal will exploit any weakness they find to extract
untraceable digital crypto currency from any potential victim. Regardless of the
degree of sensationalism, there are several reasons to advócate for and build a
CSIRT function in your organization, which is in turn supported by the SOC
function.
1. The SOC provides an active detection capability that should enable early
response and limit the long-term impact from an incident. The CSIRT can
then coordínate responding to an incident with the goal of identifying,
containing, and reducing incident impact. Further, the CSIRT function can
return Indicators of Compromise or loC's back to the SOC so that the SOC
can perform historical data analysis, such as searchingfor internaI systems
that communicated with a found IP or domain ñame. Thus, a more mature
CSIRT and SOC team can capitalize on a Threat Hunting capability in order to
seek out and find a malicious agent that was recently on the network.
2. The CSIRT influences, supports, and fully leverages the security spend. It
helps to ensure that the SOC tooling is in place will support incident
response and Security Operations. Or put a different way, having a single
CSIRT should ensure that the best tool for the environment is in place, can
be leveraged, and others are decommissioned in order to maximize the
dollar investment.
3. Maintain objectivity when interacting with ¡nternal staff, classifying
incidents, and prioritizing the response process. One of the challenges that
a CSIRT will need to deal with is staff relationships and maintaining
objectivity. Like the HR function, which is charged with enforcing company
policy and procedure in a uniform and consistent manner, the CSIRT needs
to be objective, perform its work to protect the business, and avoid playing
favorites.
4. Lastly, regulation. There are numerous aspects of the business environment
that mándate an incident response capability. These include, but are not
limited to: HIPAA/HITECH, PCI DSS 3.2, and Sarbanes Oxley compliance.
39
Security Operation Center Field Notes
In the book Pra^matic Security Metrics, W. Krag Brotby and Gary Hinson make
several key points about metrics. Some of their key definitions are Usted here
from chapter 1.6:
In Section 2.6, they State "Having valid metrics enables business managers to
make rational, sensible, and, for that matter, defensible decisions about
information security. No longer mustthey rely entirely on advice from
information security professionals or generic good practice standards, laws, and
regulations."
There are numerous metrics and measurements that can be developed and
applied to a SOC. When it comes to designing a metric, there are several criteria.
1. Metrics should be relevant to the goals, objectives, and the mission of your
SOC as a business unit. This means you should be able to describe what you
do numerically, and also explain what you don't measure.
2. As you evalúate your data, security use cases, and metrics be sure to
develop a roadmap of what you will measure, how those measurements will
be captured, the tie in to the IT General Controls and security program, and
the business valué chain where possible. Metrics can then provide guidance
for decisión making.
3. Be sure that what you are measuring will drive towards a a measurable
outcome. Don't measure for the sake of measuring. Every metric should
inform a consumer and seek to either to change behavior or demónstrate
40
Security Operation Center Field Notes
41
Security Operation Center Field Notes
42
Security Operation Center Field Notes
43
Security Operation Center Field Notes
Incident Response Metrics are not the same as SOC metrics because they pick
up where alarm processing ends in many cases. In other words, when the SOC
identifies a true incident, they will turn that overto an Incident response
function.
44
Security Operation Center Field Notes
1. Product skills: These are actual vendor solution training. This is achieved
through vendor offered on the SIEM platform itself or a major technology
vendor such as the Cisco CCNA Cyber Ops program, because that course
material targets the actual products in use.
45
Security Operation Center Field Notes
2. Vendor neutral skill development: These skills should cover the job role,
tasking, and concepts that the analyst needs to do the job regardless ofthe
technology platform. There is no shortage of education available such as:
college courses, certification courses like EC-Council Certified Security
Analyst, ISACA's Certified Information Security Manager, CompTIA
Network+, Security+,
and CompTIA
Advanced Security
Practitioner; SOC
focused training by
SANS; CyberAces; and
various Q courses
offered by Security
University. For readers
in Europe, look into
European Information
Technologies
Certification Academy
(EITCA) and European
Union Agency for
Network and Information Security (ENISA).
3. On the Job: Intemal training relevant to the position and team, internally
developed and delivered.
4. Success and Failure: there's nothing quite like "getting it right" and stopping
the bad guy or missing the bad guy and finding out afterwards.
5. Cyber Range Operator training: intensive exercises offered on a dedicated
large-scale lab with a focus on hands on skill development.
6. And many more.
Your training program should include a mix of ítems from the Ítems above, with
the goals of ensuring that your analysts develop all of the skills listed in the next
section. As new staff are onboarded onto the SOC team, you should follow a
well-defined structure in order to ensure that they will have the best possible
opportunity to succeed. Below is a sample onboarding model for a SOC analyst
l've used in several organizations. The primary goal of your orientation program
should be to develop the analyst so that they can be self-sufficient at their
particular level after four to five weeks through an onboarding program.
46
Security Operation Center Field Notes
Analysts also need to learn how to efficiently preserve case relevant information
as they work through an alarm investigation. An effective technique is to
capture relevant data while they write a ticket or draft an incident report,
instead of attempting to reconstruct data after an incident is over.
Today, in order to address the skills gap, find people who are: naturally curious,
can think abstractly, often took things apart and put them back to together
during their formative years, have a stronq attention to detail, can "connect the
dots", and lastly who can perform research.
47
Security Operation Center Field Notes
Below is an Analyst Skill Development Recipe which carne from surveying over
30 SOC and Security Managers during Ql/2017, and then refined as I
implemented that advice. SOC Analysts need to understand:
1. The "Attack" process and phases: Recon, Sean, Initial compromise, Establish
Persistence, Command and Control, Lateral Movement, Target Attainment,
Act on Objectives, Exfiltration, CoveringTracks, Leave without Trace. The
MITRE ATT&CK framework is invaluable learning tool in this space as well as
the Cyber Kill Chain as described on page 184.
2. Ethics: Ethical behavior means that they work within their limits, ask for
assistance, do not overstate conclusions, never fabrícate an opinión on
weakfacts, keep confidential all of the data they use, and other professional
responsibilities.
3. Organization specific data familiarity: Familiarity with your organizations
source data (event types and fields) so analysts can tease out fact data that
can "connect the dots" to the alarm event in the overall context of the
event stream for the suspect machine or user. This is one ofthe hardest
skills to develop. As a new data source is added into a given system, engage
the EDIS process: the sénior level analyst must prepare an overview ofthe
data forthe remainder ofthe team.
4. System: Technical system access control capabilities and how a technical
control can be applied.
5. Firewall: Firewall Principies such as log types and what the log row records;
rule attributes; actions such as block, allow, reset, drop silently; interface
meaning as it relates to flow; SNAT/DNAT, byte sent/received; and
understanding security zones.
6. Security zones: In particular, security zones are specific to each
organization. One can infer the meaning of a zone, but should not because a
zone term may not be consistently applied by the various admins or
engineers. As firewall rule sets are built, the firewall team must document
what they mean by a given zone and their underlying assumption about
traffic flow. Active Directory admins need to do the same thing for
organizational unit definitions. In this way a firewall zone named
"eComDMZ" can be connected to an ADOU named "WebSalesDMZ" when
the definitions State "severs used to support web sites used for
ecommerce".
7. PCAP collection and analysis: NetWork data collection and the use of
tepdump as the data collection to and then WireShark/TShark as the
analysis tool.
8. Forensics: Forensic principies
a. Gathering data on system following the order of volatility
b. Establishing and maintaining Chain of Custody
48
Security Operation Center Field Notes
16 Q.uick UDP Internet Connections, a protocol promoted by Google that supports multiplexed
connections between endpoints and is supported by almost 1% of webservers (August 2018).
17 Chris Sanders has the only course the author knows of on this topic.
49
Security Operation Center Field Notes
50
Security Operation Center Field Notes
SOC Roles
There are severa! roles that need to be staffed in a security operations center.
Depending on the size, scope, and budget, a SOC may have more roles defined. Roles
will also have defined interactions with other key roles, because a SIEM platform and
NSM platform actually have user community - the SOC analysts - who are supported by
the engineering, architecture, and process support side of the SOC team.
51
Security Operation Center Field Notes
SOC Manager
Owns
Recruiting, budget, business 4— CISO
' SOC '
liaison, audit support, metric
design, architecture & change
mgmt
Usabilíty, UC &
SIEM Security
Tier N Content.
Engineer Process
issues Dev
Builds Use Engineer
Analyst Roles:
Ops, Alert Cases, research,
Mgmt, Intake troubleshooti intégrate threat
ng, upgrades, intel, designs
Tier 1 new data use cases and
feeds reports
Shift Lead
This section describes a two and three-layer approach, with the word "layer"
being used as a placeholder. Job titles may be Tier 1 Analyst, Tier 2, and Tier 3
Sénior Analyst, or titles like Júnior Analyst, Analyst, Sénior Analyst, Lead Analyst,
or SOC Shift Lead. Regardless of the actual title, the essential concept is that
there are layers that relate to the analysts' skill base, comfort level, and their
ability to respond to alarm and event conditions which in turn ties to pay and
responsibility. Layers models also provide a way to measure progress and
provide job advancement by title and pay commensurate with the skill base.
52
Security Operation Center Field Notes
Regardless ofthe model and title, there is almost always a formal management
layer for the Security Operations team.
What is essential to whatever stratification model your SOC uses is that each
analyst must be willing to ask for help if they find themselves outside of their
depth when working an alarm or handling a ticket.
53
Security Operation Center Field Notes
54
Security Operation Center Field Notes
These types of organically grown teams often miss a critical factor in their
formation: the SOC function wasn't deliberately created following a needs
assessment or a formal model. As a result, varying aspects of SOC Services
described on page 17 opérate at different levels. The SOC staff members write
different reports in response to the same alerts so results are not predictable,
everyone has different skill levels, the operational structure is not internally
consistent, and staff may be frustrated.
How is this problem solved? One well respected method is to apply the Carnegie
Mellon Capability Maturity Model Integration (CMMI). There are five maturity
levels across in the CMMI, with each level representing an evolutionary plateau
in terms of process improvement. Normally, CMMI is applied holistically in an
organization across up to twenty-four process areas, so using it needs to be
adapted and focused for a Security Operations organization. It may not be
necessary, oreven desirable, to push all capabilities and processes to level 5,
because each maturity level is measured by meeting a set of objectives for the
process.
18 Remember - a project is a one-time event while running a security operations team ¡san
ongoing business and IT function. They are very, very different.
55
Security Operation Center Field Notes
Ñame Characteristics19
1: Initial Processes are ad hoc, chaotic; success depends on
heroics as opposed to following a defined process.
Services often work but exceed budget, time,
schedule. Success is not repeatable, and heroic action
frequently saves the day.
2: Managed Workgroups (the SOC) define processes, create work
plans, monitor their processes, and meet a set of
"contract requirements" with its customers (the
business). Configuration management is
implemented with quality assurance.
3: Defined Defined processes for managing work are used, well
understood, Services, procedures, tools in place.
Sound project management is implemented into each
process set. Difference in L2 and L3:
process/procedure can be quite different in each
instance of the process, whereas L3 procedures are
tailored for the workgroup.
4: Quantitatively There are quantitative objectives for quality and
Managed process performance, which are used to manage
processes. Measurement statistics are collected on
specific subprocesses.
19 These points are adapted from the SEI CMMI for Services, 1.3. Note that as this book was
written, 2.0 was just released, so you should review 2.0 material. 1.3 URL:
https://resources.sei.cmu.edu/asset_files/Webinar/2010_018_101_22253.pdf (8/16/18)
20The SOC-CMM is available at Rob's website: https://www.SOC-cmm.com/
56
Security Operation Center Field Notes
• Getting data into the SIEM can be very ad hoc. A well-meaning system
custodian sets up a syslog feed and lets SOC know that new data is
arriving.
® Someone in SOC works with system custodian to make sure that the
systems data can be gathered into the SIEM.
• Someone else in SOC works with the SIEM vendor to make sure that
data is parsed.
• Data survivability baseline is established so that if the source system
stops providing data, it can be detected "quickly enough."
57
Security Operation Center Field Notes
• The SOC manager ensure that all users have completed onboarding for
each data source and confirms that there is equivalent understanding
how to use the data.
• SOC Analysts review alerts as they arrive, with some notion of prioríty
and impact. High priority alerts which are likely to be true positives
receive attention such as reporting for resolution, data owner and
system custodian notification, or routed to Tier 2 for further
investigation.
• SOC analysts may or may not be consistent with each other in the
decision-making process on alarm resolution. During times of high
stress, alerts are not consistently managed. Critical alerts may never be
evaluated in a timely manner.
• Alerts may occasionally be reviewed daily in the aggregate such that
critical alerts always receive some form of treatment as a "stop gap".
58
Security Operation Center Field Notes
• As data sources are integrated into the SIEM and SOC processes, they
are mapped into the taxonomy, the review processes are defined, and
the SOC staff is fully trained on the data source and the alarm
conditions it brings to light.
• The SIEM platform is augmented with orchestration and automation in
orderto improve analyst responsiveness and put better data in the
hands ofthe analyst.
• Alarm and Event data is used to guide a threat hunting program, and
the threat hunting program in turn guides and improves alarm
generation capabilities with the corresponding SOC processes.
59
Security Operation Center Field Notes
60
Security Monitoring Use Cases by Data Source
1. Think about common scenarios for attacks and adversarles who target your
organization. To aide in this process, this chapter introduces a scenario, an
attacker plan, and a defense plan.
2. Ensure you know how to get the data to support the use case points.
3. Gather vendor and product documentation so analysts can easily look up
event detall as they use event data when researching an alarm.
4. Determine the risk that the use case is designed to address for the
organization.
5. Decide and document
how the analyst will
respond to the use case.
6. Review the enterprise
data inventory survey in
the SOC planning chapter.
The Scenario
Every organization has a wide
variety of types of data that
they can collect. As the SOC
considers collecting data,
evalúate that data against the
actual needs of the
organization, how that data will be used to detect an adverse security condition,
how cióse the data is to the end user focused application, and how user
attributable that data is. To ¡Ilústrate these concepts, this section will walk
through an example of systematic intellectual property theft conducted against
VictimCo.
The Setup
First and foremost, VictimCo is in possession of valuable data. After surveying its
business environment, valué chain, and identifying its valuable data, VictimCo
makes several key determinations:
61
Security Monitoring Use Cases by Data Source
62
Security Monitoring Use Cases by Data Source
21 https://digital-forensics.sans.org/blog/2016/10/29/malware-can-hide-but-it-must-run
63
Security Monitoring Use Cases by Data Source
64
Security Monitoring Use Cases by Data Source
Once operating system data is collected, then focus on valuable network level
trace data. Network level instrumentation should focus on chokepoints, flow
data, and application support intelligence such as DNS activity, web browsing
activity, and network flows between network segments. For example,
workstation to workstation traffic is highly unlikely in most corporate networks.
First, let's level set on the phrase "use case"22. A use case is "a set of actions or
steps which define the interactions between an actor, which can be a person, a
system, or a Service, to a system in order to achieve a particular objective." A
full-fledged use case témplate, tuned for Security Operations and focused on
instrumenting the environment presented in this book on page 133.
65
Security Monitoring Use Cases by Data Source
For a SIEM and a SOC team system, building a use case has several requirements
based on the definition from above:
1. You must be able to describe the observed condition that is relevant to your
security posture and realizes the use case (the objective).
2. The system providing data to the SIEM must be capable of actually auditing
the desired behavior (observe the action by person or system interaction).
3. The system must provide the event record with sufficientfidelity to the
SIEM (as defined under Log Record Data on page 223.) (define the action).
4. The SIEM must be able to process and present the event at the necessary
level of granularity for the Sec Ops fu nction (to measure or observe the
interaction for the actor).
With this definition and these criteria in mind, ITI define dozens of SIEM use
cases that should be built out along with the corresponding event data that
makes up the use case. Again, as in the rest of the content of BTHb:SOCTH, most
of these use cases are based on experience and were implemented to one
degree or another. This inventory of use cases is not exhaustive, and should not
be considered "the definitivo list". Hopefully as you look through these they will
66
Security Monitoring Use Cases by Data Source
assist you in implementing your own use cases based on your data sources and
valué chain.
23 https://news.netcraft.com/archives/2018/01/19/january-2018-web-server-survey.html
67
Security Monitoring Use Cases by Data Source
At the top. the attacker exercises the cyber kill chain to deliver some content by
some means that interacts with a user application. In the case of a malicious
email, the user may be enticed to open an attachment that could be a malicious
PDF file or an office application with a macro that has malicious script code in it.
Once the payload is opened, or successfully delivered and the user interacts
with the payload, the attacker can begin to leverage a wide variety of
techniques found in the MITRE ATT&CK matrix.
The elements illustrated in the next figure along with references to the Cyber
Kill Chain and the MITRE ATT&CK Framework.
68
Security Monitoring Use Cases by Data Source
One of the first actions is to establish persistence so that the attacker can
return. After that, attackers vary their actions. They may attempt to crack
passwords, gather hashes from the SAM database or from an inbound
authentication token on a connection, search the network for an opportunity, or
if they are lucky, pillage the system they compromised because it was just the
right one. In any event, an attacker has numerous opportunities that can be
used to take advantage of Windows and Microsoft networks. The older the
operating system and the less out of date it is where they establish the first
foothold, the better (for them, not the blue team).
Questions to Answer:
69
Security Monitoring Use Cases by Data Source
4. Is there logging enabled for the applications and the platforms they depend
on?
5. Who is the application and platform owners and custodians SOC will need to
engage with?
6. What will the SOC do in response to an alarm?
7. How will you maintain your use case library?
8. At what point does a use case produce change control Ítems?
24 https://acsc.gov.au/infosec/mitigationstrategies.htm (8/18/2018)
70
Security Monitoring Use Cases by Data Source
Use cases based on email are easier to implement for locally hosted email
systems than cloud email systems, because it is easier to get logs from on-
premises systems. Free email systems are highly unlikely to provide meaningful
data export and SIEM integration. Paid cloud email systems may not support
SIEM integration or may only provide user activity through a downloadable
report.
71
Security Monitoring Use Cases by Data Source
72
Security Monitoring Use Cases by Data Source
You can either develop an inventory yourself, or use a tool like "dnstwist" by
Marcin Ulikowski25. This tool is written in Python, and does have some
dependencies required for it to work properly on the command line.
The goal in using a tool like dnstwist is to generate likely look-a-like or domains
that a phishing campaign is likely to use against your organization. The dnstwist
tool can also be used to check and see if a twisted domain is regístered.
1. Investígate Domain Ñame: Pulí out the domain ñame portion of email sent
to/from the organization and compare it to a twist domain. If you have a
match, then drop the email. Even better, do not accept email from a twist
domain.
2. Check twist against browsing: Pulí the FQDN portion of a domain ñame
from web server logs, and alarm if a user visits a twisted domain ñame.
3. Alarm on DNS Lookups: DNS lookups against twisted domains should also
generate an alarm. Truth be told, DNS blackholing twisted domains is
actually a solid protective measure, so that if a user attempts to visit a
suspect site it will be directed to 127.0.0.1 (there is no place like home).
25 https://github.com/elceef/dnstwist
73
Security Monitoring Use Cases by Data Source
Implementing this type of a DNS sinkhole not only protects your users, it
also allows for an alarm condition.
74
Security Monitoring Use Cases by Data Source
75
Security Monitoring Use Cases by Data Source
9. A user with Elevated Access logging into an infected system: This use case
requires that you maintain an inventory of users with elevated access, or
that all of these users have a particular naming convention so elevated
access accounts can be more readily detected. If they login to an infected
machine, then at a minimum an automated notification advising them to
change their password is in order.
Windows Defender has its own set of Event ID's. In many cases, Windows
Defender also ineludes a suspicious file ñame, a unique threat ID, a severity
rating, file path, error codes, and other useful details. These details are
representativo attributes to use when building dashboards, alarms, and reports.
EventID Ñame
1000 An antimalware sean started
1001 An antimalware sean finished
1002 Sean stopped (canceled) before finished
1005 Sean terminated due to error
1006 Detected Malware
1008 Action on Malware Failed
1010 Antimalware could not restore an ítem from quarantine.
1115,1116 Malware detection
1117 Malware remediation or action taken
1119 Remediation error
2001 Failed to update signatures
2003 Failed to update engine
2004 Reverting to last known good signatures
3002 Real time protection failed
Appiácation Whitelisting
The security posture of the endpoint is more important than ever before. These
systems are designed to monitor executables running on a system when running
in detection mode. When running in prevention mode, they will stop running an
executable that doesn't match an approved policy. Application whitelisting can
identify several adverse conditions. Further, since this capability records end
user activity, it can be very useful in an employee investigation case.
76
Security Monitoring Use Cases by Data Source
Windows has two facilities of note: AppLocker for Windows 7 and above, and
Software Restriction Policies for Windows Vista and below. Both technologies
record control binary usage. AppLocker events in the AppLocker event log and
can be enabled using group policy. There are at least sixteen different events
recorded. EventID's 8020 to 8027 are focused on package deployment issues, so
they are not listed here.
77
Security Monitoring Use Cases by Data Source
Once alerts from a DLP system reach a certain threshold they may need to be
investigated by the proper internal team. Realize that in some organizations,
DLP events can be quite normal. For example, sensitive patient data is routinely
handled at a hospital or an Insurance company. If a user emails ten
spreadsheets and the DLP system intercepts them and encrypts them for
delivery to the recipient, the user may be aware this is normal and wanted that
action to occur because they are trusting the DLP system.
Before going too far down the road with DLP, the SOC team will need to
determine if there is a more applicable ¡nternal team who are a better business
fit than the SOC (there should be...). For example, an Insurance company likely
has a Member Privacy team, or HR may perform investigations using the DLP
system. One argument in favor is that by sending in alarm data to the SIEM, an
end user activity report will have a more complete picture to present during an
employee investigation. Further, since the DLP system identifies potential IP
loss, an analyst can incorpórate these alerts to gain a better picture of end user
activity and notify the appropriate internal team if warranted.
78
Security Monitoring Use Cases by Data Source
ñame queries that are outside of the norm and being able to detect the true
source IP address ifat all possible. One issue that will prove to be difficult is a
lack of infernal reverse DNS lookups and stale DNS entries. If you can't reliably
lookup an IP to a ñame, there will be a small impact on alarm Processing time.
The situation can be a bit worse when an IP comes back to múltiple systems.
Collecting DNS: Collecting DNS from a DNS server can be problematic. For
example, Windows DNS requires that you enable "debug logging", and then
fully parse that data through a either a local or remote file reader process.
Another problem with DNS is that most (90%+) of the traffic on the network are
local queries. Local queries are normal. When considering how to collect DNS,
focus on collecting internal to external queries, find where those queries are
resolved, and collect data at that point using network extraction as the
collection method. If there is a mirror port available at the perimeter, DNS query
and responses will be logged/rom the internal DNS server(s), as they are
forwarding queries on behalf of the end user. If you collect DNS traffic via a
mirror port on the same switch as the DNS server, you will collect a significant
amount of normal query traffic for the intemal network that will have low to no
valué for identifying attackers. There are at least 30 defined record types
available for use, with the more common being A, CNAME, PTR, SPF, AAAA, NS,
and MX. TXT records are seen, but in low volume. There are at least two well-
known tools to collect DNS: PassiveDNS and Bro IDS.
79
Security Monitoring Use Cases by Data Source
26 Note, though that you may see DE:AD:BE:EF:CA:FE on the network. And there are a few humans
who can natively read hexadecimal network traffic.
80
Security Monitoring Use Cases by Data Source
10. Look-a-Like or fuzzed domains: Review the section Email and Web:
Interactions with Look a Like or Doppelganger Domains on page 73 when
working through DNS use case development.
11. DNS queries not from authorized servers: An enterprise should only have a
small number of internal DNS servers that can forward queries to servers on
the Internet. Any DNS query outside of this boundary should be
investigated, if for no other reason that ensuring the sender is properly
configured in order to provide operational assurance.
12. Volume and volume profile changes: Establish a baseline profile for DNS
traffic. These indicators can become alarm conditions once baselines are
established. Examplesare:
a. Average queries per hour during working hours/off hours.
b. First time use domain queries (new domain ñame seen).
c. Volume of SRV RR, TXT, and MX queries.
d. Interna! failures - lookup for domain fails.
13. Ñame analysis: High volume queries with hostnames that are random for
the same 2nd level domain and the same length indícate a DNS tunneling
tool is sending data to the attacker's site, because the DNS server is
consuming the host ñame as encoded data.
14. Foreign countries: You should study your organization's communication and
operating model to determine how much communication occurs to
countries outside of your own country. For example, a University with a
varied foreign student population would considerthis normal, but an
Insurance company that operates in a few States in the US would consider
several q ueries to foreign countries abnormal. Note that if you are reading
this book and you are in a foreign country, queries to ñame servers in the
US and several European countries may be very common and may make this
analysis more difficult.
15. Queries to Dynamic DNS providers: There are several dozen dynamic DNS
providers operating today27 who provide nearly free or inexpensive ñame to
IP DNS resolution. A common model is for a home user to register their IP
address and allow certain Services through, with a ñame unique to them
that their ISP would not provide. For example, a VPN client. Attackers can
easily use these Services as an avenue for hosting malicious Services such as
C2 DNS Service because DDNS providers allow for rapid changes of a ñame
to an IP address and can be used at nearly no cost.
16. Abused Top Level Domains (TLD's): Spamhaus maintains an ever changing,
evidence-based inventory of the top ten most abused domains ñames28,
which is expresses as an aptly named "badness índex". Integrating this
81
Security Monitoring Use Cases by Data Source
functionally into the SIEM may not be practical, but integrating a check of
the domain TLD into the incident response process and the analyst checklist
certainly is. As of June 28, 2018, there are 1,503 TLD's.
17. Traffic to external IP without DNS query: Direct HTTP, HTTPS, FTP, SSH, and
likely other protocols directly to an IP address is suspicious. It is not
common for an end user to type in https://#.#.#.#/. With whatever method
you have, review which end systems are communicating outbound directly
to an IP without a ñame. A caution: a reverse lookup could be performed,
with some risk of alerting the site owner that you are trying to get a ñame
for an IP. It is best to use an intermediary, like a cali to any site that offers a
NSIookup function. (Root DNS servers don't count!)
18. Use of non-authorized DNS: There are several free DNS Services available
on the Internet other than the DNS that the sites ISP provides. Queries to
these DNS servers, such as Google's al 8.8.8.8 and 8.8.4.4, may indícate a
condition that needs resolution.
Here, alerts from an endpoint protection system can have a high "signal to
noise" ratio because those alerts are pre-validated. These events occur because
a binary matched an entry on a known feed list or matched a detection
criterion. By consuming and analyzing endpoint protection system alerts only
(and not all endpoint activlty), you can actually perform threat intelligence and
assessment on the user population. If users are predominantly becoming
iníecled and Ihen bringing their notebook back into work, or if users insert USB
drives in their computers and there is infectionware on the USB, then SOC has a
better-defined path for security awareness training and remediation.
82
Security Monitoring Use Cases by Data Source
In order to get data into the SIEM platform for SaaS EDR, some form of
encrypted syslog Service or some sort of REST API needs to be configured to
send data into the SIEM. Do not send all endpoint data that the EDR platform
collects to your SIEM. Rather, send the "condition detection" data to the SIEM.
83
Security Monitoring Use Cases by Data Source
EventID Ñame
4720 A user account was created.
4722 A user account was enabled.
4723 An attempt was made to change an account's password. **
4724 An attempt was made to reset an accounts password. **
4725 A user account was disabled.
4726 A user account was deleted.
4738 A user account was changed.
4781 The ñame of an account was changed.
** These events should be monitored differently.
1. Short cycle account create and account delete events: This use case
catches accounts that are created and removed within a very short time
window. As a bonus, the severity would be raised if the account was used,
such as a logon event between the create and delete event.
2. Short Cycle elevated group add and group remove events: This use catches
accounts added to highiy priviieged groups like "Domain Admins" and then
quickly removed from the group. Extensive damage can be done in a short
time.
3. Accounts created/modified/disabled by staff other than designated
account managers: This condition helps identify policy violation, rogue
admins, or attackers who gain access to a domain admin level credential. In
more mature organizations there will be a few IT that manage user accounts
and group changes. When this situation exists and others manage accounts
there may be a policy violation, a social engineering event, or true
maliciousness.
4. Accounts managed by users other than the designated Service account: A
Service account is used by a mediated access application such as NetlQ DRA,
84
Security Monitoring Use Cases by Data Source
85
Security Monitoring Use Cases by Data Source
are several websites with lists of default accounts and passwords29. Default
account usage has been in the OWASP Top 10 for several years.
11. Local account creation and elevated access. One of the key tenants in an
Active Directory domain is centralized authentication. In reality, practices
vary widely when it comes to local accounts. For example, an organization
may grant administrative access to a workstation by creating a local account
for the workstation user to accomplish an administrative task, or a domain
level account and then add that account to the local administrators' group.
The SOC should understand how elevated access is applied and be able to
detect elevated account usage.
86
Security Monitoring Use Cases by Data Source
group is used in an AD forest, while the type defines the intended usage. This
model translates into dozens of event IDs that need to be monitored. In practice
AD groups are often used to control access to a resource that must be
monitored, such as a directory where financially significant data is stored in a
publicly traded company31. Further, there are several groups within Active
Directory that provide elevated access. Even worse than that, groups can be
nested. For example, an organizaron may have an "Administrative Service
Accounts" group embedded within the Domain Admins group. If you are only
monitoring the Domain Admins group, you will miss a user being added to or
removed from a nested group which has Domain Administrative privileges.
When monitoring group changes with a SIEM, a subset of changes may prompt
a notification. There are many other means to support the intent of a control,
such as creating a report for group membership through scripting out a report
or using a purpose-built application.
Security Distribution
Local Global Universal Local Global Universal
Created 4731 4727 4754 4744 4749 4759
Changed 4735 4737 4755 4745 4750 4760
Deleted 4734 4730 4758 4748 4753 4763
Member 4732 4728 4756 4746 4751 4761
Added
Member 4733 4729 4757 4747 4752 47620
Removed
31 In the United States, this requirement is derived from Sarbanes Oxley Controls.
87
Security Monitoring Use Cases by Data Source
Changes to a select set of NTFS and application control groups may be in order.
When you create a SOC alarm or an email notification for a resource owner
make sure that the message explains what the group Controls access to -
meaning the resource Itself or the application right managed by the group.
Don't automate a notification that a user was added to "NTFS_g45_Direc_RW".
Instead, find out the purpose and path of the directory and us that instead. for
example, "Shared Drive, Monthly Financial Summaries, path ñame
\\Storage\NY\FinRep_Monthly" for the monthly financial performance reports
for the NY business unit.
Applications can also use Active Directory groups to control access by mapping
an AD group to an ¡nternal role within the application. Following the same
example, a notification that States a user was added to "PeopleSoft Production
Admin Members" is betterthan "AppCtlFinPplAdminsPRD".
88
Security Monitoring Use Cases by Data Source
(assuming that the policy is set or the system is Windows 10 or Server 2016.)
The "Security ID" field is set to the local system ñame and the local user
account. The account domain is set to the local system, meaning that the SAM
database used to authenticate the user is one resident on the physical system.
By default, a standalone system has the default valué "WORKGROUP" in the
Account Domain field unless the NetBIOS workgroup ñame is changed. For a
standalone system where someone logs in with a Microsoft account (a cloud
account used to connect the system to the Windows Store and other identity),
such as a home user's system, the process is similar to a standalone login.
Domain System: This process is different when a user logs onto a workstation
and the user is authenticated from the domain. There is a 4768-event written to
the DC that authenticates the user, a 4624 event is written to the local security
log with the domain ñame in the Security ID and the Account Domain fields.
When users termínate their session, there will be a 4647 followed by a 4634
event. As users authenticate to Windows file shares, there are 4624 Type 3
events record on the serving system. As users authenticate to other Kerberos
integrated Services, there are 4769 events registered on the DC.
89
Security Monitoring Use Cases by Data Source
32 For Windows 2000/2003/XP, the Event IDs are 529, 530, 531, 532, 533, 534, 535, 536, 537, and
539. Hopefully you will not need this information @.
90
Security Monitoring Use Cases by Data Source
presence such as a PC consolé logon and a VPN login, not an RDP login and a
VPN login.
3. Geographically improbable VPN Logins: Modern VPN systems can provide
country and city of the source IP for a connecting IP, or the data can be
enriched with geo lookup data as it arrives at the log collection point. More
sophisticated platforms like Splunk actually have built-in functionality to
detect geographically improbable access. This means that a user logged in
from one location and subsequently logged in soon after from another
location that they could not travel to in the time allotted. For example, from
France at 10 AM and then Cañada at 10:15 AM, same day. If your SIEM
doesn't have this functionality, then as a simple check, check the country
code to make sure it matches the country for your user population.
Depending on the user population, this check may reveal a compromised
account. For systems that have a tracking list functionality, check the
current login State or province and country with the prior login. If they are
different and an insufficient time has elapsed from the prior login to allow
for travel, there may be a problem.
4. Network Switches/Routers/Devices: These devices represent the network
infrastructure beneath the network operating system and application stack.
Thus, they must be kept secure. Even in large corporations, there are often
a small group of well-known users that access the network support fabric.
Regardless ofthe authentication method (RADIUS, TACACS+), if a user not
from this group attempts to access network hardware, an alarm should be
raised due to an unauthorized user access attempt.
91
Security Monitoring Use Cases by Data Source
Normal logging needs to be enabled in the RRAS consolé. Choose "Log all
events". Normal events are written to an actual log file:
% w¡ nd i r%\syste m32\LogFiles
92
Security Monitoring Use Cases by Data Source
set of jump box resources, those resources are located on their own routable
segment, and only designated accounts can login to the jump boxes. Once the
jump box farm is setup, deny inbound access into all other server segments for
port 3389 (RDP) for Windows and 22 (SSH) for Linux machines. Some systems
may use VNC on port 5800/5900, or Xll Services on port 600X. Inelude any
other remote desktop equivalent not listed here.
Jump boxes are an excellent example where an EDR applications can really shine
because they provide highly granular awareness of EXEs launched, network
connections, registry activity, and so forth. All jump boxes should have
WEC/WEF enabled.
Table 15 RDP Events from Applications and Services Logs -> Microsoft -> Windows ->
TerminalServices-LocalSessionManager
EventID Ñame
21 Remote Desktop Services: Session logon succeeded. Records user,
session ID, and source address.
22 Remote Desktop Services: Shell start notification received.
Records user, session ID, and source address.
23 Remote Desktop Services: Session logoff succeeded. Records user
and session ID.
24 Remote Desktop Services: Session has been disconnected.
Records user, session ID, and source address
25 Remote Desktop Services: Session reconnection succeeded.
Records user, session ID, and source address.
93
Security Monitoring Use Cases by Data Source
1. Identify Network Hardware: You can achieve this objective (or at least
make progress towards achieving it) by scanning the network with nmap,
looking for a response from ports 443/TCP, 80/TCP, and possibly 22/TCP. If
systems are responding and not generating a log record, or at a mínimum
not seen as a source client IP for authentication on an AD DC, then an
appliance of some sort is identified. Next step is to identify the system and
determine if it can log, and should be configured to log.
2. Collect authentication and change activity: Depending on the device, you
may want logging from them. At a mínimum, you want change activity, user
logins (success and failure), and system reboots.
3. Monitor for default account attempts: Logins to network hardware should
be monitored so that default accounts like admin, root, and supervisor are
not used.
94
Security Monitoring Use Cases by Data Source
4. Monitor for outbound traffic: Most network devices should generate very
little outbound traffic outside of a few specífic sites, which are most often
for contení or system updates. Alarm conditions will vary. For example,
tuned alerts if a piece of hardware makes DNS requests or communicates to
sites outside of a small list. Alternatively, review outbound traffic from a
piece of hardware periodically to detect anomalies. Items of note include
NTP requests, DNS requests not to the local DNS, and software updates
from vendor network ñames.
Printing
Print servers can be configured to record when a user prints a document, the
document ñame, and document size. You are more likely to need to enable print
job monitoring through event log forwarding to support long term employee
investigation. In order to support any assertion other than "User X printed a job
to printer Y at time Z", you will need to enable supplemental auditing33 to
capture the print job ñame and conduct a forensic examination of the
workstation or have an EDR application in place in order to fully support this
degree of attribution.
Based on empirical observation, the Windows Print event order is: 800 -> 801 ->
311 -> 842 -> 804 -> 307 -> 802. Of these, the most relevant are 311 and 307.
33 The GPO path is: Computer configuration » Administrative Templates » Printers» allow job
ñame in event logs.
95
Security Monitoring Use Cases by Data Source
34 This number is based on using Server 2008 and 2012 in a highly virtualized environment. The
Windows admins found that was a size that provided several days to months of record keeping
and still allowed the Event Viewer to be responsive. YMIVIV.
96