WRT 1006: Digital Engineering

Competency Framework

Sponsor: OUSD(R&E) | CCDC

By
Dr. Jon Wade
11th Annual SERC Sponsor Research Review
November 19, 2019
FHI 360 CONFERENCE CENTER
1825 Connecticut Avenue NW, 8th Floor
Washington, DC 20009

www.sercuarc.org

SSRR 2019 November 19, 2019 1

 Outline

• Background & Objectives

• Methodology
• Results
―Taxonomy Framework
―Examples
―Gaps Analysis
• Next Steps

SSRR 2019 November 19, 2019 2

SSRR 2019 November 19, 2019 3
SSRR 2019 November 19, 2019 4
SSRR 2019 November 19, 2019 5
 The Power of Digitalization
Exploiting the digital power of
computation, visualization and communication
to take better, faster actions
Dynamic
System
Validation

Virtual
Physical

SSRR 2019 November 19, 2019 6

 Systems Thinking & Digital Engineering

Digital engineering (DE):

• an engineering approach that captures and analyzes data that is
in a digital format which is semantically rich and interconnected

• enables people to leverage the power of computing,

visualization, and communication to significantly enhance
efficiencies, quality, and innovation across the complex system
development lifecycle

- Sandy Friedenthal, SERC DE Workshop, Nov. 15, 2019

SSRR 2019 November 19, 2019 7

 Digital Convergence

Analysis
Big Data
(AI/ML)

Data-Driven
Model-Driven
Decisions
Architectures

Internet Cloud
of Things Computing
DoD definition of model as ‘a physical, mathematical,
or otherwise logical representation of a system, entity,
phenomenon, or process’ (DoD 1998)
Thomas Siebel, “Digital Transformation: Survive and Thrive in an Era of Mass Extinction”, 2019.
SSRR 2019 November 19, 2019 8
 Research Needed

In support of the DoD’s implementation of the Digital Engineering

Strategy, researchers shall investigate the critical digital engineering
knowledge, skills, and abilities needed by the DoD acquisition
workforce. Researchers shall develop a Digital Engineering
Competency Framework and take into consideration the following:

• Must support the implementation of 2018 Digital Engineering

Strategy by each Component based on their organic DE processes
and tools.

• Must be Component agnostic

• Must follow DoDI 1400.24 volume 250

SSRR 2019 November 19, 2019 9

 Research Tasks

The SERC researchers shall perform seven tasks:

1. Define the digital engineering activities – and supporting competencies

(knowledge, skills, and abilities) – required to support lifecycle phases from
concept through disposal. These activities form the foundation of the Digital
Engineering Competency Framework; and must include the considerations
outlined above.

2. Building on task 1, identify aspects of the digital engineering lifecycle activities

and competencies – that are specific, unique and relevant to the acquisition
engineering (ENG) workforce.

3. Develop a Digital Engineering Competency Model, using the DoDI 1400.24

vol.250 “Competency Taxonomy. The competency model should be structured as
follows:
―Full set of digital engineering competencies
―Subset of digital engineering competencies unique to the acquisition ENG workforce
―Subsets of digital engineering competencies unique to the acquisition workforce
broken out by career fields.
SSRR 2019 November 19, 2019 10
 Research Tasks (cont.)

4. Based on the work done in task 1, map each competency set identified in task
3 to the lifecycle phase. This completes the Digital Engineering Competency
Framework.

5. Conduct a gap analysis comparing Defense Acquisition University's (DAU)

current curricula against the competency requirements.

6. Provide recommendations on creating a digital engineering curriculum as well

as modifying the applicable acquisition career fields’ curricula to build
interdisciplinary digital engineering knowledge and abilities.

7. Map digital engineering knowledge and abilities to commercial job titles and
job descriptions and requirements of Digital Engineering.

SSRR 2019 November 19, 2019 11

 Use Cases

DECF use cases are a critical input into its design, structure, and scope. The
critical objective is for the DECF to enable transformation of the acquisition
workforce – in particular the ENG workforce – for successful acquisitions in a
digital engineering environment. The following are some of these avenues:
• Increase skills of current workforce
―Workforce evaluation
―Career Planning
―Creating DE training programs
• Grow workforce
―Creating Position Descriptions
―Hiring for Digital Engineering Positions
• Transform organization
―Identifying Critical Roles

SSRR 2019 November 19, 2019 12

 Outline

• Background & Objectives

• Methodology
• Results
―Taxonomy Framework
―Examples
―Gaps Analysis
• Next Steps

SSRR 2019 November 19, 2019 13

 DECF Development Approach Methodology
• Zoom out to Big Picture for context

• Identify focus areas at intersections

• Approach from multiple directions simultaneously

―Review existing material for DE specific competencies and KSAB
o Develop a start on DECF Competencies, Definitions, and KSAB
―Review SECCM* Competency Definitions, Tasks & Proficiency Levels for DE
Enhancements
o Inform development of DECF
o Identify specific SECCM/ENG DE enhancement edits
―Investigate SECCM/ENG competency models for gaps and opportunities

*Systems Engineering Competency Career Model

SSRR 2019 November 19, 2019 14

 Big Picture Context Approach Map
Defense Acquisition Work Force
Engineering Systems Engineering Digital Engineering
Functional Area

Life Cycle Logistics

Information Technology

Production, Quality, and

Program Management

Property

Test and Evaluation

Auditing

Business (Financial Management)

Contracting

Engineering

Facilities Engineering

Manufacturing

Purchasing
Business (Cost Estimating)

Science and Technology Manager

Unknown/Other
X X X
X X X X
X X X X
X X X
X X X X
Engineering X X X X X X
X X
X X X X
X X X
X X
X X
X X X
X X
X
X X X X
Systems Engineering X X
X X X X
X X X X X X
X X
X X X
X
X
X X
X X X
X X
Digital Engineering X
X X X X
X X X
X
X X X

SSRR 2019 November 19, 2019 15

 Cross Walking SE Competencies to Inform DECF

Reviewing Many SE Informing Development of

Competency Models DECF

Identifying SE Competencies
and Gaps Digital Engineering Competencies

Systems Engineering
Competencies

SSRR 2019 November 19, 2019 16

SSRR 2019
DECF
Mapping Between

Digital Engineering
Informing Development of
ENG/SECCM and DECF

Auditing

November 19, 2019

Business (Financial Management)
Business (Cost Estimating)
Contracting

X
X
X
X
X
X
X

Engineering
Facilities Engineering
Information Technology
X
X
X
X
X

Life Cycle Logistics

Production, Quality, and Manufacturing
X
X
X
X

Program Management
Property
Purchasing
Science and Technology Manager
X
X
X
X
X

Test and Evaluation

Defense Acquisition Work Force Functional Area

Unknown/Other
for DE Enhancements
Reviewing ENG/SECCM

17
Cross Walking DE to ENG/SECCM
 DECF Development Approach

• DECF will be
developed through
―Workshops with SMEs
and DE practitioners
―Review by SME panel
―Review by Sponsor,
validating use models
―Broader Community
Feedback

SSRR 2019 November 19, 2019 18

 Outline

• Background & Objectives

• Methodology
• Results
―Taxonomy Framework
―Examples
―Gaps Analysis
• Next Steps

SSRR 2019 November 19, 2019 19

 Taxonomy

• Positions

• Roles

• Competencies

• KSABs

• Proficiencies

SSRR 2019 November 19, 2019 20

 ENG Career Field Competency Model

SSRR 2019 November 19, 2019 21

 Systems Engineering Career Competency
Model (SECCM) Background
• The SECCM was aligned with the ENG model competencies to maintain
consistency within DOD.

• The model was developed and verified by a collaborating team with members
from the US Office of Personnel Management (OPM), Navy, Army, Air Force,
Marine Corps, and the Missile Defense Agency to develop and verify the
competencies used by defense systems engineers.

• OPM process used to ensure SECCM is verified IAW Uniform Guidelines on

Employee Selection Procedures.

• Verification IAW Uniform Guidelines is critical to allow the SECCM to be used as

a basis for “high stakes” human resource functions for all of the US Department
of Defense.

• The US Deputy Assistant Secretary of the Navy, Research, Development, Test

and Evaluation, sponsored the development of the SECCM.

• SECCM currently deployed by OCHR as the SECCM Competency Network.

SSRR 2019 November 19, 2019 22
 SECCM

Systems Engineering Career Competency Model

Technical Management Business Acumen Analytical Professional

• Acquisition • Industry Awareness • Transition • Communication

• Risk Management • Organization • Integration • Leading High
• Requirements • Cost Estimating • Design Considerations Performance Teams
Management • Proposal Process • Tools and Techniques • Personal
• Configuration • Supplier Management • Stakeholders Effectiveness/Peer
Management Requirements Interaction
• Negotiations
• Technical Assessment Definition • Problem Solving
• Cost, Pricing and
• Data Management Rates/Cost • Requirements Analysis • Professional Ethics
• Software Engineering Management • Validation • Strategic Thinking
Management • Financial Reporting • Verification • Coaching & Mentoring
• Decision Analysis and Metrics • Mission-Level • Managing Stakeholders
• Interface Management • Business Strategy Assessment • Mission and Results
• Technical Planning • Industry Motivation, • Architecture Design Focus
Incentives, Rewards • Implementation • Sound Judgment
• Contract Negotiations • Engineering Disciplines • Continual Learning

The SECCM consists of 44 competencies and 179 tasks

SSRR 2019 November 19, 2019 23

 Context: DoD-I 1400.25 vol 250

SSRR 2019 November 19, 2019 24

 DOD-wide Competency Taxonomy

Five-Tiered Competency Framework

Competency Components

Tier 1: Core Competencies Each competency in the Five-Tier Framework is

Apply across DOD regardless of component or occupation, e.g., described by the following components:
DOD leadership competencies

Tier focused on:

Competency Title

Tier 2: Primary Occupational Competencies Competency Definition

Apply across discrete occupational series/functions, i.e., one or
more functionally related occupations that share distinct, common
technical qualifications, competencies, career paths, and
progression patterns Proficiency Level Definition/ Job Tasks
Illustration

Tier 3: Sub-Occupational Specialty Competencies*

Proficiency Levels (tied to assessments)
Unique to sub-occupational specialty, e.g., set of geo-technical
indicate the degree to which employees
competencies within the civil engineering occupation performed a competency.
Level 1 = Awareness
Level 2 = Basic
Tier 4: Component-Unique Competencies* Level 3 = Intermediate
So unlike any of the other competencies identified that they exist Level 4 = Advanced
at the component level and are unique to the context or Level 5 = Expert
environment in which the work is performed.

Tier 5: Position-Specific Competencies*

Required for a particular position within an occupation and are not
addressed in the Tiers above, e.g., a specific civil engineer may
require financial management competencies

* To be developed at a later date

SSRR 2019 November 19, 2019 25

 Proficiency Level Definitions

0 1 2 3 4 5
Basic Intermediate Advanced Expert
Proficiency Level General General Detailed In-Depth
None Awareness
Knowledge Knowledge Knowledge Knowledge
(Entry) (Junior) (Senior) (SME)
Applies the Applies the Applies the
Applies the competency in Applies the competency in competency in
competency in the somewhat difficult competency in considerably exceptionally
simplest situations. situations. difficult situations. difficult situations. difficult situations.
Requires Generally requires Serves as a key
No experience with Requires close and Requires frequent occasional little or no resource and
Definition or knowledge of the extensive guidance. guidance. guidance. guidance. advises others.
competency. Demonstrates
Demonstrates comprehensive,
Demonstrates Demonstrates Demonstrates broad expert
awareness of familiarity with understanding of understanding of understanding of
concepts and concepts and concepts and concepts and concepts and
processes. processes. processes. processes. processes.

SSRR 2019 November 19, 2019 26

 Proficiency Levels Definitions Are
Organization Dependent

SECCM Critical Competencies Across

Career Levels by Service
Number of Critical Competencies

GS Level
SSRR 2019 November 19, 2019 27
 DECF v0.25 Competency Title and Definition Start

DoD definition of model as ‘a physical, mathematical, or otherwise logical

representation of a system, entity, phenomenon, or process’ (DoD 1998)

SSRR 2019 November 19, 2019 28

 DECF v0.25 DE KSAB Alignment - Partial View
Competency Titled
mapped to KSAB
Number Atomic Level Competency Description KSAB Atomic Level
Able to access and manipulate digital models to gain
understanding of relationships to capture
perspectives to show intent and translate model
1 Model Usage information to various contexts. Analyzes system model and architecture
Able to access and manipulate digital models to gain
understanding of relationships to capture
perspectives to show intent and translate model Apply architecture to the remaining system disciplines and specification
1 Model Usage information to various contexts. development
Able to access and manipulate digital models to gain
understanding of relationships to capture Apply integration using the integrated modeling environment to execute
perspectives to show intent and translate model the MBSE process for complex system development efforts to represent
1 Model Usage information to various contexts. the system requirements and design
Able to access and manipulate digital models to gain
understanding of relationships to capture
perspectives to show intent and translate model
1 Model Usage information to various contexts. Determine system model and architecture
Able to access and manipulate digital models to gain
understanding of relationships to capture
perspectives to show intent and translate model Ensure that DE activities and digital artifacts development are performed
1 Model Usage information to various contexts. according to intent
Able to access and manipulate digital models to gain
understanding of relationships to capture
perspectives to show intent and translate model Evaluate multiple system models and architectures at the mission or
1 Model Usage information to various contexts. system-of-system level.
Able to access and manipulate digital models to gain
understanding of relationships to capture
perspectives to show intent and translate model Extend SysML-based MBSE at an enterprise by penetratating the concept
1 Model Usage information to various contexts. into New Departments or Product Areas
Able to access and manipulate digital models to gain
understanding of relationships to capture
perspectives to show intent and translate model Familiarity with integrating SysML system model with other product and
1 Model Usage information to various contexts. analytical models including physics-based models.

SSRR 2019 November 19, 2019 29

 SECCM Competency Definitions with DE
Enhancement – Partial View

1.0 Mission Level Collaborates with user community to assess mission areas end-to-end,
Assessment across system and platform boundaries, to identify and close
integration and interoperability (I&I) gaps in mission critical capabilities.
Uses SE models to describe high level mission needs.
2.0 Stakeholder Works with the user to establish and refine operational needs,
Requirements Definition attributes, and performance parameters based on established
processes in a requirements model and ensures all relevant
requirements and design considerations are addressed to establish a
set of baseline capability requirements
3.0 Requirements Analysis Ensures the requirements derived from the stakeholder-designated
capabilities are, analyzed, decomposed, functionally detailed across
the entire system, feasible, and effective represented in a SysML
model.
4.0 Architecture Design Creates and maintains architectural products throughout the life-cycle
integrating hardware, software, and human elements; their processes;
and related internal and external interfaces that meet user needs and
optimize performance. Uses SE models to describe the system
solution/ parameters IAW a SE methodology.

SSRR 2019 November 19, 2019 30

 Investigating SECCM/DE Job Task Possibilities
– Partial View
Competency Competency Description Task
Assesses mission areas end-to-end, across system
and platform boundaries, to identify and close
1.0 MISSION-LEVEL ASSESSMENT integration and interoperability (I&I) gaps in mission
critical capabilities.
Analyzes gaps in the portfolio between mission objectives, existing or planned Able to view relationships in the
capabilities, and available funding. V digital model and inputs.
Analyzes mission-level requirements to determine if they are obtainable across programs
or an enterprise. V
Analyzes the solution space to identify potential solutions that could address a problem Able to directly use the digital
or take advantage of an opportunity. U model.
Creates various scenarios for system use, functions, and performance that include the Able to create and manipulate
Concept of Operations. Cr digital models
Defines operational and top-level systems requirements that are reasonable, complete, No direct interaction with the
and testable. N digital model needed.
Demonstrates how requirements relate to key performance parameters and measures of
effectiveness. V
Defines the mission problem(s) or opportunity(s). N
Works with the user to establish and refine
operational needs, attributes, performance
parameters, and constraints that flow from the Joint
2.0 STAKEHOLDER REQUIREMENTS Capability Integration and Development System
DEFINITION described capabilities, and ensure all relevant
requirements and design considerations are
addressed to establish a set of baseline
requirements.
Interact with others by
Collaborates with the customer/sponsor to achieve understanding and agreement about collaborating within the digital
systems requirements. Co model
Conducts a needs assessment to elicit user requirements. Co
Creates a Concept of Operation to address the results of a needs assessment. Cr
Translates stakeholder expectations into system or program requirements. Cr
Defines the business and mission need for systems that will provide services, capabilities
or platforms to end-users and other stakeholders. N
Directs the base lining of stakeholder expectations for a system or program. N
Documents the decisions and rationale for end user requirements to ensure
understanding during the final development stages. V
Validates stakeholder requirements for a system or a program. V
Analyzes capability needs and operational constraints, in collaboration with the customer,
to derive system requirements and technical performance measures for system
development. Co
Identifies the individual stakeholders or stakeholder classes who have a legitimate
interest in the system throughout its life cycle. N
Elicits stakeholder requirements from the identified stakeholders. Co

SSRR 2019 November 19, 2019 31

 SECCM/DE Proficiency Levels Example
Tools & Techniques: Applies tools, techniques, and procedures to enable systems engineering practice.
Level Example
1 Awareness of the systems engineering tools and • Researches the systems engineering tools and techniques to identify the most
techniques; understanding of their purpose and proper current tools available.
uses. • Correlates the tools with the general products developed by systems engineers.
• Operates tools (e.g., CSM, SysML/UML) unrelated to a systems engineering task,
for the purpose of gaining familiarity with them.
2 Uses systems engineering tools and techniques with • Identifies one or more tools that are available options for a given systems
assistance. engineering task.
• Uses tools (e.g., CSM, SysML/UML) in the execution of a simple task to generate a
draft product.
3 Assesses and determines the proper tool for a systems • Provides assessment of all available systems engineering tools and recommends
engineering task based on the capabilities and limitations; the appropriate tool(s) for use on the project or program.
recognizes the impact of the tools’ limitations on the ability • Uses tools (e.g., CSM, SysML/UML) in the execution of a project to generate a
to complete the task; uses systems engineering tools and product(s).
techniques. • Uses systems engineering tool(s) to analyze the existing data and evaluate
performance.
4 Coordinates the exchange of data between programs or • Selects the systems engineering tool(s) for use on a given project or program.
system commands using systems engineering tools and • Reviews and approves the technical analysis and recommendations resulting from
techniques; ensures guidance on preferred tools and the data analysis and distributes for external action.
techniques for a given task is implemented; manages the • Assesses user guides and manages and documents the standard processes for
maturation of tool development. which the tools are used within the program or system command.
5 Provides guidance as to what tools should be used within • Establishes processes in accordance with policy and guidance for the standard
the program or system command; sets required training application of the tools within the program or system command.
levels for the tools and techniques; determines which tools • Guides tool integration and/or defines the common data model for an
and techniques need to be developed or modified in order organization(s).
to meet program goals. • Approves user guides and manages and documents the standard processes for
which the tools are used within a system-of-systems.
• Contributes to communities of interest for systems engineering tools
SSRR 2019 standardization.
November 19, 2019 32
 Gaps Analysis Approach

• At the competency level, compare the non-DoD

models/frameworks and those identified by the ENG model

• Identify gaps not covered in ENG (draft)

• Break the competency descriptions into tasks (not common

outside DoD models)

• Compare the task-level items in the identified gaps with the

ENG models (“true” gaps)

• INCOSE SE Competency Framework • Mission Engineering Competency Model

• NASA SE/PM Competency Model (SERC)
• Helix Atlas Proficiency Model • SECCM Mission Level Assessment
• US Department of Labor Engineering Competencies
Competency Model • MITRE SE Competency Model (2007)
• IEEE Software Engineering Competency
Model (in progress)
SSRR 2019 November 19, 2019 33
 Competency Model Identified Gaps

• 57 competency-level • Example Gap: Communication (3

items not reflected in models)
ENG
• Tier 1 (Core Competencies)
• Broken down into 260
task-level items • Questions:
―What are the critical implications for
communication skills in a DE
Environment?
―How might these differ from a more
traditional acquisition environment?

SSRR 2019 November 19, 2019 34

 Example Gap: Thinking Patterns

• Big-Picture Thinking (SERC Helix) • Example Tasks (from source

descriptions):
• Systems Thinking (INCOSE SECF) ―Understand the System in
Operation
• Critical Thinking (INCOSE SECF) ―Understand Interfaces with Other
Systems
• Big Picture Thinking (SERC MECF) ―Understand the Development in
Organizational Context
• Paradoxical Mindset (SERC Helix) ―Understand the Development in
Broader Acquisition Context
―Create Vision for the End
How does a DE environment Goal/System
enhance/complicate the ability to think ―Understand Variety of
holistically/systemically? Stakeholder Perspectives
What Tier is this? (i.e. is this specific ―Facilitate Trade-offs between
to ENG or more broad, e.g. other Stakeholder Perspectives
AQU professionals?)

SSRR 2019 November 19, 2019 35

 Example Gap: Teaming

• Building and Orchestrating a • Tasks (from models):

Diverse Team (SERC Helix) ―Build Diverse Teams
―Guide Diverse Teams
• Team Dynamics (INCOSE SECF)
―Coach Diverse Teams
• Team Building (SERC MECF) ―Identify Team Members’
[Strengths, Weaknesses,
• Working in a Team (SERC Helix) Capacities, Capabilities,
Limitations, Personalities,
Expertise, Working Styles]
How does a DE environment ―Empower Diverse Teams
enhance/complicate the ability to work
with team members? ―Build Team Trust
What Tier is this? (i.e. is this specific to ―Delegate to Team Members
ENG or more broad, e.g. other AQU
professionals?)

SSRR 2019 November 19, 2019 36

 Gaps Analysis Approach

• Critical questions:
―Are these tasks relevant/critical to ENG activities in a DE environment?
―Which Tier(s) do these tasks support?
―How should these tasks group together into new competencies?

SSRR 2019 November 19, 2019 37

 Outline

• Background & Objectives

• Methodology
• Results
―Gaps Analysis
―Taxonomy Framework
―Examples
• Next Steps

SSRR 2019 November 19, 2019 38

 Next Steps

• Review and update draft DECF based on Nov. 15th Workshop feedback
• Update DECF Competency categories to provide support of DE artifact
lifecycle
• Create draft DECF and SECCM/Eng update for Use Case validation by sponsor
• Continue work on DECF analysis, SECCM/Eng augmentation, and Gaps
analysis
• Prepare for gaps analysis of DAU curricula

SSRR 2019 November 19, 2019 39

 WRT 1006 Team

Dr. Jon Wade, PI - Stevens Institute of Technology

Dr. Dinesh Verma, Co-PI, Stevens Institute of Technology

Dr. Nicole Hutchinson, Co-PI, Stevens Institute of Technology

Dr. Mark Blackburn, Stevens Institute of Technology

Ms. Kara Pepe, Stevens Institute of Technology

Dr. Cliff Whitcomb - Naval Postgraduate School

Ms. Rabia Khan - Naval Postgraduate School

Dr. Russell Peak - Georgia Tech

SSRR 2019 November 19, 2019 40

 Questions?

SSRR 2019 November 19, 2019 41