Engineering Procedure

SAEP-26 7 June 2017

Capital Project Benchmarking Guidelines
Document Responsibility: Project Management Office Standards Committee

Contents
1 Introduction ....................................................... 2
2 Applicable Documents ...................................... 3
3 Methodology ..................................................... 4
4 Responsibility/Workflow .................................... 6
5 Data Collection.................................................. 9
6 Reporting and Analysis ................................... 10
7 Exhibits ........................................................... 12
Revision Summary................................................. 12

Exhibit I - Review Phase and Required

Benchmarking Metric Matrix .................. 13

Previous Issue: 23 April 2013 Next Planned Update: 7 June 2020

Page 1 of 32
Contact: Doiron, Shannon E. (doironse) on phone +966 13 880-9161

©Saudi Aramco 2017. All rights reserved.

Document Responsibility: Project Management Office Standards Committee SAEP-26
Issue Date: 7 June 2017
Next Planned Update: 7 June 2020 Capital Project Benchmarking Guidelines

1 Introduction

1.1 Purpose

This engineering procedure serves as the framework to ensure that the

Benchmarking Process is a continuous and seamless process and is incorporated
as an integral part of the planning and execution of Saudi Aramco Capital
Projects and reach parity with Best in Class. Saudi Aramco has an imperative to
transform corporate performance into “Best in Class.” To reach full parity with
the “Best in Class”, the company must constantly “Measure (against the
Industry), Assess, and Improve” and keep repeating the process. It also
measures the company performance with the Industry in the same line of
business. Benchmarking process will focus on identifying aspects of the project
that are not in line with the industry.

1.2 Definition

A Benchmarking Study is performed on a project to compare that project’s cost,

schedule, FEL (Front-End Loading), Operability, etc., to similar projects within
Saudi Aramco as well as the industry. The benchmarking process will be
centralized so that all benchmarking activities are managed through the Project
Management Office Department (PMOD). PMOD will consult with the
Integrated Project Team (IPT), CPED, and SAPMT to identify projects that will
be benchmarked during FEL and at mechanical completion of a project.

1.2.1 Internal Benchmarking

Internal Benchmarking is one of the processes required in preparation of

a Company estimate. The IPT is responsible for providing the
benchmarking data to PMOD as part of the ER Estimate Package.
This data includes but is not limited to the key quantities, specific project
metrics, project type/subtype and work breakdown structure (WBS)
format as required in SAEP-25, Estimate Preparation Guidelines.

1.2.2 External Project Benchmarking

External project benchmarking analysis based on international accepted

methodology is performed on selected projects by an outside consultant.
This benchmarking analysis is intended to help the project team assess
the completeness of the project scope if carried out at FEL 2 and/or
FEL 3 and measures the performance of the project against set criteria if
carried out towards the end of the execution phase. The benchmarking
analysis at FEL 2 and/or FEL 3 provides further clarity on project
readiness for the next phase. This guideline explains in detail criteria for
Saudi Aramco: Company General Use
Page 2 of 14
Document Responsibility: Project Management Office Standards Committee SAEP-26
Issue Date: 7 June 2017
Next Planned Update: 7 June 2020 Capital Project Benchmarking Guidelines

selection, methodology, timing, data collection, and reporting/analysis

for external project benchmarking.

1.2.3 External System Benchmarking

System benchmarking analysis taking into account Saudi Aramco

specific processes and standards is performed on selected projects prior
to mechanical completion by an outside consultant based on type and
size of the project. A number of projects nearing completion will be
selected by PMOD for benchmarking every year or as necessary, using a
mix of various project types, sub-types and sizes. Select projects may be
benchmarked throughout their project life cycle for the specific purposes
noted in Section 3.2 below. This sample provides a historical overview
of Saudi Aramco’s capital projects performance over the life cycle of the
projects. Selected projects may be at various stages of execution.
In this study “key” inputs mutually agreed between PMOD and the
benchmarking consultant as drivers of project performance of a
representative cross-section of projects are compared against industry-
wide sample of projects, comparable in size and complexity.

These “key” inputs include at least the level of project definition quality,
or front end loading (FEL), the role of value improving practices, and the
level of team development.

2 Applicable Documents

The latest edition of the following applicable reference documents shall be applied:

 Saudi Aramco Engineering Procedures

SAEP-12 Project Execution Plan (PEP)
SAEP-14 Project Proposal (PP)
SAEP-25 Estimate Preparation Guidelines
SAEP-40 Value Assurance Process
SAEP-1350 Design Basis Scoping Paper (DBSP) Preparation
and Revision Procedure

 Saudi Aramco General Instructions

GI-0020.500 Expenditure Requests
GI-0216.965 2017 – 2019 Cost Distribution Rates

Saudi Aramco: Company General Use

Page 3 of 14
Document Responsibility: Project Management Office Standards Committee SAEP-26
Issue Date: 7 June 2017
Next Planned Update: 7 June 2020 Capital Project Benchmarking Guidelines

3 Methodology

3.1 Project Selection

Saudi Aramco will internally benchmark selected capital projects using the
internal database which has been developed by PMOD. However, PMOD will
coordinate with the IPT (where applicable during FEL 2/DBSP) to identify
projects (if any) to be externally benchmarked. PMOD will perform the leading
role for the selection of the Consultant and will coordinate with the IPT (where
applicable during FEL 2/DBSP) in the development of the benchmarking criteria
and in the implementation of the formal benchmarking study.

The analysis shall commence at the appropriate stage of the project, comparing
key parameters of the project with an external industry-wide database. The risk
analysis will address key project outcomes and associated risks based on the
available information. PMOD will make the determination of what projects will
be subject to either external or internal benchmarking.

External Benchmarking Selection Criteria

PMOD will coordinate with the IPT (where applicable during FEL 2 and FEL 3)
to identify Types A, B, C, and/or C1 projects to be benchmarked.

The following projects are excluded:

 Pipeline rehabs
 Master Appropriations
 Marine vessel projects
 Utilities as stand-alone projects

3.2 Benchmarking Stages

Benchmarking on selected Types A, B C and/or C1 projects may be done at the

following stages:
 At end of FEL 2 (DBSP) stage
 At end of the FEL 3 (Project Proposal) stage
 At Project Close-out (before Mechanical Completion)
 Operability Review after facility has been in operations for 12 months as
deemed necessary.

Saudi Aramco: Company General Use

Page 4 of 14
Document Responsibility: Project Management Office Standards Committee SAEP-26
Issue Date: 7 June 2017
Next Planned Update: 7 June 2020 Capital Project Benchmarking Guidelines

3.3 Work Breakdown Structure

Benchmarking will reflect the Work Breakdown Structure (WBS) and Saudi
Aramco’s project types and subtypes as defined for the project.

The Consultant’s Report Format will be structured to present a balanced

approach between Scope/Cost/Schedule Data and other Value Added and Best
Practices. The Benchmarking process and report must be structured, organized
to provide benchmarks and recommendations according to the following
structure:

3.3.1 Project Output Capacity

Saudi Aramco projects must be benchmarked with similar projects in the

industry taking into account the project capacity output. Saudi Aramco
is interested in a benchmark of a data set with similar project capacity
versus a potentially larger dataset with capacity not indicative of the
project being benchmarked.

3.3.2 Project Type and Sub-type

The benchmarking process needs to select only industry projects and

process units that are similar to the project types and WBS.

3.3.3 Project Location

The benchmarking analysis needs to reflect datasets for projects in

similar locations and provide benchmarks that evaluate Saudi Aramco’s
performance as it compares to:
 Saudi Aramco standard vs. Industry
 Gulf Cooperation Council (GCC) countries (these include: Kingdom
Saudi Aramco: Company General Use
Page 5 of 14
Document Responsibility: Project Management Office Standards Committee SAEP-26
Issue Date: 7 June 2017
Next Planned Update: 7 June 2020 Capital Project Benchmarking Guidelines

of Saudi Arabia, Kuwait, Bahrain, Qatar, the United Arab Emirates,

and the Sultanate of Oman). (OPTIONAL)
 Best in Class

3.3.4 Inside/Outside Battery Limits

The benchmarking data needs to differentiate between the Inside Battery

Limits (ISBL - defined as all equipment and associated components
(piping, etc.) that act upon the primary feed stream of a process. ISBL is
functional-based and refers to equipment and other components that are
solely dedicated to a single process whether or not the equipment is
physically located within the geographical boundaries of the unit.) and
Outside Battery Limits (OSBL) parts of the project where relevant.
The dataset and the benchmark report need to provide performance and
recommendations separately for these two areas where relevant.
The report must be structured as such.

3.3.5 Grass Root or Revamp/Expansion

The benchmark report must identify whether the project is a grass roots
project or an expansion / revamp of an existing facility. The industry
projects used for benchmarks must be of similar nature as the Saudi
Aramco project being benchmarked.

3.3.6 Project Exclusions

The benchmark report and analysis must define excluded factors specific
to Saudi Aramco such as long transmission lines, excessive infrastructure
outside the OSBL, long roads or highways to reach the site, security
fences, flood lights, etc.

3.3.7 Normalization

The benchmark report and analysis shall clearly state the normalization
process used including (but not limited to) timeline, location, year of
funding, project execution methodology, specific owner requirements,
safety and standard requirements.

4 Responsibility/Workflow

4.1 Responsibility

PMOD shall be responsible for the selection of projects to be benchmarked,

for data collection, and for coordination with the IPT, Benchmarking Consultant
and other parties. External benchmarking studies will be conducted by the
Saudi Aramco: Company General Use
Page 6 of 14
Document Responsibility: Project Management Office Standards Committee SAEP-26
Issue Date: 7 June 2017
Next Planned Update: 7 June 2020 Capital Project Benchmarking Guidelines

Consultant in accordance with the benchmarking workflow discussed in

Section 4.2. PMOD will share the draft report with the IPT (where applicable
during FEL 2/DBSP), and discuss and agree for any comments and/or measures
necessary to implement the consultant’s recommendations. PMOD will
coordinate the response to the Benchmarking Consultant who will then issue the
final agreed report.

4.2 Benchmarking Workflow

The following sections outline the workflow that will normally be followed to
implement benchmarking:

4.2.1 Identify candidate projects

PMOD will identify suitable candidate projects to undergo either

external or internal benchmarking and identify timing and frequency to
allow benefits from feedback. Projects will be selected based on project
type and size.

4.2.2 Select projects for Benchmarking

PMOD will review the candidate projects with SAPMT (and FPD where
applicable for FEL 2) and agree on selected projects for external
benchmarking and determine a likely start date for initiating the
benchmarking process. The IPT and/or Construction Agency must
appoint a designee for contact and coordination. This person will usually
be the Senior Project Engineer or IPT Lead.

4.2.3 Establishment of a Work Order through ASC

PMOD will establish a work order with the Benchmarking Consultant.

A contract will be set up between Aramco Services Company (ASC) and
the Benchmarking Consultant to serve this need. The consultant fees
will be paid out of TC-68 funds reserved for early FEL benchmarks and
from approved ER funds for projects under execution.

4.2.4 Data Gathering Cycle

Required metrics and data will be collected for the benchmarking effort
at any given stage of a project, FEL 2 (DBSP), FEL 3 (Project Proposal)
and Project Close-Out.

The result of the Benchmarking Metrics (Exhibit I) for each type of

project shall be collected and stored in Saudi Aramco’s Enterprise
Project Management System (EPM).

Saudi Aramco: Company General Use

Page 7 of 14
Document Responsibility: Project Management Office Standards Committee SAEP-26
Issue Date: 7 June 2017
Next Planned Update: 7 June 2020 Capital Project Benchmarking Guidelines

All cost and schedule data will be provided by PMOD and the relevant
Construction Agency according to availability. Any additional data,
e.g., schedules (Level 1, 2 or 3 [Primavera format]), construction
productivity, quantities, constructability, safety, contractor data, best
practices, value improvement practices, etc., will be obtained from the
Construction Agency or FPD, as required.

On receipt of the Work Order the benchmarking consultant will prepare,

in addition to the standard questionnaire, a Saudi Aramco-specific
questionnaire. This questionnaire will be prepared based on the agreed
metrics appropriate to the project status. The questionnaire will be
issued to PMOD who will coordinate its timely completion with the IPT
and/or Construction Agency as appropriate. It is expected that this
questionnaire will be complementary to the data collected and additional
input required by the consultant to perform his analysis.

Upon completion of the questionnaire, PMOD and the designated IPT

and or Construction Agency contact person will forward this data to the
benchmarking consultant. An interview of the Project Team by the
benchmarking consultant will be scheduled at an agreed time and
location.

4.2.5 Review Cycle

Following the interview, the Benchmarking Consultant will prepare his

report and submit it to PMOD, who will distribute it to the relevant
stakeholders. It is expected that up to four weeks would normally be
required to prepare the report. At approximately two weeks an interim
progress meeting will be held at the consultant’s office or other locations
mutually agreed. The purpose of interim progress meeting is to provide
early insight into any issues arising, to address any further queries the
benchmarking consultant may have, and to enable PMOD, IPT and other
stakeholder to respond in a timely manner to the benchmarking
consultant’s queries.

4.2.6 Draft and Final Report

All benchmarking requests and draft or final reports should compare

Saudi Aramco’s deliverables with the industry during all the
benchmarking phases, FEL 2 (DBSP), FEL 3 (Project Proposal), and
Execution.

Draft and Final Reports will specifically address the following issues:
 Assessment of scope clarity;

Saudi Aramco: Company General Use

Page 8 of 14
Document Responsibility: Project Management Office Standards Committee SAEP-26
Issue Date: 7 June 2017
Next Planned Update: 7 June 2020 Capital Project Benchmarking Guidelines

 Completeness and quality of data available at preceding project

phases (FEL 2, 3) if any;
 Scope evolvement between FEL 2 (DBSP) and FEL 3 (Project
Proposal);
 Comparison of Saudi Aramco with Industry averages as well as Gulf
Region averages; (Optional)
 Assess market conditions impact on projects;
 Descriptions of comparable baseline projects in the consultant’s
database;
 Benchmarking reports need to share a common format to ensure
consistency of presentation;
 Items unique to Saudi Aramco should be identified and excluded
from the project benchmarking.

PMOD will work with the IPT to evaluate the Consultant’s comments
and /or recommended measures for the project to implement the
consultant’s recommendations. PMOD will coordinate the response to
the Benchmarking Consultant who will then issue the final agreed report.

4.2.7 Completion of the Study

Upon completion of the study and agreement, a plan will be developed

by the relevant stakeholders (led by CPED or the Construction Agency
where appropriate) to implement any recommendations that may be
accepted.

5 Data Collection

PMOD will be the main contact and coordinator for collecting the data to perform the
benchmarking for all projects. The data collected for each phase needs to be tailored to
the phase at which the benchmarking is done. Thus, the questionnaire provided by the
benchmarking consultant needs to be specific to the phase at which the project is
benchmarked.

Saudi Aramco will provide the following key data at each phase:

5.1 FEL 2 (DBSP) Phase

 Copy of draft DBSP;
 Milestone Schedule;
 Budget Estimate broken down according to project WBS;

Saudi Aramco: Company General Use

Page 9 of 14
Document Responsibility: Project Management Office Standards Committee SAEP-26
Issue Date: 7 June 2017
Next Planned Update: 7 June 2020 Capital Project Benchmarking Guidelines

 Available engineering data;

 Answers to the prospective consultant benchmarking questionnaire.

5.2 FEL 3 (Project Proposal) Phase

 Copy of project Proposal/Final DBSP;
 Level III Schedule;
 Estimating data (ER Estimate) broken down accordingly to project WBS.
 Proposal contractor data;
 Answers to the consultant prospective benchmarking questionnaire.

5.3 Close out Phase

 Copy of close-out report;
 Detailed CPM schedule;
 Actual cost organized according to WBS model;
 Available engineering/construction data;
 Answers to the consultant prospective benchmarking questionnaire.

6 Reporting and Analysis

The Benchmarking Report should be organized according to the WBS set by Saudi
Aramco in the methodology section of this procedure. Furthermore, the report must
contain all the metrics and analysis set forth in the results and outcome sections of this
report.

All benchmarking requests and draft/ final reports should review Saudi Aramco’s
deliverables vs. the industry during all the benchmarking phases, FEL2 (DBSP), FEL 3
(Project Proposal), and Execution.

6.1 Preface

6.2 Executive Summary

 Project Benchmarks;
 Other Project Issues;
 Conclusions;
 Recommendations.

6.3 Introduction
 Project Background;

Saudi Aramco: Company General Use

Page 10 of 14
Document Responsibility: Project Management Office Standards Committee SAEP-26
Issue Date: 7 June 2017
Next Planned Update: 7 June 2020 Capital Project Benchmarking Guidelines

 Scope of Work and Project Technology, Assessment of scope clarity;

 Retrospective evaluation of completeness and quality of data available at
preceding project phases (FEL 2, 3) if any;
 Other Project Issues including Contracting Strategy;
 Description of Benchmark Projects and basis for their selection as
benchmarks.

6.4 Project Drivers

 Front End Loading;
 Value Improvement Practices;
 Project Management Practices;
 Contracting Strategies;
 Target Setting Deliverables (where applicable)
 Team Development Index;
 Estimating / Planning for Control;
 Execution Strategy;
 Market Conditions Impact;
 Key Quantities.

6.5 Project Outcomes (taking contracting strategy into account)

6.5.1 Cost
 Contingency Allocation/Use;
 Cost Performance;
 Cost Capacity Model Results;
 Cost Effectiveness;
 Location Factors including Construction Productivity;
 Cost Growth/Predictability.

6.5.2 Project Schedule

 Cycle Time;
 Execution Schedule;
 Construction Schedule;
 Start-up duration;
 Safety Performance.

Saudi Aramco: Company General Use

Page 11 of 14
Document Responsibility: Project Management Office Standards Committee SAEP-26
Issue Date: 7 June 2017
Next Planned Update: 7 June 2020 Capital Project Benchmarking Guidelines

6.5.3 Operational Performance

 Impact of Major Late Design Changes;
 Turnover of Key Personnel;
 Stage Gate Effectiveness;
 Achieved Internal Rate of Return.

6.6 Conclusions and Recommendations

 Conclusions
 Recommendations

6.7 Appendix A: Basis for Analysis and Location Adjustments

6.8 Appendix B: Current Estimate of Cost

6.9 Appendix C: Location Adjustment

6.10 Appendix D: Current Estimated Schedule

7 Exhibits

Exhibit I - Review Phase and Required Benchmarking Metric Matrix

Revision Summary
23 April 2013 Revised the “Next Planned Update” to occur after ATP recommendations. Reaffirmed the
content of the document, and reissued with editorial revision to revise department name.
7 June 2017 Major revision to incorporate CMS processes and reaffirm content.

Saudi Aramco: Company General Use

Page 12 of 14
Document Responsibility: Project Management Office Standards Committee SAEP-26
Issue Date: 7 June 2017
Next Planned Update: 7 June 2020 Capital Project Benchmarking Guidelines

EXHIBIT I -
Review Phase and Required Benchmarking Metric Matrix

Arabian Gulf Region

Project Close Out

Industry Average
Saudi Aramco
FEL 2 (DBSP)

Best in Class
FEL 3 (PP)
SCHEDULE
Feed Duration/Total Direct Construction Man-hours * * * * * * *
Execution Duration/Capacity * * * * * * *
Overall Duration/Capacity * * * * * * *
FEED Duration/Overall Duration * * * * * * *
Detailed Eng. Duration/Overall Duration * * * * * * *
Construction Duration/Overall Duration * * * * * * *
Shutdown Duration * * * * * * *

PRODUCTIVITY
Direct Detailed Engr. Man-hours / Quantity (by discipline) * * * * *
Direct Const. Man-hours/Quantity (by discipline) * * * * *
Direct Eng. Rework Man-hours/Total Direct Eng. Man-hours * * * * *
Direct Const. Rework Man-hours/Total Direct Constr. Man-
* * * * *
hours
Total FEED Eng. Man-hours/Capacity * * * * * *
Total FEED PMT Man-hours/Capacity * * * * * *
Total PMT Man-hours/TIC * * * * *
Detailed Engr Man-hours/TIC * * * * *
Total Direct Const Man-hours/TIC * * * * *
Process Eng. Man-hours * * * * *
PMT Man-hours/Eng. Man-hours * * * * *

SAFETY
Total Recordable Incident Rate * * * * *

Saudi Aramco: Company General Use

Page 13 of 14
Document Responsibility: Project Management Office Standards Committee SAEP-26
Issue Date: 7 June 2017
Next Planned Update: 7 June 2020 Capital Project Benchmarking Guidelines

Arabian Gulf Region

Project Close Out

Industry Average
Saudi Aramco
FEL 2 (DBSP)

Best in Class
FEL 3 (PP)
Dart Rate * * * * *

OTHER
Complexity * * * * * * *
Facility Location * * * * * * *
Contract Strategy for FEED * * * * * * *
Contract Strategy for EPC * * * * * *
Project Team Size * * * * * * *
Owner Team Size (FTE) and Composition * * * * * *
PMT Turnover * * * * * *
Equipment Count/Capacity * * * * * * *
Construction Quantity/Capacity * * * * * *
Plot Area/Capacity * * * * * * *
Avg. Constr. Craft Hours/Week * * * * *

PROCESS TYPE PARAMETERS

Hydrotreaters * * * * * * *
Recycle Rate * * * * * * *
Design Pressure * * * * * * *
ppm Sulfur Product * * * * * * *
ppm Sulfur Feed * * * * * * *
ppm Nitrogen product * * * * * * *
ppm Nitrogen feed * * * * * * *
Hydrogen consumption * * * * * * *
# of Reactors * * * * * * *

Saudi Aramco: Company General Use

Page 14 of 14