Ada403452 PDF
Ada403452 PDF
Ada403452 PDF
TEST PLANS
NOTICE : This publication is available digitally on the Edwards AFB WWW site at: http://
www.edwards.af.mil/pim/index.html. If you lack access, contact your Publishing Distribution Office .
The purpose of this Instruction is to establish the responsibilities and procedures for the development,
review and approval of test plans for AFFTC, contractors and joint AFFTC/contractor test projects. It
applies to all units, agencies and contractors for which the AFFTC is the responsible test organization
(RTO) or participating test organization (PTO). It does not apply to USAF Test Pilot School curriculum
plans; however, it does apply to non-curriculum plans. It also does not apply to test plans for which the
AFFTC has no technical responsibility.
SUMMARY OF REVISIONS
The distribution list and office symbols were updated to reflect recent organizational changes and to add
the AFFTC assigned test representative offices. References to other guidance documents were corrected
or updated. AFFTC Form 5028, Test Project Safety Review, was added to para 4.5. Attachment 4 was
updated to reflect current office symbols and to clarify intent.
Test Plans for tests conducted at the AFFTC must be approved from a technical as well as from a safety
point of view. This Instruction addresses development, review, and approval of test plans from a technical
point of view. Safety planning for AFFTC testing is covered by AFFTCI 91-5, AFFTC Test Safety
Review.
1. General. Most test plans fall within the categories of management test plans and detailed test plans.
Management plans for major programs are normally called System Test Plans and contain flight test man-
agement concepts and general (as opposed to detailed) test plans that only identify the types of tests to be
accomplished. Test Information Sheets (TISs) are actually appendices to test plans and contain sufficient
information for use by a flight test engineer to develop flight test cards and for management to discern the
Report Documentation Page
Report Date Report Type Dates Covered (from... to)
28 Jan 2002 N/A -
Task Number
Distribution/Availability Statement
Approved for public release, distribution unlimited
Supplementary Notes
Abstract
Subject Terms
Number of Pages
15
2 AFFTCI99-1 28 January 2002
overall technical approach being taken. Therefore, TISs are frequently the key documents that describe
the specific tests to be accomplished and how they will be accomplished. In order to ensure proper and
adequate preparation and planning, a thorough technical review is required of all Test Plans and associ-
ated TISs. A major revision to an existing plan must also be reviewed. In addition, a safety review of all
TISs is required to identify potential hazards and determine minimizing procedures in accordance with
AFFTCI 91-5 for all tests utilizing AFFTC assets. Both technical and safety reviews must be completed
before testing begins.
2. References:
2.1. AFMCPAM 91-1, Flight Safety and Technical Considerations Guide for Flight Testing.
2.2. AFFTCI 91-5, AFFTC Test Safety Review.
2.3. AFI 91-202, The US Air Force Mishap Prevention Program (as supplemented).
2.4. AFMCPD 99-1, Test and Evaluation (T&E) Risk Management.
2.5. AFFTCI 99-4, Flight Test Center Deficiency Reporting.
2.6. AFFTCI 99-5, Test Control and Conduct.
2.7. AFI 99-101, Developmental Test and Evaluation.
2.8. AFI 99-106, Joint Test and Evaluation.
2.9. AFFTC-TIH-93-01, Air Force Flight Test Center Test Plan Preparation Guide.
3. Responsibilities.
3.1. Development of Plan. The designated project manager (PM) from the RTO is responsible for
ensuring that AFFTC personnel prepare or make inputs to test plans as soon as possible. AFFTC per-
sonnel should also prepare or make thorough and timely inputs to detailed TISs through integrated
product team meetings, test plan working groups or other means. The PM is also responsible for
ensuring that test plans are prepared and submitted to the Office of Primary Responsibility (OPR) for
Technical Review in time to allow comprehensive technical and safety reviews and to meet program
requirements and schedules. The PM is also responsible to assure that test plans are properly coordi-
nated and ready for the required reviews. Copies of the plan will be provided as soon as it is ready for
a technical review but normally no later than six weeks before the start of testing and no later than five
working days before the scheduled technical review.
3.2. Establishment of a Technical Review and Technical Review Board (TRB).
3.2.1. All test plans shall receive a technical review. The technical review may take any form
from the Division Chief or 2-letter level individual reviewing it themselves, to a formal TRB with
numerous stakeholders present. The intent of a technical review is to establish a committee of
experienced personnel not intimately associated with the project to provide an independent techni-
cal assessment and executive review of the plan. It is the responsibility of the OPR to ensure an
adequate review of the plan from a technical standpoint. The OPR for technical review should
have expertise in all the test areas that will allow adequate review of the plan. If the OPR does not
have sufficient expertise in the appropriate areas the OPR shall request assistance from the appro-
priate AFFTC organizations. All appropriate organizations will support the technical reviews as
requested by the OPR and PM.
AFFTCI99-1 28 January 2002 3
3.2.1.1. Normally, the OPR will be one of the following: 412 TW/CA, 412 TW/EN, 412 TW/
EW, 412 TW/ENF, 412 TW/ENR, 412 TW/ENV, 412 TW/EWD, 412 TW/EWW.
3.2.1.2. Other equivalent level offices may exercise OPR authority for test projects under
their management, so long as adequacy and independence of the technical assessments are
maintained.
3.2.1.3. The technical review process is started by the PM submitting the test plan to 412 TW/
CA who will assign the appropriate OPR. The OPR will normally be the three-letter organiza-
tion (division) that has the bulk of the testing in the plan. In cases where a test plan incorpo-
rates many different disciplines or has high political visibility, an OPR higher than the division
level may be chosen. The two-letter organizations always maintain the option to review the
test plans themselves. See attachment 4 for detailed instructions.
3.2.2. The OPR will determine whether or not a formal TRB is required. A formal TRB is a
face-to-face meeting that includes project personnel and the test plan reviewers. That decision will
be based on the OPR's judgment about adequacy of the technical planning, similarity to previously
reviewed plans, complexity of the proposed test, and similar aspects. If a TRB is required, the
OPR will ensure that the board is made up of experienced personnel to include at least operations
and engineering personnel. These personnel should be chosen on the basis of their experience in
areas addressed in the plan. The expertise should not be limited to that dictated by the test program
objectives, but should be broad enough for these individuals to critically review all aspects of the
planned test. The PM and operations and engineering personnel responsible for conducting and
supporting the test program will participate in the review as consultants. Appropriate contractor
representatives should also be invited. The OPR should ensure that notification of the time, date,
and location of the TRB meeting and the test plan are distributed at least five working days before
the meeting.
3.2.3. If a TRB is not deemed necessary, the OPR should ensure that any feedback is provided to
the responsible test agency. Test agencies can hold their own preliminary review of plans, but this
does not constitute the independent review required by this instruction.
4. Procedures.
4.1. Overall Process. Attachment 1 shows the overall process for detailed test plan and/or TIS devel-
opment. The following paragraphs amplify significant parts of the process.
4.2. Preparation of the Plan. In preparing the test plan the author should refer to Attachment 2 and ref-
erence 2.7.
4.2.1. The RTO will ensure that a test plan is prepared to meet the objectives mutually agreed to
by the AFFTC, the Program Management Office (PMO) or other customer, and participating
agencies. Specifications, operational requirements, and management plans, such as Test and Eval-
uation Master Plans, Project Management Plans and the System Operation Requirement Docu-
ment (SORD) should be used as sources for objectives, issues and items that require management
emphasis. If necessary, Test Plan Working Group sessions should be held with appropriate orga-
nizations, including contractors, to ensure a plan is ready for a technical review.
4.2.2. A TIS will be used to define and amplify specific tests identified in a system test plan and
will either stand alone or be a part of an overall plan that is general in nature. In either case, a TIS
will have sufficient information for a flight test engineer to develop flight test cards, and for man-
4 AFFTCI99-1 28 January 2002
agement to discern the overall technical approach being taken. Attachment 3 lists the information
required.
4.2.3. For AFFTC TISs, AFFTC Form 5232b, AFFTC Test Information Sheet (with signature
blocks), will be used as the cover sheet for stand-alone plans and AFFTC Form 5232a, AFFTC
Test Information Sheet, for TISs that are included in overall plans. The AFFTC Form 5232b will
also be used as a cover sheet for contractor-prepared detailed test plans which come under the pur-
view of this regulation. The TIS will contain background and authority information for the test to
be performed, leading the reader to better understand the test plan attached to the form. AFFTC
project managers will prepare the forms in the appropriate number of copies for the test agency.
Copies are distributed to the management office, contractor, and using command as needed. Inter-
nal distribution is made as determined by the project manager.
4.3. Review of the Plan.
4.3.1. The OPR for Technical Review shall conduct a technical review as specified in the respon-
sibilities section of this OI and follow the procedures as specified below.
4.3.2. If a formal TRB is convened then each TRB will start with a briefing by personnel from the
test organization. The briefing will include a summary of the test objectives, status of the develop-
ment of the test item and operational limitations, proposed overall test procedures, instrumenta-
tion, data requirements and the philosophy used in establishing these items. Data handling and
analysis plans will also be briefed. Potentially high risk tests from a technical standpoint will be
identified and their relationship to sequence of testing and known development problems will be
addressed.
4.3.3. A detailed technical review of the test plan will then be accomplished. This discussion will
cover the program objectives, status of preparation and planning, executability of the plan, pre-
dicted test item characteristics associated with individual tests, review of the test matrix and test
procedures, adequacy of the test effort, technical go/no-go criteria, alternate courses of action,
resources needed and available, and other items important to test planning. Preliminary aspects of
safety planning for the test program should be discussed. Attachment 2 contains a checklist to aid
in the process. Action items and recommendations will be identified. These could include addi-
tional studies or analyses, qualification tests or unique recommended training or the addition and
deletion of test points. The TRB minutes will contain an overall assessment of the plan and
address each of the above items as appropriate. The TRB minutes, including a list of attendees,
will accompany the plan during final coordination and approval.
4.4. Finalization and Submission of Plan for Approval.
4.4.1. The OPR will write a letter to the test organization stating who reviewed the test plan and
attach the TRB minutes, recommendation, action items or comments as appropriate (see Attach-
ment 4 for detailed policy and instructions). The RTO will modify the plan to incorporate the
appropriate recommendations from the technical review. Any recommendations or action items
not incorporated will be addressed in a memorandum for the record explaining the reason for the
noncompliance.
4.4.2. Following the technical review, a package will be prepared for submission by the test orga-
nization to include the test plan, signature page, and memorandum for the record explaining rea-
sons for noncompliance with recommendations or action items. These documents will normally be
included in the Test Documentation Package as specified in AFFTCI 91-5. If there is noncompli-
AFFTCI99-1 28 January 2002 5
ance with technical review recommendations, then it is highly advisable to coordinate with the
technical review OPR in advance of submitting the Test Documentation Package to prevent delays
in the approval cycle.
4.5. Plan Approval. Approval of the test plan will normally be accomplished concurrently with the
approval of AFFTC Form 5028 as described in AFFTCI 91-5. The approval authority for the test plan
is the same as for the AFFTC Form 5028. This does not preclude early coordination of the test plan
itself for concurrence by coordinating officials. Coordinating officials will sign the AFFTC Form
5232b, Test Information Sheet (TIS) sheet, indicating their concurrence or non-concurrence, and
whether they have comments. Any comments during coordination will be written on the back of the
TIS sheet and signed. Approval of the test plan and Form 5028 (reference AFFTCI 91-5) will be
obtained before testing. The PM will ensure that a copy of the approved Test Plan and/or TIS is filed
in the AFFTC Technical Library for reference and historical information.
4.6. Distribution of Plan. Following approval of the plan, distribution will be made to organizations
requiring the plan for participation or support.
5. Test Plan Changes. Before or during a test program, changes can occur which necessitate a revision
of the original test plan or TIS. These changes could be due to unexpected simulator results or flight test
results, unexpected safety impacts, changes in management emphasis or specification requirements, or
other causes. The PM is authorized to determine whether a revision is minor or major. Minor changes
include changing flight conditions of test points as long as they are within the envelope of test points
approved in the original plan, adding test points within the envelope of test points approved in the original
plan, deleting test points if preliminary results show they are unnecessary and are not part of a safety build
up.. A major revision is defined to be any change to test objectives, technical approach or test methodol-
ogy, or substantive changes to test procedures. If there is any question about whether a change is minor or
major the PM should consult with senior supervisory personnel. A minor change does not require a new
technical review. For a major change, the technical review and approval process is identical to that for the
original test plan. The PM shall document the change and reasons for it. Based on this documentation, the
OPR will determine whether a TRB is required. In determining the need for a TRB, the OPR will consider
the extent of the changes, results of testing to date, predicted results of proposed testing, and other perti-
nent details. Regardless of the need for a TRB, a Test Documentation Package must be prepared for the
test plan revision in accordance with AFFTCI 91-5. If a TRB is not required, the same letter mentioned in
para 4.4.1, from the OPR to the test agency stating this , must be included in the Test Documentation
Package.
6. Forms Prescribed:
6.1. AFFTC 5232a, AFFTC Test Information Sheet (without signatures). See attachment 3.
6.2. AFFTC 5232b, AFFTC Test Information Sheet (with signatures).
Attachment 1
AFFTC TEST PLAN PROCESSING
TRB
COORDINATION CHAIRPERSON PARTICIPANTS,
& OTHER RESPONSIBLE
AND REVIEW SAFETY OFFICE
PARTICIPANTS TST
APPROVAL PLAN & OPR
AGENCY
REFER TO REVIEW ASSIGN
PURSUE ACTIONS RECORD
AFFTCI91-5 APPROVE ATTENDEES
FINALIZE ACTIONS
TEST PLAN SAFETY
PACKAGE ACKNOWLEDGE
PREPARE PREPARE
MEETING MINUTES REC OF NOTICE
COMPLIANCE LTR
ATTEND
ASSEMBLE
TRB
TEST PLAN AND
MEETING
SAFETY PLANNING
DOCUMENTATION
(REFER TO
AFFTCI91-5
RESPONSIBLE
TEST
AGENCY
DISTRIBUTE
ALL
PLANS
AFFTCI99-1 28 January 2002 7
Attachment 2
TEST PLAN CHECKLIST
Project Management
- Organization Has the overall project management and organization been
identified
- Schedule -- Has a schedule been prepared showing all major mile-
stones?
8 AFFTCI99-1 28 January 2002
Test Objectives
- General - Do AFFTC objectives cover critical developmental and
operating concerns?
- Specific - Have specific measures of performance been identified for
each objective?
- Evaluation Criteria - Are operating concerns shaped by system effectiveness or
user employment?
- What will define satisfactory system performance?
- Why is test required?
- Are test objectives clearly defined?
- Do objectives fall within AFFTC mission?
- To what extent do AFFTC objectives differ from or dupli-
cate contractor objectives?
- Is additional info required from management office?
- Are the objectives achievable and complete?
- What is required in management directives & plans (e.g.,
AFI 99-101, Test and Evaluation Master Plan, System Test
Plan)?
- Are objectives prioritized in order to ensure some success
on extremely limited schedules?
- What are success criteria? Do they give a measurable de-
termination? Do you know when the test is completed?
- Are expected results kept out of the wording of the objec-
tives?
AFFTCI99-1 28 January 2002 9
Instrumentation System
- General description - Are planned instrumentation systems adequate (ground,
flight, and data systems)?
- Parameters - Is adequate time available to thoroughly check instrumen-
tation/data production cycle before first data flight?
- Calibration procedures - What are go/no-go parameters for both safety-of-flight
and technical requirements?
- Are proposed accuracy’s and sensitivities adequate? Are
system noise levels attenuated to an acceptable level?
Data Processing and Analysis
Requirements
- Data requirements - Is sufficient time allowed between tests for data turn-
around and analysis?
- Data distribution plan - Are the data processing procedures in place?
- Processing and analysis plan - Has analysis plan been formulated? Will the data address
the test objectives?
- What are requirements for encrypted data?
- To what extent will government analysis be independent
from contractor's?
- Are routines being developed for each of the various gen-
eration levels of analysis and will they be ready when re-
quired? How will they be verified?
- Have algorithms been defined in order to answer each
measure of performance?
AFFTCI99-1 28 January 2002 11
(1) This checklist is a guideline for the type of information that should be considered during test plan
preparation. The basic topics (General, Test Item Description, etc.) should be considered for all plans.
Appropriate sub-items can be used or combined as appropriate.
(2) This checklist is intended to ensure that all relevant areas are specifically addressed. Some of the ques-
tions may not have clear answers but as a minimum should be surfaced and understood
AFFTCI99-1 28 January 2002 13
Attachment 3
SAMPLE TIS SHEET
14 AFFTCI99-1 28 January 2002
AFFTCI99-1 28 January 2002 15
Attachment 4
TW/CA LETTER, DETAILING REVIEW PROCEDURES