Chapter 9

Testing and evaluation

of systems

Use with The Audit Process: Principles, Practice and Cases, 6th edn
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015
 Learning objectives

• To suggest audit and systems objectives for selected

components of the financial statements.
• To evaluate systems in use in selected areas and draw up
audit conclusions.
• Explain the role of tests of control, and in particular those
used to test computer systems.

Use with The Audit Process: Principles, Practice and Cases, 6th edn
2
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015
 Case studies: setting system objectives, use of flow charts,
identification of key questions, and evaluation of systems (1)

• Case study 9.1 Broomfield plc: integrated computerized sales, trade

receivables, cash and inventory system.

1.What do you think are the objectives of this system? Phrase them as
audit objectives.
2.Identify control points in this system and suggest controls that should
be in force.
3.Refer to the suggested key and subsidiary questions for receipt of
sales order and credit control and decide how you would evaluate the
parts of the system relating to these aspects.
4.There are at least two matters that might affect the inventory figure
appearing in the financial statements. Identify these matters and
suggest how the company might overcome them.

Use with The Audit Process: Principles, Practice and Cases, 6th edn
3
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015
 Setting system objectives, use of flow charts,
identification of key questions and evaluation of
systems (2)

• Case study 9.2 Troston Limited: integrated

production and production wages system.

1.State the broad objectives of the wages system.

2.List the key questions in the wages area.
3.Review the payroll system of Troston plc and
comment on the strengths and weaknesses you
have identified.

Use with The Audit Process: Principles, Practice and Cases, 6th edn
4
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015
 Setting system objectives, use of flow charts,
identification of key questions, and evaluation of
systems (3)
• Case study 9.3 Broomfield plc: part of computerized
purchases and trade creditors system.
1.State the broad systems objective of a purchases and related
trade payables system.
2.Redraft the part of the flow chart as Ivor does not always
send out an order on the basis of the requisition and has the
authority to change inventory reorder levels.
3.Suggest key questions in the following areas
1. Requisitioning
2. Purchasing
3. Receipt of goods and services
4. Processing of suppliers invoices
5. Entry in purchases ledger
6. Entry in general ledger
4.Evaluate the system you have recorded.
Use with The Audit Process: Principles, Practice and Cases, 6th edn
5
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015
 Setting system objectives, use of flow charts,
identification of key questions, and evaluation of
systems (4)
• Case study 9.4 Burbage Limited: sales order
processing system and identification of application
controls
• Analyse the Burbage Limited sales order
processing system and identify areas of significant
weakness in internal control and suggest steps the
company should take to rectify any weaknesses.
Your answer should be framed under:
– application controls:
(a) input (b) processing (c) output
– other matters

Use with The Audit Process: Principles, Practice and Cases, 6th edn
6
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015
 Case studies:
setting system
objectives, use of
flow charts,
identification of
key questions and
evaluation of
systems (5)

Figure 9.1

Use with The Audit Process: Principles, Practice and Cases, 6th edn
7
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015
 Case studies:
setting system
objectives, use
of flow charts,
identification
of key
questions and
evaluation of
systems (6)

Figure 9.2

Use with The Audit Process: Principles, Practice and Cases, 6th edn
8
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015
 Case studies: setting system objectives, use of flow charts,
identification of key questions, and evaluation of systems (7)
Figure 9.3
A. Suppliers’ records are updated using a terminal
in the buying department.
B. The purchases budget is prepared by Ivor
Jordan and stock reorder levels set (annually).
C. When reorder level is reached the computer
automatically prepares purchase requisitions
which form the basis for preparation of the
purchase order by Ivor Jordan.
D. There is formal transfer of purchase orders and
control totals pre-list and control logs are
signed by both Jordan and Owler.
E. The exception report is used by Owler to make
corrections in a separate purchase order run.
F. Copy 4 of the official purchase order becomes
a goods received note, that is passed to Janet
Black for matching with purchase invoice. She
enters supplier and general ledger codes on
the face of the invoice.
G. There is a formal transfer of purchase invoices
and control totals pre-list and control logs are
signed by both Black and Owler.
H. The exception report is used by Owler to make
corrections in a separate stock/creditors
update run.
I. The chief accountant reviews monthly
purchases and creditors listings for
reasonableness and signs to indicate approval.
Likewise he reviews the information for
reasonableness.

Use with The Audit Process: Principles, Practice and Cases, 6th edn
9
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015
 Case studies:
setting system
objectives, use
of flow charts,
identification
of key
questions, and
evaluation of
systems

Figure 9.4

Use with The Audit Process: Principles, Practice and Cases, 6th edn
10
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015
Definitions: walk through test, tests of controls, substantive
tests

Table 9.1

Use with The Audit Process: Principles, Practice and Cases, 6th edn
11
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015
Tests of controls and substantive testing (1) - Para A4 ISA 330

• ‘The auditor’s assessment of the identified risks at the assertion

level provides a basis for considering the appropriate audit
approach for designing and performing further audit procedures.
For example, the auditor may determine that:
(a) Only by performing tests of controls may the auditor achieve an
effective response to the assessed risk of material misstatement for
a particular assertion;
(b) Performing only substantive procedures is appropriate for
particular assertions and therefore, the auditor excludes the effect
of controls from the relevant risk assessment. This may be because
the auditor’s risk assessment procedures have not identified any
effective controls relevant to the assertion, or because testing
controls would be inefficient and therefore the auditor does not
intend to rely on the operating effectiveness of controls in
determining the nature, timing and extent of substantive
procedures; or
(c) A combined approach using both tests of controls and
substantive procedures is an effective approach.’

Use with The Audit Process: Principles, Practice and Cases, 6th edn
12
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015
 Tests of controls and substantive testing (2)
• Para 18 ISA 330:
‘Irrespective of the assessed risks of material misstatement, the
auditor shall design and perform substantive procedures for
each material class of transactions, account balance, and
disclosure.’
• Para A43:
‘Depending on the circumstances, the auditor may determine
that:
– Performing only substantive analytical procedures will be
sufficient to reduce audit risk to an acceptably low level. For
example, where the auditor’s assessment of risk is supported
by audit evidence from tests of controls.
– Only tests of details (a substantive procedure) are
appropriate.
– A combination of substantive analytical procedures and
tests of details are most responsive to the assessed risks.’

Use with The Audit Process: Principles, Practice and Cases, 6th edn
13
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015
Important note on recording systems, tests of control
and substantive procedures S1

• Distinguish between:
1. tests recording systems and
2. testing controls and
3. substantive tests of detail.

• Objective of tests differ, but actual procedures

may be similar.

Use with The Audit Process: Principles, Practice and Cases, 6th edn
14
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015
Important note on recording systems, tests of control
and substantive procedures S2
1. Determine system and controls in place - walk-through tests - tracing a
few transactions through financial reporting system.
• Select sales order from credit customer and trace it to:
• (a) granting credit decision (b) despatch note (c) sales invoice (d) entry
in trade receivables account (e) entry in inventory movement record.
• Objective: NOT to prove ALL transactions properly recorded but to
understand system, record it, and see if entity has appropriate
controls.
• Para A13 ISA 315 (walk through tests in existing client): ‘The auditor is
required to determine whether information obtained in prior periods
remains relevant, if the auditor intends to use that information for the
purposes of the current audit. This is because changes in the control
environment, for example, may affect the relevance of information
obtained in the prior year. To determine whether changes have
occurred that may affect the relevance of such information, the auditor
may make inquiries and perform other appropriate audit procedures,
such as walk-throughs of relevant systems.’

Use with The Audit Process: Principles, Practice and Cases, 6th edn
15
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015
Important note on recording systems, tests of control
and substantive procedures S3
2. Auditors have to decide whether the system
appears strong enough for them to rely on it in
arriving at conclusions.
• Auditors perform tests of control to satisfy
themselves initial conclusion about system is valid.
Auditors might select 20 sales despatch notes and
trace in the same way as for walk-through test.
• Objective: to enable them to decide if they can in
fact rely upon the system and controls.
• Note: an important element is to ensure the
transactions selected have been authorized by
someone in authority.

Use with The Audit Process: Principles, Practice and Cases, 6th edn
16
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015
Important note on recording systems, tests of control
and substantive procedures S4
3. Auditors decide on level of control risk, leading to decision on level of
substantive procedures to enable conclusion at assertion level on
validity of transactions and balances in population.
(a) if control risk LOW, perform analytical review – do figures make
sense in the light of what is known about the company. Means
auditor would rely on substantive analytical procedures.
(b) If control risk HIGH, auditor may decide NOT to rely on controls
and use tests of detail only, selecting (say) 100 sales despatch notes
and tracing to other records = substantive tests of detail.
(c) If system WEAK, but auditors decide to RELY ON IT TO SOME
EXTENT, might select a combination of substantive analytical
procedures and tests of detail.
• Tests of controls and substantive tests of detail may be carried out at
the same time = dual purpose test.
• Sometimes only tests of controls will give needed audit satisfaction
because effective substantive tests cannot be designed – as in
complex systems.

Use with The Audit Process: Principles, Practice and Cases, 6th edn
17
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015
Relationship: walk through tests, tests of controls and
substantive testing Figure 9.5

Use with The Audit Process: Principles, Practice and Cases, 6th edn
18
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015
 Activity 9.1
• Rummond Limited is a trading company and performs inventory
count at end of year to be satisfied:
1. Quantities of recorded inventories are correct.
2. Physical condition is such that they are saleable.
3. Correct cut-off.
a) examine the inventory count instructions
b) at count auditor observes count procedures being carried out
by Rummond staff, testing a few of the items counted
themselves
c) select items counted at random and record in working files for
comparison later with quantities in inventory valuation
sheets.
• Refer to important note on recording systems, tests of control
and substantive testing and explain what the audit objectives
are at each stage, (a), (b) and (c) above.

Use with The Audit Process: Principles, Practice and Cases, 6th edn
19
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015
 Examples of tests of controls
• Tests of information/audit trail.
• Testing outputs on a restricted basis.
• Interviews with company staff (inquiry) using interviewing
style conducive to getting people to be open with them.
• Observing staff at work (observation), keeping eyes open
and not assuming staff will always operate in the manner
they have told you they do.
• Re-performance of control procedures.
• Examination of management reviews.
• Testing reliability of budgets prepared by management.

Use with The Audit Process: Principles, Practice and Cases, 6th edn
20
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015
 Approaches to computer systems

•Auditing round the computer

•Auditing through the computer:
computer assisted audit techniques
(CAATs)
•Auditing with the computer

Use with The Audit Process: Principles, Practice and Cases, 6th edn
21
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015
Specific tests of control in computer systems

• Code reviews of programs.

• Use of test data.
• Use of program code comparison.
• Continuous review of data and its processing.
• Integrated Test Facility (ITF).
• Systems Control and Review File (SCARF).

Use with The Audit Process: Principles, Practice and Cases, 6th edn
22
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015
 Evaluation of systems and audit conclusions
– Activity 9.2 (1)

• Weaknesses highlighted: Case study 9.3 (Broomfield)

A.No responsible official reviews minimum inventory levels. Management
authorises Ivor Jordan to disregard requisition if he believes goods not required
AND gives him authority to change minimum inventory levels on master file.
Weakness: Ivor could circumvent management's initial budget decision.
B.Ivor Jordan does not check official PO to original PO.
C.Eric Owler wholly responsible for investigating and correcting errors in exception
report – Ivor should be involved. Ivor should compare official POs with initial
orders as risk official orders incorrect. Main store does not get copy of initial
order.
D.Janet Black has no access to current prices and terms – no independent check.
Too much power with Ivor Jordan.
E.Insufficient independent checking of exception report and invoice listing.
F. Janet Black codes invoices, but no one checks her work before processing.
Strength is that chief accountant reviews cost information for reasonableness.

Use with The Audit Process: Principles, Practice and Cases, 6th edn
23
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015
 Evaluation of systems and audit conclusions
– Activity 9.2 (2)
• Case study 9.3 (Broomfield): suggest in broad terms what you might
do as a result of weaknesses found in the purchases system.
• The auditor might perform tests in the following areas:
– Review exception reports to ensure appropriate corrective action
taken.
– Check coding of purchase invoices to ensure costs have been
charged to appropriate cost and expense accounts. Support this
work by analytical reviews of costs.
– Check interventions to amend inventory master files using
information/audit trail as an aid.
– Match purchase orders and purchase invoices to ensure prices and
terms are in agreement.
– Check on reasonability of prices charged by suppliers.
• Some of this work might be carried out by internal auditors, in which
case the work of the internal auditors should be reviewed by the
external auditors.

Use with The Audit Process: Principles, Practice and Cases, 6th edn
24
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015