Outcome-based Quality

Assurance System
Nofie Iman
Quality Assurance Unit, Faculty of Economics and Business
Universitas Gadjah Mada

8 August 2018
Contents
Background

UGM and FEB UGM

Assurance of Learning at FEB UGM

Case study: MM FEB UGM

Lesson learned and concluding remarks

Who are we?
• The oldest state owned university • Universitas Gadjah Mada (UGM)
– established in 19 December • Higher Education Institution Accreditation (AIPT)
1949 as a response towards • Accreditation Board for Engineering and Technology
(ABET)
Universiteit van Indonesie in • Akkreditierungsagentur für Studiengänge der
Jakarta. Ingenieurwissenschaften, der Informatik, der
• It has 18 faculties, 1 vocational Naturwissenschaften und der Mathematik (ASIIN)
school, 1 graduate school, with • Philippine Accrediting Association of Schools, Colleges and
Universities (PAASCU)
251 programs. It comprises more • Institution of Chemical Engineers (IChemE)
than 56,000 students, 3,600 • The International Medical Informatics Association (IMIA)
lecturers, and 4,300 staffs.
• Faculty of Economics and Business (FEB-UGM)
• State-owned university legal
• National Accreditation Board-Higher Education (BAN-PT)
entity (PTN BH) – manages • The Association to Advance Collegiate Schools of Business
academic and non academic (AACSB)
matters autonomously.
Why do we need to make changes?
General Background

Global Disruptive
competitiveness changes
International Emerging

Which resulted in …
dynamics millennials
Technological Delivery changes
innovations
Outcome-based?
Level Input Output Outcome Impact
Students acquired the Students used the
Classroom Course Lesson attitude, skills and acquired attitude, skills
lecture syllabus delivered knowledge (learning and knowledge to solve
outcomes) of the lesson. real life problems.
Graduates with relevant
attitude, skills and Graduates contributed to
Study Lab Number of
knowledge are employed the improvement of the
Programme equipment graduates
and able to perform as organisations and society.
expected.
High employability of
Number of Graduates contributed to
Resources graduates and quality
University graduates and the betterment of society
available workforce for the labour
postgraduates and the country.
markets.
Faculty of Economics and Business (FEB-UGM)

Mission
Integrity
Enriched by our vibrant international network but rooted in local
wisdom, we develop leaders with the integrity and knowledge
needed to serve society. Our contribution to the advancement of
knowledge is through quality research and teaching. Social Professio-
Concerns nalism

Values
Vision
To be the leading faculty of economics and business in
Indonesia by rising to international challenges.
Objectivity
Academic
and
Freedom
Fairness
AIPT/BAN-PT AACSB

• Vision, Mission, Goals and Objectives • Strategic Management and Innovation

• Standard 1: Mission, Impact, Innovation
• Output and Impact of “Tridharma”
• Standard 2: Intellectual Contributions, Impact, and Alignment with Mission
• Teaching • Standard 3: Financial Strategies and Allocation of Resources
• Research • Participants – Students, Faculty and Professional Staff
• Community service • Standard 4: Student Admissions, Progression, and Career Development
• Standard 5: Faculty Sufficiency and Deployment
• Students
• Standard 6: Faculty Management and Support
• Human Resources (HR) • Standard 7: Professional Staff Sufficiency and Deployment
• Finance, Facilities and Infrastructure • Learning and Teaching
• Standard 8: Curricula Management and Assurance of Learning
• Governance and Cooperation.
• Standard 9: Curriculum Content
• Standard 10: Student-Faculty Interactions
AMI-SPMI • Standard 11: Degree Program Educational Level, Structure, and Equivalence
• Standard 12: Teaching Effectiveness
• Internal Quality Audit (AMI) • Academic and Professional Engagement
• Internal Quality Assurance System (SPMI) • Standard 13: Student Academic and Professional Engagement
• Standard 14: Executive Education
• Standard 15: Faculty Qualifications and Engagement
“Triple-helix” complexity
INTERNATIONAL
STANDARDS 5-year
(GLOBAL)
AACSB

INTERNAL
NATIONAL
STANDARDS 5-year
STANDARDS
(UNIVERSITY-
1 or 2 year (COUNTRYWIDE)
WIDE)
AIPT/BAN-PT
AMI/SPMI
Our experiences
No Milestone Date
1 AACSB Membership 10 March 2006
2 Pre-Accreditation Eligibility Application submission 10 August 2006
3 Eligibility Application accepted 3 November 2006
4 Accreditation Plan submission 15 April 2009 FEB UGM was finally
5
6
Accreditation Plan accepted by PAC and IAC
Assurance of Learning Workshop
12 January 2010
29-30 January 2010
accredited
7
8
Progress Report submission to IAC to review (year 1)
Progress Report submission to IAC to review (year 2)
22 October 2010
14 October 2011
by AACSB on
9 Mentor visit by Michael Tearney 31 January - 2 February 2012 May 12, 2014
10 Mentor visit by Michael Tearney & Eileen Peacock 28-29 August 2012
11 Progress Report submission to IAC to review (year 3) 28 September 12
12 Assurance of Learning Workshop 10-13 January 2013
13 Invitation by IAC to submit SER and confirm PRT visit 8 August 2013
14 Submission of the Self-Evaluation Report 31 October 2013
15 Mock Team Visit 19 January 2014
16 AACSB Peer Review Team visit 28 February - 3 March 2014
Our milestones
Year 1 (July 1, 2014 – June Year 2 (July 1, 2015 – Year 3 (July 1, 2016 – June 30, Year 4 (July 1, 2017 – June Year 5 (July 1, 2018 – June
30, 2015) June 30, 2016) 2017) 30, 2018) 30, 2019)
• Review and Refine • Review and Refine • Review and Refine Strategic • Review and Refine • Review and Refine
Strategic Management Strategic Management Plan Strategic Management Strategic Management
Plan Management Plan • Complete Key Data and Plan Plan
• Complete Key Data and • Complete Key Data Accreditation Data Sections of • Complete Key Data and • Complete Key Data and
Accreditation Data and Accreditation the Business School Accreditation Data Accreditation Data
Sections of the Business Data Sections of the Questionnaire for prior Sections of the Business Sections of the Business
School Questionnaire for Business School academic year School Questionnaire for School Questionnaire for
prior academic year Questionnaire for prior • July 1, 2016 - Submit prior academic year prior academic year
academic year Continuous Improvement • July 2017 - confirmation of • Distribute Completed
Review Application(s) team visit date: Applicant Continuous Improvement
• December 2016 - January advices no known of Review Report(s) 60 Days
2017: CIRC/AAC Rules on conflict of interest and Prior to Visit (deadline:
exclusions and the scope of invitation is sent; length of December 26, 2019)
accreditation visit time to confirmation and • Accreditation Statistical
• February 2017: Scope of date is dependent on Reports will be distributed
Accreditation Confirmed responses to applicant and team
• February 15, 2017: Open date • Peer Review Team and members by AACSB at
for BSQ data entry visit date confirmed. See least 60 days prior to visit
• March 9, 2017: Return date and Accreditation Policies and date upon request only
PRT Nomination Form Procedures for Volunteer • Work with PRT Chair to
• May 26, 2017: Deadline BSQ Deployment and Selection finalize the visit schedule
• June 2017: Proposed PRT • Begin communications • PRT Visit (February 24-26,
provided to the applicant to with Peer Review Team 2019)
identify potential conflict of
interest
Assurance of Learning System
LGs and LOs Development
A. Step 1: Develop and establish the LGs/LOs of each course
B. Step 2: Align the curriculum and course with LGs/LOs that
has been agreed and map the relevant LGs/LOs into the
course
Measurement
A. Step 3: Develop measurement instruments and appoint
measurement schedules
B. Step 4: Conduct measurements, collect, and analyze
measurement data
Closing Cycle
A. Step 5: Review the results of the measurements and make
recommendations; scrutinize the LGs/LOs measurements
attached to specific courses (embedded courses) with
relevant faculty members for feedback and suggestions
B. Step 6: Communicate the measurement results along with
its recommendations to all faculty members in the
department
C. Step 7: Implement decisions of amendments or
improvements approved as a result of AoL (close the loop)
Some examples…
• The school has developed guidelines for theses writing and guidelines for preventing plagiarism in
academic writing
• Structured soft skills classes were introduced in 2017 at the undergraduate level, consist of
leadership, anti-plagiarism, writing skills, ethics, and presentation skills.
• In Master in Accounting, a number of students’ papers based on their thesis are accepted to be
presented in accounting conferences. Thesis writing has shortened from 10,7 months to 6,8 months.
• An immersion program has been initiated at Magister in Management since 2017. This program is
designed to provide students with an exposure to the different industries and corporate cultures.
• In accordance with our mission, the students are also required to attend at least two executive series
during their period of study.
• Our doctorate program has actively adjusted the curriculum to become more structured and
systematic, and increases the level of the students’ completion time (max. 5 years) (the average is
about 6-7 years).
• Doctorate program also provides soft skills trainings such as writing and communication skills.
Case study: MM FEB UGM
GLOBAL ENVIRONMENT REGULATORY CHALLENGE
• Business education at the Masters level requires the • According to the Rector Regulation No. 11 of 2016 regarding
integration between Being, Knowing, and Doing. It has Postgraduate Education, the total credit hours for masters
become the best practice among the best business school in program is 40-50, consist of 28-38 lectures and 8-12 thesis
the world. writing. The study period is 2-6 semester without any
• The integration between technology, innovation, and particular provisions for executive and international
sustainability is somewhat mandatory. programme.
• Business school curriculum should be aligned with the
mission.
AOL FINDINGS OUR OBSERVATION
• Our students were failed in several aspects of • As of January 2016, there was a bottleneck in which 59% of
communication learning goals, such as reasoning in oral the students were writing their thesis.
communication, argumentation in written communication. • There was 43 students entering the extension period.
• While they were able to meet the standard in knowledge • The percentage of students who are able to finish their
learning goals, several aspects requires attention, i.e. the studies in time is only around 3%.
ability to apply knowledge.
• With regard to leadership learning goals, the students are
lack of the ability to inspire others and to envision
organisational changes.
Case study: MM FEB UGM
MISSION: Enriched by our Key Element Previous Learning Goals New Learning Goals
vibrant international
1. Students demonstrate the ability
network but rooted in local 1. Students demonstrate the
to communicate in an effective,
wisdom, we develop Communication ability to communicate
persuasive, and professional
strategic business leaders professionally
with integrity, creativity, and manner
knowledge needed to serve 2. Students demonstrate the 2. Students demonstrate the ability
society Knowledge ability to apply knowledge in to translate business knowledge
VISION: To be the leading the field of management into action
graduate business school in 3. Students demonstrate the ability
Indonesia in advancing to make ethical and responsible
knowledge of business and 3. Students are able to decisions
promoting sustainable and Integrity and
exercise ethical and creative
ethical business practices creativity
decision making 4. Students demonstrate the ability
by rising to the international to solve practical business
challenges problem with creative ideas
VALUES: Integrity,
4. Students demonstrate the
Professionalism, Objectivity 5. Students demonstrate the ability
Leadership ability to be a transformational
and Fairness, Academic to be a strategic business leader
leader
Freedom, Social Concerns
Case study: MM FEB UGM
Mission
Enriched by our vibrant international network, but rooted in local wisdom,
we develop strategic business leaders with integrity, creativity and knowledge needed to serve society

Engagement Innovation Impact

• Alumni connection
• Partnership with
• Social project
foreign universities • Continuous
• Book of cases
• LEAP improvement:
• EDP
• Internship Redesigning
• Research and
• EDP curriculum and
consulting
• Research and learning process
services
consulting services
Case study: MM FEB UGM
MISION VISION LEARNING GOALS
Enriched by our vibrant international To be the leading graduate business 1. The ability to communicate in an effective,
network but rooted in local wisdom, we school in Indonesia in advancing persuasive, and professional manner.
2. The ability to translate business knowledge into
develop strategic business leaders with knowledge of business and
action.
integrity, creativity, and knowledge promoting sustainable and ethical 3. The ability to make ethical and responsible
needed to serve society business practices by rising to the decisions.
international challenges 4. The ability to solve practical business problem
with creative ideas.
5. The ability to be a strategic business leader.

Being Knowing Doing

• Assessment • Immersion
• Self-awareness • Case Methods • Internships
• Leadership autobiography • Student Centered Learning • Field projects
• Personal mission • LEAP • Thesis
Case study: MM FEB UGM
• Change/merge courses
• Master syllabus
• Parallel exams, etc.
Curriculum redesign
• Better engagement and interaction between
students, faculty members, business
LEAP (Leadership community, and society
Enhancement and • Faculty members provide more significant
Acceleration Program) intellectual contributions
• Positive impact on society through social
project and immersion programme
Guidance and
assistance for our • On average, the students finish school
students • Plagiarism-check (graduate) faster
• Co-working spaces • Higher overall student satisfaction surveys
Performance-based • Faculty qualification systems is also being
incentives for research adopted at university-level
• Performance-based incentives for research
publications publications as well (grants, funding, etc.)
How does it differs?
National Standard International Standard
Assessment method Input-based Impact-based
Measurement Standardisation, looking for similarity Accreditation, embrace diversity
Main focus Homogeneity Accountability
Threshold Very low baseline High quality baseline
Exclusive, distinguish the best from
Inclusion criteria All inclusive, no one left behind
the rest
Timing One point at a time Continuous improvement
Done once the report has been
Completion stage Never complete
submitted
Performed by the government or Performed by independent
Main actor
regulatory body association
Concluding remarks
• Internal changes is inevitable, but we can’t/couldn’t/won’t do that
• International accreditation as a pressure to enforce changes
• If we focus on impact and outcomes, the rest will follow
• Focus on the right, the left follows – better input, better process
• Self-selection – prospective students, faculty members, university partners, etc.
• It is all about culture, changing mindset, and strong commitment
• It goes beyond labeling and certification
• “Business-as-usual mentality” is no longer an option
• Structural changes is required
• Calls for convergence between national and international standards
• Separation between authority/regulatory function and supervisory/advisory function
• Private schools/universities might face bigger challenges, affirmative action might be necessary
THANK YOU
Questions and comments?
[email protected]