WELCOME

Computational Modeling and Simulation to Support Implant Design and Evaluation

and Clinical Decision Making – Practical Considerations on Model Credibility
PART 2: Code and solution verification considerations for translational computational
modeling & simulation applications

MODERATORS:
ORGANIZERS: ORS Orthopaedic Implants Section
Dr. Ahmet Erdemir
Dr. Jeffrey E. Bischoff
Associate Staff
Research Director
Cleveland Clinic
Zimmer Biomet
Dr. Marc Horner
Dr. Jonathan A. Gustafson
Senior Principal Engineer
Instructor
Ansys
Rush University Medical Center
 ORGANIZERS

Jeffrey E. Bischoff, PhD Jonathan Gustafson, PhD

Zimmer Biomet Rush University Medical Center
Join the ORS Orthopaedic Implants Section

As a Section member, you will have unique and exclusive opportunities to engage
and connect with the #ORSNation year-round whether in-person or virtual at any
career stage through: The Section launched in 2016 with the mission to
represent the voice of orthopaedic implants
Community Connections: networking/social hours, online community forums, related research within the ORS community and
Section member awards recognizing excellence in research, and the quarterly beyond; and to advance implants research and
Section e-newsletter,
Orthopaedic Implants Quarterly or via Twitter with #ORSImplants. related sciences to improve patient care through
basic, translational and clinical research.
Cultivating Leadership and Volunteer Experiences: Serve as a Section Officer or
on a Section Committee or micro-volunteer opportunities (Section Awards
Reviewer)

Education and Career Development: annual Section Scientific Meeting during the
ORS Annual Meeting, virtual scientific sessions, satellite meetings/events, and
other Section organized programs.

Research and Latest Discoveries: Fostering the multidisciplinary interactions

which is required to improve the quality of the implants, ensure that new implants
are adequately tested at the preclinical stage, that implants are introduced
carefully to the orthopaedic market and that the clinical outcome is appropriately
registered and much more!

Section awards, fellowship, and grant opportunities including:

• Annual Section Podium and Poster Awards based on accepted ORS Annual
Meeting abstracts.
 Find your home in our research community. It’s more than a membership.

JOIN ORS ORS is the leading research society supporting biologists, clinicians,
engineers, orthopaedic surgeons, and other professionals in
musculoskeletal research fields across the globe.

Education & Self Development Engage with the Scientific Community

• Journal Subscription to JOR, free manuscript submission, • Attend ORS Annual Meeting, Scientific meetings,
discounts on JOR Spine Research Section meetings & workshops at special rates
• LearnORS courses: The Art of Grantsmanship, Principles of • Join Research Sections in 7 specific areas & connect with
Clinical Research, Enhanced Writing & Publication Skills, smaller, intimate peer communities
Innovation & Commercialization • Moderate Scientific Sessions
• Scientific and Career Webinars • Organize Workshops & Research Interest Groups
• ORS Career Center • Exclusive ORS Online Member Directory
• Find Mentors • Find a Collaborator Directory for ORS Members only
• Virtual Scientific Sessions, events, social networking

Awards & Opportunities Get Involved with ORS!

Travel grants, Research Grants & Fellowships including Participate, organize, volunteer
• Highly-Coveted New Investigator Recognition Awards (15) • JOR Reviewer
• ORS Pilot Grant Awards, Collaborative Exchange grants • ORS Ambassador
• ORS Research Section Awards for Section members (7 Sections) • ORS Committee Member, Leader, Mentor
• Stryker ORS Women’s Research Fellowship • ORS Insider, Volunteer
• ORS/RJOS Young Female Investigator Grant • Join ORS Community Networks, launch new Networks
• ORS/ON Foundation Educational Grants • Contribute to the ORS Ideas bank
• ORS/ON Orthoregeneration Award • Lead & collaborate in virtual sessions, events
• JOR & JOR Spine Awards, Excellence in Science, Early Career
Awards plus more awards at ors.org
Join now! HTTPS://WWW.ORS.ORG/
 WELCOME

Computational Modeling and Simulation to Support Implant Design and Evaluation

and Clinical Decision Making – Practical Considerations on Model Credibility
June 23 PART 1: Defining translational applications of computational modeling @ simulation in orthopaedics,
understanding the need for model credibility, and assessing model risk

Sep 22 PART 2: Code and solution verification considerations for translational computational
modeling & simulation applications

Nov 10 PART 3: Validation considerations for translational computational modeling & simulation applications

Feb 4 Annual Meeting symposium: Model credibility

Verification – Why do we care?
• Conclusions from (validated, not verified) FEA study
presented @AAOS 2021 on shoulder arthroplasty:
• ‘This FEA study shows that rTSA can tolerate large
retroversion angles up to 25°, with the micromotion and
forces generated between the glenoid baseplate and
host bone staying well below the threshold for successful
implant fixation, in both normal and osteoporotic bone.’
• ‘For patients who have significant glenoid deformities
with large retroversion angles, rTSA can provide stable
initial fixation when left in retroversion up to 25°,
without the need for eccentric reaming, loss of glenoid
bone stock, and medialization of the glenoid component.’
• Clinician presenter / mentor / user: ‘I rely on my
engineering colleagues to do good technical work’

Dharia et al., Frontiers in Physiology 9, 2018.

 Verification in Medical Device Review
• Verification is necessary for all
modeling reports
• FDA guidance provides detailed
expectations

https://www.fda.gov/regulatory-information/search-fda-guidance-documents/reporting-computational-modeling-studies-medical-device-submissions
 Verification in Medical Device Review
• FDA reviewed 5 years of spinal
cage 510(k) submissions1
• 65 contained FEA reports
• Quantified conformance to FDA
Reporting Guidance
• 3 reports contained Code
Verification

1 https://onlinelibrary.wiley.com/doi/full/10.1002/jsp2.1137
MODERATORS: Dr. Ahmet Erdemir Dr. Marc Horner
Associate Staff Senior Principal Engineer
Cleveland Clinic Ansys
 Computational Modeling and Simulation to Support Implant Design
and Evaluation and Clinical Decision Making – Practical
Considerations on Model Credibility
PART 2: Code and solution verification considerations for translational
computational modeling & simulation applications

INTRODUCTION TO SESSION
Ahmet Erdemir Marc Horner
Cleveland Clinic ANSYS

September 22, 2021

Academia’s Blind Spot

Search terms (PubMed) Results

Simulation medical 11,054
Simulation medical verification 95

In silico medical 62,605

in silico medical verification 470
in silico medical discretization 874

verification are <1%

Mention of discretization <1.5%

WHAT IS OUR PRIORITY?

Perceptions in Medical Community

• Survey on perception of in silico technologies in medical community

163 responders - 11 countries, 8 medical specialties
• Summary results for level of trust
6.48 out of 10
• Highlights from trust related entries:
• Computer modelling and simulation provides me with more confidence in planning
procedures
72% agree/strongly agree
• Patient-specific computer modelling and simulation is accurate enough for clinical
application
53% agree/strongly agree
• Results of computer modelling and simulation are easy to understand
28% agree/strongly agree

Courtesy of Avicenna Alliance, Research & Technology Working Group, Medical Community Outreach Task Force, 2021.
For more information, refer to Claudio Capelli, [email protected].
Categories of Verification

• Code Verification

Process of determining that the computer code is correct and

functioning as intended.

• Solution Verification

Process of determining solution accuracy of a particular calculation.

From Thacker et al., Concepts of Model Verification and Validation, LA-14167-MS, Los Alamos National Laboratory, October 2004.
Modeler’s Responsibility

• To understand verification
requirements

• To plan and execute verification

activities

• To communicate the need and

burden to
• Clinicians
• Management
 Program on Verification

Jeff Weiss Steve Maas

! Check Handouts
Fill in your example @ https://forms.gle/dNA2cBRuAoqW2Pos9
 Program on Verification

Ismail Guler

! Check Handouts
Fill in your example @ https://forms.gle/dNA2cBRuAoqW2Pos9
 Program on Verification

Marc Horner

! Check Handouts
Fill in your example @ https://forms.gle/dNA2cBRuAoqW2Pos9
 Program on Verification

Cheryl Liu

! Check Handouts
Fill in your example @ https://forms.gle/dNA2cBRuAoqW2Pos9
 Program on Verification

Ahmet Erdemir

! Check Handouts
Fill in your example @ https://forms.gle/dNA2cBRuAoqW2Pos9
 Code Verification
with FEBio
Jeffrey Weiss, PhD
Steve Maas, PhD
Musculoskeletal Research Laboratories
Departments of Biomedical engineering and Orthopedics, and
Scientific Computing and Imaging Institute

September 22, 2021

1
Short Intro to FEBio
• Started in 2005, the goal was to create a FE solver for
biomechanics.
• True Multiphysics capabilities
• Combines nonlinear mechanics of solids, fluids and mixtures, reaction-
diffusion, electrokinetics.
• Supports quasistatic and dynamic (or transient) analyses, interstitial fluid
transport, fluid-solid interaction, frictional contact and sliding, interfacial
mass transport and electrical conduction.
• Large library of constitutive models focused on representing
biological tissues.
• Anisotropic constitutive models, viscoelastic, growth, remodeling,
damage, degeneration, diseased, pre-strained, active contraction, etc.

2
Short Intro to FEBio
• Easily customizable via a plugin framework.
• FEBio Studio: Integrated development environment for creating,
running, and visualizing FEBio models.
• Significant and growing impact in the biomechanics community
(and beyond)
• Over 60,000 downloads of the FEBio software.
• Over 12,000 registered users on febio.org
• Over 500 publications that have used FEBio (incl. journal articles, PhD
theses, book chapters).
• For more information, www.febio.org

3
Software Quality Assurance
• Software Quality Assurance (SQA) is the practice of
monitoring the software development process to ensure
quality of the software.
• SQA involves design, coding, review, testing, release of
software.
• Verification and Validation is an important aspect of SQA.
• Verification, and in particular how we address it with FEBio,
will be the main focus of this presentation.

4
Relevant Articles on V&V from Dr. Weiss lab
• Anderson AE, Ellis BJ, Weiss JA: Verification, validation and sensitivity
studies in computational biomechanics. Computer Methods in
Biomechanics and Biomedical Engineering, 2007 (Pubmed ID
17558646)

• Henninger HB, Reese SP, Anderson AE, Weiss JA: Validation of

computational models in biomechanics. Journal of Engineering in
Medicine, 2010 (Pubmed ID 20839648)

5
 Motivation
The goal of Verification and Validation is:

• Gain confidence in modeling predictions Hmm, I might

We calculated the need a second
• Range of applicability optimal shape of the opinion.
• Sensitivity to model inputs patient’s hip implant.

• Achieve acceptance of modeling predictions

• By your peers
• In publications: many journals have policies regarding
modeling studies and the use of standard modeling
practices
• Provide confidence to clinicians, experimentalists,
analysts for information extrapolation and decision
making.

6
 Code Verification and Solution Verification1

• Code Verification: Is the code implemented correctly? Reality

• Is the mathematical model correctly translated to efficient
code? Conceptual model
• Does not require unacceptably long runtimes or memory resources
• Mostly responsibility of the developers
• However, users who develop plugins are also developers and Mathematical model
must practice code verification strategies.
• Users can also aid with this: Algorithms
• through independent testing, bug reporting.
Code verification
• Solution Verification1: Is the solution sufficiently accurate? Code implementation
• Accuracy of solution can be affected by spatial and temporal
discretization, iterative algorithm termination criteria, etc.
• This is to some extent the responsibility of the users. Computational Model
Solution
• Users need to understand the underlying assumptions and verification
range of validity of a particular implementation. Simulation results

1. Sometimes also referred to as calculation verification, Anderson et.al., CMBBE, 2007.

7
 FEBio Code Verification
• A code is considered “verified” if it satisfies a collection of accepted
benchmark problems.

• Existing guidelines suggest comparison to test problem hierarchy1,2

• Analytical solutions
• direct solution of ODE/PDE
• method of manufactured solutions
• Method of rotated solutions3
• Observed order of accuracy
• Numerical integration of ODEs / PDEs
• Comparison to existing codes
• New features must be verified independently and in combination with
other code features.

1 - ASME (PT60) Guide to V&V, 2006 2 – AIAA Guide to V&V, 1998 3 – Horner, JVVU, 2021 8
 Example 1 – Analytical solution comparison
0.10

• Analytical solutions can often be 0.08

Analytical Solution
FEBio Ogden

calculated for constitutive models 0.06

0.04

Stress (MPa)
under simple deformations: 0.02

• uniaxial tension and compression 0.00

• simple shear.
-0.02

-0.04

• This only verifies implementation -0.06

0.6 0.8 1.0 1.2 1.4

of stress. What about tangents? Stretch

Simulation of uniaxial tension-compression

material tests for Ogden constitutive models,
showing excellent agreement between the analytical
solutions and the predictions from FEBio.1

1. Maas et. al., JBME, 2012

9
FEBio Diagnostics
• FEBio offers several diagnostics that can help code verification.
• The FEBio diagnostics are a collection of algorithms that aims at
verifying that different aspects of the implementation are consistent.
• Many of these are based on approximating various derivatives with finite
differences and comparing to implementation.
• Examples:
• Secant test: evaluates the elasticity of a material implementation using finite differences
of stress.
• Tangent test: evaluates single element stiffness matrix by approximating it with finite
difference of residual.
• Similar tests are also available for other features such as contact, biphasic, etc.

10
Example 2 – comparison with other FE codes
A B

• Implementations of constitutive models can

sometimes be compared with other FE C

codes.
• Can be challenging if codes do not provide 1.6

sufficient details on algorithmic

1.4 FEBio No Contact
NIKE3D No Contact

Lateral Displacement (mm)

1.2 ABAQUS No Contact
FEBio Contact

implementations. 1.0 NIKE3D Contact

ABAQUS Contact
0.8

• Are differences due to coding errors, or

0.6

0.4

0.2

algorithmic differences? 0.0

0 20 40 60 80 100
Percent Compression

Verification of FEBio predictions by comparing to

other FE codes.
Adapted from Maas et.al., JBME, 2012

11
 Example 3 – FEBio prestrain feature
• Implementation of a prestrain feature C
A B Effective
in FEBio was challenging because Stress (MPa)

• Analytical solutions were difficult to

obtain for the constitutive models of
interest.
• Other FE codes did not implement this
capability.
• We came up with verification strategy (A) Forward model, a uniaxial stretch is applied, followed by a shear.
(B) Prestrain analysis from deformation gradient at the end of the
where predictions from forward uniaxial stretch (C) Similar prestrain analysis but using only the fiber
stretches. Adapted from Maas et.al., JMBBM, 2016
analyses were compared to predictions
of equivalent prestrain analyses.

12
FEBio Test Suite
• This is a collection of small test problems that can
be run quickly.
• Continuously monitors changes to the code and
ensures that code continues to produce the same
answers.
• Test suite run nightly on several different OS,
including Windows, Mac, Linux.
• Currently contains 464 problems, and growing with
each new feature.

13
FEBio Test Suite
• Users have access to the test suite via either
• the model database repository, accessible via FEBio
Studio.
• from github
(https://github.com/febiosoftware/TestSuite)
• Users can start a discussion on our forum
(https://forums.febio.org/)
• Issues can be reported to our GitHub site.
(https://github.com/febiosoftware)
FEBio Studio Model Repository, which
provides access to FEBio Test Suite.

14
Note on Reproducibility
• There is an inherent problem with using simulation software: Results can
vary from run to run!
• Computers use finite arithmetic and solution can be affected by the order
of operations.
• This order of operations can be affected by
• Compiler (using different compiler optimization)
• OS (different compiler; differences in implementation of run-time libraries)
• Number of CPUs (differences in order of execution of threads).
• Fortunately, most often differences are small, and are mostly noticeable in
convergence statistics, not in the final solution.
• Ill-conditioned problems often suffer from this problem
→ lack of reproducibility can be used to identify possible problem with model.

15
Solution Verification

“The process of determining that a

computational model accurately represents
the underlying mathematical model and its
solution”1

1 - ASME (PT60) Guide to V&V, 2006

16
How can we help?
• As a first step to solution verification, users need to make sure that the model is
setup of correctly.
• What can we (as developers) do to minimize user error?
• FEBio Studio performs several “common sense” checks to make sure the model is set up
correctly.
• Are all parts assigned a material?
• Do shells have a non-zero thickness?
• Are rigid constraints setup correctly?
• …
• FEBio does additional checks
• parameter range verification
• Checks for incompatible constraints
• Of course, these checks will only catch “beginner” mistakes, and do not
guarantee the model is setup correctly.
• Users need to take additional steps to verify model and solution.

17
 VERIFICATION

Computational Modeling and Simulation to Support Implant Design and Evaluation

and Clinical Decision Making – Practical Considerations on Model Credibility
Session 2: Code and solution verification considerations for translational computational
modeling & simulation applications

Part I: Background on Code and Solution Verification

SPEAKER:
Solution/Calculation Verification Ismail Guler
Senior R&D Manager
September 22, 2021 Boston Scientific
 Speaker’s background
• B.S. in Mechanical Engineering, 1993, Bogazici University, Istanbul
• M.S. in Mechanical Engineering, 1995, Bogazici University, Istanbul
• M.S. in Aerospace Engineering, 1997, University of Minnesota
• Non-degree Graduate Student, 2006 – 2015, University of Minnesota
• Member of CM&S Team, 1998 – Present, Boston Scientific, Maple Grove, MN
(Neurovascular Stent R&D, 2008 – 2010, Fremont, CA)
• Lecturer in Biomedical Engineering, 2014 – Present, University of Minnesota
• Member of ASME VVUQ 40 Subcommittee, 2014 – Present
VVUQ is a process to assess the accuracy/credibility/uncertainty
of a model and its prediction.
“Essentially, all models are wrong, but some are useful.” wrong → inaccurate or uncertain

“Remember that all models are wrong; the practical question is how wrong
do they have to be to not be useful.”
-- George E. P. Box

• Model form uncertainty (“solving the right equations”)

o Lack of understanding
o Oversimplification!
• Data input uncertainty
o System configuration (geometry)
o System properties (material properties, etc.)
o System conditions (boundary & initial conditions)
• Numerical uncertainty (“solving the equations right”)
o Algorithmic & coding errors
o Discretization errors
o Iterative errors
o Round off errors
• Use error
ASME Verification, Validation, and Uncertainty Quantification (VVUQ) Committee

Coordinate, promote, and foster the development of standards that

CHARTER provide procedures for assessing and quantifying the accuracy and
credibility of computational models and simulations.
VVUQ 10 Computational Solid Mechanics 2001
VVUQ 20 Computational Fluid Dynamics and Heat Transfer 2004
Computational Simulation of Nuclear System
VVUQ 30 2010
Thermal Fluids Behavior
VVUQ 40 Computational Modeling of Medical Devices 2011
VVUQ 50 Computational Modeling for Advanced Manufacturing 2016
VVUQ 60 Computational Modeling in Energy Systems 2017
VVUQ 70 Machine Learning Applied to Mechanistic & Process Modeling 2019
https://www.asme.org/codes-standards/publications-information/verification-validation-uncertainty 4
“Four ups” of VVUQ (adapted from “integral theory” of Ken Wilber)
VVUQ is a process for realizing the reality (credibility) of computational models.
Waking up ASME VVUQ 40 (How much VVUQ?)
(enlightenment) Being conscious of COU of model, model risk
(model influence, decision consequence), etc.

Growing up ASME VVUQ 10 & 20 (How to do VVUQ?)

(maturity) Being well versed in VVUQ methodologies
(not just textbook knowledge, but experiential knowledge)

Cleaning up Shadow work: allergies and addictions

allergies → dislike/resistance to rigorous VVUQ because it
puts burden on us (convenience bias), etc.
addictions → urge to follow old way of doing things, etc.
(following legacy standard procedures in VVUQ)

Showing up Progress made in “three ups” listed above allows one to

become a better citizen of the modeling world and engage
fully in that world. 5
 Waking up Showing up

VVUQ chain
(the chain of credibility)

You, the modeler, the realizer

6
Resources
Journal of VVUQ

https://asmedigitalcollection.asme.org/verification
 Events

https://event.asme.org/V-V
 What is model verification & validation (V&V)?

Code Calculation
Verification Verification

Mathematical Computational Simulation

Model Model Results

Comparison Validation
Verification is “solving the equations right”.

Validation is “solving the right equations”. Experimental

Data

Adapted from ASME V&V 10

 What are the uncertainties inherent in CM&S?

Numerical Uncertainties in Calculations

Inputs Calculation of Uncertainties
(Calculation Verification)
Numerical
Uncertainty Metrology analogy:
Uncertainties in Measurements
Measurement of Uncertainties
Mathematical Computational Model Parameter
(Experimental Error in
Model Model Inputs
Model Input Validation)
Form Parameter
Uncertainty Uncertainty

Simulation Uncertainties are

Results propagated into
simulation results.

Adapted from ASME V&V 20

DISCLAIMER: numerical uncertainty versus model-form uncertainty

It is better to solve the right problem approximately than to solve the wrong problem exactly.

-- John W. Tukey
 ASME VVUQ 40 credibility factors

Activity Credibility Factor

Software quality assurance
Code
Numerical code verification
Verification Discretization error
Calculation Numerical solver error
Use error
Computational Model form
model Model inputs
Test samples
Validation Comparator
Test conditions
Equivalency of input parameters
Assessment
Output comparison
Relevance of the quantities of interest
Applicability
Relevance of the validation activities to the COU
14
 Numerical uncertainty

U NUM = U DE + U IT + U RO

U NUM : uncertainty due to numerical approximation error

U DE : uncertainty due to discretization error (mesh refinement level)
U IT : uncertainty due to iterative error (nonlinear iterations)
U RO : uncertainty due to round off error (finite-precision arithmetic)

Numerical uncertainty is epistemic since it can be reduced (e.g., using a finer mesh or a smaller
tolerance for nonlinear iterations or double precision arithmetic instead of single precision, etc.).

Roy, C. J. and Oberkampf W. L., A comprehensive framework for verification, validation, and uncertainty quantification in
scientific computing, Computer Methods in Applied Mechanics and Engineering 2011; 200: 2131-2144.
 Systematic mesh refinement
For calculation verification, one needs a triplet of meshes (coarse, medium, fine).

hi
rij =
hj
r = r32 = r21 = 2
h3 h2
r= = =2
h2 h1
(mesh
refinement
factor)

Systematic mesh refinement

requirements:
1) Uniform refinement
2) Consistent refinement
16
 Concept of asymptotic range

Comparison of Solution
Verification Techniques

V. Gregory Weirs, William J.

Rider, James R. Kamm

Sandia National Laboratories,

Albuquerque, NM
July 24, 2013
 Richardson extrapolation

f exact : exact solution to mathematical model

f h : approximate numerical solution ( fi = f hi )
f h = f exact + C  h p + O (h p +1 ) h : characteristic element size for a given mesh
H .O .T p : order of accuracy for the numerical method
C : a constat which does not depend on " h "
H .O.T : higher order terms
mesh with smallest element
size in group considered
 f3 − f 2  (observed order
pˆ1 = ln   / ln r
−
 2 1
f f of accuracy)
1 2 3
h1  h2  h3 f1 − f 2 (Richardson extrapolation
f1 = f1 + pˆ1
r −1 estimate of exact
solution)
p̂1 f1 ( E1 )i = f i − f1 (discretization error)
18
 Grid convergence index (GCI)

( Fs )1 (Roache’s grid convergence index pˆ : observed order of accuracy

(GCI )1 = p1 f 2 − f1
r −1 for fine mesh solution) p f : formal order of accuracy
(expected, theoretical)
pˆ − p f Fs : factor of safety
 0.1 → Fs = 1.25 & p = p f
pf
GCI is an uncertainty estimate (an
pˆ − p f uncertainty bar or uncertainty
 0.1 → Fs = 3.0 & p = min(max(0.5, pˆ ), p f ) band, not an error estimate).
pf
(Page 326, Table 8.1 in Oberkampf & Roy 2010)

1) Roache P. J., Fundamentals of Verifivation and Validation, Hermosa, New Mexico, 2009.
2) ASME V&V 20-2009
3) Oberkampf W. L. and Roy C. J., Verification and Validation in Scientific Computing, Cambridge University
Press, New York, 2010. 19
 ASME V&V 40.4 example problems

Nitinol stent FEA Hip stem FEA Knee tibial tray FEA

ASTM F2514 ASTM F2996 ASTM F3334

 Nitinol stent FEA – mesh refinement study – example results (C3D8R)

Max. Prin. strain distribution

at stent delivery diameter
Nitinol stent FEA – conventional mesh refinement study (4x5, 5x6, …)
WxT

Mesh refinement level Mesh refinement factor (r)

T
Strut width Strut thickness
W 4x5 to 5x6 1.25 1.20
5x6 to 6x7 1.20 1.17
6x7 to 7x8 1.17 1.14

4x5 5 x 6 (coarse, mesh 3) 6x7 7x8

 Nitinol stent FEA – systematic mesh refinement study (r = 2)

mesh 3 mesh 2 mesh 1

(coarse) (medium) (fine)

Mesh Mesh Element

Number Name Size

r=2
(um)
3 coarse 25
2 medium 12.5
1 fine 6.25
 Nitinol stent FEA – mesh refinement study results
fine medium coarse
7x8 6x7 5x6 4x5

1 2 3

Observed order of accuracy

f −f 
pˆ1 = ln  3 2  / ln r
 f 2 − f1 

Richardson
 f −f 
pˆ1 = ln  3 2  / ln r Extrapolation
 f 2 − f1  (RE)
conventional
Richardson extrapolation +/- GCI
mesh refinement
estimate of exact solution ( pˆ1 = 0.96)
1
f1 − f 2 2 systematic mesh
f1 = f1 +
r pˆ1 − 1 refinement with
3 a refinement
Grid Convergence Index (GCI)
factor of 2
(numerical uncertainty)
( Fs )1 f 2 − f1
(GCI )1 =
r p1 − 1 f1
 Nitinol stent FEA – mesh refinement study results

Mesh refinement level

Element type Quantity of interest 4x5 coarse (5x6) 6x7 7x8
Max Prin Strain (mm/mm) 0.0686 0.0692 0.0709 0.0724
C3D8R
Relative Change (%) 0.86 2.43 2.06

Richardson Can we conclude on a

Extrapolation mesh convergence?
fi +1 − fi
 = (RE)
conventional
fi
mesh refinement
1
2 systematic mesh
refinement with
3 a refinement
factor of 2
 Origins of Roache’s grid convergence index (GCI)
“… considering the state of affairs in the early 1990s regarding the reporting of mesh refinement
studies. This time period is before the appearance of numerical accuracy policies for journals dealing
with scientific computing simulations. In some cases, authors mistakenly reported discretization
error estimates by giving the relative difference between two discrete solutions computed on
different meshes, i.e.,
f 2 − f1
 =
f1

… This relative difference can be extremely misleading when used as an error estimate.

… The desire to prevent misuse of this relative difference between discrete solutions as an error
estimator led to the development of Roache’s grid convergence index.”

-- Oberkampf W. L. and Roy C. J., Verification and Validation in Scientific Computing, Cambridge
University Press, 2010.

26
 f 2 − f1 Mesh refinement scenarios with pf = 1.0
 = → 5%
f1 ( f 2 = 0.95 & f1 = 1.00) f −f
= 2 1
f1
p f = 1.0  f1 − (GCI )1 f1 + (GCI )1
 f 3 f 2 f1 f1
r = 1.1  → Fs = 1.25
pˆ = 0.9  (GCI )1 (GCI )1

p f = 1.0 

r = 1.3  → Fs = 1.25
pˆ = 0.9 

p f = 1.0 

r = 2.0  → Fs = 1.25
pˆ = 0.9 
 f 2 − f1 Mesh refinement scenarios with pf = 2.0
 = → 5%
f1 ( f 2 = 0.95 & f1 = 1.00) f −f
= 2 1
f1
p f = 2.0  f1 − (GCI )1 f1 + (GCI )1
 f 3 f 2 f1 f1
r = 1.1  → Fs = 1.25
pˆ = 1.8  (GCI )1 (GCI )1

p f = 2.0 

r = 1.3  → Fs = 1.25
pˆ = 1.8 

p f = 2.0 

r = 2.0  → Fs = 1.25
pˆ = 1.8 
 Factors negatively impacting observed order of accuracy

• Under-resolved meshes due to computational resource limitations – discrete solutions are not in
the asymptotic range. Both the coarse mesh solution and the fine mesh solution needs to be in
the asymptotic range.
• Poor quality spatial mesh – can result in reduced order of accuracy (observed)
• Poor smoothness of the solution, presence of discontinuities – can result in reduced order of
accuracy (observed)
• Non-systematic mesh refinement – mesh refinement needs to be both uniform & consistent.
• Other relatively large numerical error sources – round-off error & iterative error needs to be
much smaller than the discretization error

Epsilon (||) can be extremely misleading when convergence is slow or stalled!

Oberkampf W. L. and Roy C. J., Verification and Validation in Scientific Computing, Cambridge University Press, New York, 2010.
29
 ASME V&V 40.4 example problems

Nitinol stent FEA Hip stem FEA Knee tibial tray FEA

ASTM F2514 ASTM F2996 ASTM F3334

 Hip stem FEA example problem (Danny Levine)

ASTM F2996 hip stem geometry Linear-elastic material model

F (Ti-6Al-4V)

Geometric nonlinearity is turned ON.

 Hip stem FEA – submodel mesh refinement (neck region)

submodel location
mesh 4 mesh 3
mesh for global model

mesh 2 mesh 1

submodel Mesh Element

Number Size
(mm)
4 1.5000
3 0.7500
2 0.3750
1 0.1875

edge
 Hip stem FEA – submodel mesh refinement results (neck region)

First principal stress distribution for model without

the edge at neck surface (virtual composite face)
mesh 4 mesh 3

mesh 2 mesh 1
 ASTM F2996 hip stem geometry (neck region)

ANSYS SpaceClaim

• Are the curvatures of two

neighboring surfaces
continuous at the edge? Is there section view at xz-plane
tangency? of unoriented geometry
• Is there a geometric singularity
at the edge?

edge
ASTM F2996 hip stem geometry (neck region)

Investigation by Kenny Aycock

ASTM F2996 hip stem geometry (neck region)

CAD modification in SolidWorks

by Kenny Aycock
ASTM F2996 hip stem geometry (neck region)

CAD modification in SolidWorks

by Kenny Aycock
 Hip stem FEA – global model mesh refinement
Starting with a relatively finer mesh at the regions of interest at the neck and refining uniformly across
the entire mesh.
global model (medium mesh) global model (fine mesh) global model (finer mesh)

r=2 r=2 quadratic

elements for
displacements
Hip stem FEA – global model mesh refinement results (neck region)

p̂3 = 1.93
p̂3 = N/A

f3 − f 2
pˆ1  0 for 1
 f3 − f 2  (observed order f 2 − f1
pˆ1 = ln   / ln r
−
 2 1
f f of accuracy) f3 − f 2
pˆ1 is undefined for 0
f 2 − f1
Hip stem FEA – submodel (neck region)
medium submodel mesh
(red) overlayed on
medium global mesh
 Hip stem FEA – submodel mesh refinement (neck region)
submodel
medium mesh (5) fine mesh (4)
Mesh Element
Number Size
(mm)
5 0.5000
4 0.2500
3 0.1250 r=2
2 0.0625
1 0.03125

finer mesh (3) finest mesh (2) finest2 mesh (1)

 Hip stem FEA – submodel mesh refinement results (neck region)

medium global mesh fine global mesh finer global mesh

p̂2 = 2.29
p̂1 = 1.72

p̂2 = 2.46

p̂2 = 2.36
p̂1 = 0.88

p̂1 = 0.90
p̂3 = 0.59

p̂3 = 0.82
p̂3 = 5.13
p̂1 = 0.97

p̂1 = 0.89
p̂3 = 3.01

p̂2 = 1.03
p̂2 = 0.98

p̂2 = 1.03
p̂1 = 0.90
p̂3 = 0.61

p̂3 = 0.61
f3 − f 2
pˆ1  0 for 1
 f3 − f 2  (observed order f 2 − f1
pˆ1 = ln   / ln r
 f 2 − f1 
of accuracy) f3 − f 2
pˆ1 is undefined for 0
f 2 − f1
 Hip stem FEA – mesh refinement results (neck region)

finer global finer global

model mesh model mesh

finer submodel mesh

finer submodel mesh
 ASME V&V 40.4 example problems

Nitinol stent FEA Hip stem FEA Knee tibial tray FEA

ASTM F2514 ASTM F2996 ASTM F3334

 Knee tibial tray FEA example problem (courtesy of Walter Schmidt)

ASTM F3334 knee tibial tray geometry Linear-elastic material models

tibial tray (Ti-6Al-4V) bone cement

E = 110 GPa E = 2.8 GPa
 = 0.3  = 0.3

Geometric nonlinearity is turned ON.

Loading steps
apply 1N
vertical load
 Knee tibial tray FEA – global model mesh refinement
Starting with a relatively finer mesh at the regions of interest at the posterior & anterior fillets and
refining uniformly across the entire mesh.
global model (medium mesh) global model (fine mesh) global model (finer mesh)
Knee tibial tray FEA – submodel (posterior fillet)

fine submodel mesh (red)

overlayed on
medium global mesh
Knee tibial tray FEA – submodel mesh refinement (posterior fillet)
submodel
medium mesh (5) fine mesh (4)
Mesh Element
Number Size
(mm)
5 0.2000
4 0.1000
3 0.5000 r=2
2 0.0250
1 0.0125

finer mesh (3) finest mesh (2) finest2 mesh (1)

Knee tibial tray FEA – submodel mesh refinement results (posterior fillet)
Knee tibial tray FEA – Submodel mesh refinement results (posterior fillet)
Knee tibial tray FEA – Submodel mesh refinement results (posterior fillet)
Knee tibial tray FEA – Submodel mesh refinement results (posterior fillet)
Knee tibial tray FEA – Submodel mesh refinement results (posterior fillet)
 Thank you!

Questions?

[email protected]

Acknowledgements: Members of ASME V&V 40.4 Verification Working Group

Prof. Chris Roy at Virginia Tech, Prof. Glenn Sinclair at Louisiana State
Majority of icons are from: www.flaticon.com

54
 ASME VVUQ-40 Standard
– Risk Analysis and Model Credibility
Marc Horner, Ph.D.

Vice Chair, ASME VVUQ-40

Sr. Principal Engineer, Ansys, Inc.

e: [email protected]

1
Risk-Informed Credibility Assessment Framework
- A brief review of Session 1

The VVUQ-40 standard outlines a process for making risk-

informed determinations as to whether CM&S is credible for
decision-making for a specified context of use.
Question of Interest and Context of Use
- A brief review of Session 1

The question of interest describes the specific question,

decision or concern that is being addressed.

Context of use defines the specific role and scope of the

computational model used to inform that decision.
 Risk Assessment

Model risk is the possibility that the model may lead to a

false/incorrect conclusion about device performance,
resulting in adverse outcomes.

- Model influence is the contribution of the

computational model to the decision relative to other
available evidence.

- Decision consequence is the significance of an adverse

outcome resulting from an incorrect decision.
* Blood pump image courtesy
Mark Goodin, SimuTech Group
Establish Credibility Goals

Model credibility refers to the trust in the

predictive capability of the computational
model for the COU.

Trust can be established through the collection

of VVUQ evidence and by demonstrating the
applicability of the VVUQ activities to support
the use of the CM for the COU.

Goals for each credibility factor are based on

model risk.
Establish Credibility Goals
- A Closer Look at the Credibility Factors

The standard includes a description of each

credibility factor along with a suggested
Description gradation of activities for each factor.

LOW RISK

HIGH RISK

Gradation of Activities
The gradations of activities are mapped
to the risk associated with the COU.

6
Establish Credibility Goals

RISK
low medium high

7
In Summary

9
THANK YOU

10
Solution verification
case study
ASTM standard round robin

Cheryl Liu, Ph.D.

Sr. Principal Engineer | Modeling and Simulation

Stryker Orthopaedics

GSNPS-PRE-213_31089
Introduction

• ASTM F04.22 committee published a hip femoral stem FE

analysis standard document (F2996-13 Standard Practice
for Finite Element Analysis (FEA) of Non-Modular Metallic
Orthopaedic Hip Femoral Stems) in 2013 after six
years and three round robins
• A new standard for a total knee replacement (TKR) femoral
ASTM F2996-13
component was initiated in 2013
o Range of results of first round robin was larger than acceptable
• A sensitivity study was conducted on model inputs
o Resulted in revised instructions for the second round robin
o Included requirement to report mesh convergence results

GSNPS-PRE-213_31089
Methods

• The proposed linear, static FEA model is a knee femoral component under a unit
closing load applied to the medial posterior condyle while the anterior flange is
held fixed, consistent with a physical test method

GSNPS-PRE-213_31089
Methods

• The context of use for the FEA model is to identify the worst-case maximum stress
in the condyle and intercondylar notch within a family of sizes to reduce physical
testing
• The specific model is a generic Cobalt Chromium alloy knee femoral component.
Variations investigated:
o Boundary conditions
o Mesh refinement levels
o FEA packages

GSNPS-PRE-213_31089
Results

• The anterior flange boundary condition had a significant

effect on the computed peak stresses
o Fixing the bone cut plane resulted in a higher peak
stress than fixing the surfaces of the anterior flange
o Fixing the surfaces of the flange better represents the
physical test
o Second round robin instructions were clarified
accordingly

GSNPS-PRE-213_31089
Results

• The model was very sensitive to the mesh refinement level

o An absolute criterion for mesh convergence was difficult to establish and
potentially misleading
o The participants were requested to perform at least three levels of mesh
refinement studies and conclude for themselves when a converged solution had
been reached
• There were minimal differences between the two FEA softwares studied when
equivalent boundary/loading conditions and similar mesh refinement were used

GSNPS-PRE-213_31089
Results

• With these new instructions, the second round robin resulted in much higher
consistency among participants
o Less than 1% overall range for the condyle stress and less than 1.5% overall
range for the notch stress from 13 participants

GSNPS-PRE-213_31089
Discussion

• When mesh convergence results are required to be reported, participants of round

robin were able to reach solutions that are much closer with each other
o Despite the fact each participants chose their own criteria for mesh convergence

• It’s important to conduct solution verification for the quantity of interest

• Engineering judgement can be used to define acceptance criteria

o Based on the context of use of the model

GSNPS-PRE-213_31089
 Acknowledgement
Jeff Sprague
Janaki Penmetsa
Thank
Walter Schmidt
you
Disclaimer
A surgeon must always rely on his or her own professional clinical judgment when
deciding whether to use a particular product when treating a particular patient.
Stryker does not dispense medical advice and recommends that surgeons be trained
in the use of any particular product before using it in surgery.

The information presented is intended to demonstrate the breadth of Stryker's

product offerings. A surgeon must always refer to the package insert, product label
and/or instructions for use before using any of Stryker's products. Products may not
be available in all markets because product availability is subject to the regulatory
and/or medical practices in individual markets. Please contact your sales
representative if you have questions about the availability of products in your area.

Stryker Corporation or its divisions or other corporate affiliated entities own, use or
have applied for the following trademarks or service marks: Stryker. All other
trademarks are trademarks of their respective owners or holders.

GSNPS-PRE-213_31089
 Computational Modeling and Simulation to Support Implant Design
and Evaluation and Clinical Decision Making – Practical
Considerations on Model Credibility
PART 2: Code and solution verification considerations for translational
computational modeling & simulation applications

CASE 2 – NATURAL KNEE MODEL

Ahmet Erdemir
Department of Biomedical Engineering
Lerner Research Institute
Cleveland Clinic

September 22, 2021

DISCLOSURES

• NO UNLABELED USE
• NO INVESTIGATIONAL PRODUCTS
• AFFILIATIONS WITH COMMERCIAL FIRMS
• Innodof, LLC (Owner, non-medical modeling and simulation)
• Kitware, Inc. (Collaborator, federal grant)
• IBM (Collaborator, institutional)
• CURRENT FUNDING
• Reproducibility in Simulation-Based Prediction of Natural Knee Mechanics
• NIH/NIBIB R01EB024573 (09/21/2017 – 06/30/2022)
• Software for Practical Annotation and Exchange of Virtual Anatomy
• NIH/NIBIB R01EB025212 (07/02/2019 – 03/01/2023)
GOAL OF THE PRESENTATION
• To summarize verification
activities for a natural knee
model

• To establish correspondence of
verification activities to ASME
VVUQ-40

WHAT DID WE DO?

WHAT WILL YOU DO?
CONTEXT OF USE

• To capture knee-specific
movement signature during
passive flexion (demonstration
model)

• To predict joint movement and

tissue mechanics in prospective
use cases (template model)
MODEL RISKS

• Demonstration of knee-specific
movement signature
• Evaluation of modeling forms
• Knee-specificity after calibration
-> MEDIUM RISK
high influence, low consequence

• Prospective use cases Model form vs verification uncertainties

• Risk needs to be reassessed
• Depends on new context of use Calibration fit
• Depends on output of interest error vs
• e.g., diagnostic biomarker verification
-> HIGH RISK uncertainties
high influence, high consequence
SIMULATION OUTPUTS

• Joint level
• Tibiofemoral joint kinematics-
kinetics (6 dof)
• Patellofemoral joint kinematics-
kinetics (6 dof)

• Tissue level
• Tissue loads
• Ligament forces
• Contact forces
• Tissue stress-strain distributions
• MIND FOR
!
Cartilage contact pressures
• Meniscus contact pressures
• Ligament fiber stretch CONTEXT RELEVANT
• Meniscus fiber stretch
OUTPUTS
SIMULATION SOFTWARE CONSIDERATIONS

• Nonlinear finite element analysis

• Large deformations
• Constitutive models
• Contact/tie formulations
• Implicit quasi-static analysis

• Specific to knee modeling

• Joint connectors (joint movements)
• In situ strain (ligaments)

• Free and open source (enabling

access)
CODE VERIFICATION

• Broad software capabilities and

quality inferred from simulation
software resources

• Manuals
• Publications
• Test problems
• Forums
• Release notes
CODE VERIFICATION

• Features specific to knee

modeling evaluated through
• Collaborative development
• Scholarly publications
• in situ strain
• Comparison to analytical solutions
• joint connectors
A B C

Verification of in situ strain application

A. Forward simulation
B. Deformation gradient based
C. Fiber stretch based
SOLUTION VERIFICATION

• Mesh convergence for multiple

tissue structures
• Compartmental modeling
• Tensile testing of ligaments
• Compression testing of cartilage &
menisci
• Primary criteria
• 5% change in peak reaction force
• Secondary criteria
• Fiber stretch (ligaments)
• Contact pressure (cartilage, menisci)
• Other considerations
• Simulation cost
SOLUTION VERIFICATION
ACL Mesh Convergence IPSR4 IPSR6 IPSR8 IPSR10
Linear tetrahedral elements
• Initial mesh
• previous modeling experience
• visual inspection
• inherited from geometry
generation
• Uniform increase in mesh size
• Multiple mesh densities Tensile Force
• 4 levels of resampling Fiber Stretch
• e.g., IPSR4, IPSR6, IPSR8, IPSR10 0.89 – 1.19
(range)
• Element types
• linear vs quadratic tetrahedral Reaction Force (N) 264 205 191 188
Computation Time (sec) 41 95 150 364
SOLUTION VERIFICATION
Lateral Meniscus IPSR4 IPSR6 IPSR8 IPSR10
Mesh Convergence
• Initial mesh Linear tetrahedral elements
• previous modeling experience
• visual inspection Contact Pressure (MPa)
• inherited from geometry 0.89 – 1.19 (range)
generation
ACL Element Types
• Uniform increase in mesh size linear quad
linear quad
• Multiple mesh densities
• 4 levels of resampling
• e.g., IPSR4, IPSR6, IPSR8, IPSR10 IPSR6 IPSR8
• Element types
• linear vs quadratic tetrahedral
205 N 161 N 191 N 160 N
95 s 924 s 150 s 2067 s
SOLUTION VERIFICATION

Tibiofemoral kinematics during passive flexion before and after mesh convergence study
w/ initial coarser meshes w/ converged finer meshes
SOLUTION VERIFICATION

• Simulation control settings

• Contact settings
• Constraint settings
• All iteratively changed from
• default parameters
• previous model experiences

Reduce computational cost

~ 6 hours for passive flexion
Facilitate numerical convergence
SOLUTION VERIFICATION
• Review of inputs/outputs
• Specifications
• Protocol deviations

• Geometry verification
• Smoothing
• Segmentation variability

• Correspondence to independent literature

• Material properties (input)
Adjustment of ACL properties
• Structural properties (output)
Fiber modulus:
• Public dissemination -> 535 MPa to 5000 MPa
• Data
• Model Tensile stiffness:
• Documentation -> 126 N/mm to 195 N/mm
TAKE HOME MESSAGES

• In-depth verification is possible but

tedious.
• Guidance is available to design and
conduct verification in relation to context
of use and model risk.
• Automation is helpful.
• Bookkeeping
• Model assembly
• Simulation
• Post-processing
• Modeler is responsible
• to understand simulation software features,
• to know software limitations, and
• to prescribe inputs intentionally. Check our work @
https://simtk.org/projects/kneehub
• Software developers share burden. https://simtk.org/projects/openknee
TO EXERCISE …

https://forms.gle/dNA2cBRuAoqW2Pos9

• Enter your example with your own

modeling problem

• Enter your example for a reuse case of

the natural knee model

• Check handouts for references and

further reading on good practices