J Telemed Telecare OnlineFirst, published on July 20, 2016 as doi:10.

1177/1357633X16660640

RESEARCH/Original Article

Journal of Telemedicine and Telecare

0(0) 1–10
Real-time teleophthalmology ! The Author(s) 2016
Reprints and permissions:
versus face-to-face consultation: sagepub.co.uk/journalsPermissions.nav
DOI: 10.1177/1357633X16660640

A systematic review jtt.sagepub.com

Irene J Tan1, Lucy P Dobson1, Stephen Bartnik1,2,

Josephine Muir1,3 and Angus W Turner1,3

Abstract
Introduction: Advances in imaging capabilities and the evolution of real-time teleophthalmology have the potential to provide
increased coverage to areas with limited ophthalmology services. However, there is limited research assessing the diagnostic
accuracy of face-to-face teleophthalmology consultation. This systematic review aims to determine if real-time teleophthalmol-
ogy provides comparable accuracy to face-to-face consultation for the diagnosis of common eye health conditions.
Methods: A search of PubMed, Embase, Medline and Cochrane databases and manual citation review was conducted on 6
February and 7 April 2016. Included studies involved real-time telemedicine in the field of ophthalmology or optometry, and
assessed diagnostic accuracy against gold-standard face-to-face consultation. The revised quality assessment of diagnostic
accuracy studies (QUADAS-2) tool assessed risk of bias.
Results: Twelve studies were included, with participants ranging from four to 89 years old. A broad number of conditions were
assessed and include corneal and retinal pathologies, strabismus, oculoplastics and post-operative review. Quality assessment
identified a high or unclear risk of bias in patient selection (75%) due to an undisclosed recruitment processes. The index test
showed high risk of bias in the included studies, due to the varied interpretation and conduct of real-time teleophthalmology
methods. Reference standard risk was overall low (75%), as was the risk due to flow and timing (75%).
Conclusion: In terms of diagnostic accuracy, real-time teleophthalmology was considered superior to face-to-face consultation
in one study and comparable in six studies. Store-and-forward image transmission coupled with real-time videoconferencing is a
suitable alternative to overcome poor internet transmission speeds.
Keywords
Remote consultation, teleconsulting, telemedicine, teleophthalmology

Date received: 10 May 2016; Date accepted: 29 June 2016

Introduction between the clinician and the referrer is established.

Telemedicine is defined as the use of information and Real-time teleophthalmology attempts to more closely
communication technologies to provide health care ser- mimic a traditional face-to-face consultation. It allows
vices to patients from a distance.1 A major area of appli- the clinician to explore additional history or examination
cation of this system has been in the field of findings and enables dialogue between the clinician, refer-
ophthalmology, with the first project published in 1975.2 rer and/or patient.
Since then, advancements in imaging capabilities and Multiple studies have evaluated the accuracy,11–20 reli-
technologies have seen the integration of telemedicine ability,11,13,15,19,21,22 cost-effectiveness23–35 and level of
into clinical practice. patient satisfaction7,36,37 in asynchronous teleophthalmol-
Asynchronous, or store-and-forward, teleophthalmol- ogy. Although asynchronous teleophthalmology has been
ogy relies on the capturing of clinical information and
images, and evaluation of this information by an off-site 1
Lions Outback Vision, Lions Eye Institute, Australia
clinician. The clinician makes a diagnosis and formulates a 2
Sydney School of Public Health, University of Sydney, Australia
3
management plan, relaying this information to the referrer Centre for Ophthalmology and Visual Science, University of Western
at a later time. Diabetic retinopathy screening is perhaps Australia, Australia
the best example of this teleconsultation, with multiple
Corresponding author:
screening programmes currently in use worldwide.3–10 Irene J Tan, Lions Outback Vision, Lions Eye Institute, 2 Verdun St, Nedlands,
Synchronous, or real-time, teleophthalmology is an Western Australia, 6009, Australia.
emerging application, whereby a real-time connection Email: [email protected]

Downloaded from jtt.sagepub.com at CORNELL UNIV on October 13, 2016

2 Journal of Telemedicine and Telecare 0(0)

successfully applied to diabetic retinopathy, its utility Studies were excluded if the telemedicine technology
is largely restricted in a screening capacity. In con- employed store-and-forward methods only and/or if
trast, real-time teleophthalmology is focused on the animal subjects were used. Reasonable attempts were car-
diagnosis, management and therapeutic relationship ried out to source abstracts and full texts missing from the
between an ophthalmologist and a patient, and is less initial search citations. These attempts included requests
well studied. through library sources, contact with editorial staff of rele-
The aim of this systematic review is to determine vant journals and in one case direct contact with the
whether real-time teleophthalmology provides compar- author. However, inability to source the full text resulted
able accuracy to face-to-face consultation for the diagno- in exclusion of the study.
sis and recommended treatment of common eye health
conditions. Studies involving synchronous teleconsulta-
tion are logistically difficult to design in a real-world set-
Study selection
ting, and thus we endeavour to assess the quality of Two reviewers (IT and LD) independently examined titles
research in this field to answer our review question. and abstracts from the database search. Studies were
selected for inclusion based on the eligibility criteria.
Abstracts with ambiguity in either the method of tele-
Methods ophthalmology utilised or comparison to face-to-face con-
sultation were included for further full text review.
Search strategy
Reviewers compared a selection of full texts for review
We searched the electronic databases of PubMed, Embase and conflicts around inclusion were resolved by consensus
(OVID interface), Cochrane and Medline (OVID inter- discussion. An independent full text review with applica-
face) for studies that compared real-time teleophthalmol- tion of the eligibility criteria was carried out by the third
ogy with face-to-face consultation. We did this by reviewer (SB). The final studies for inclusion were com-
constructing our search strategy (Table 1) based on med- pared and conflicts were again resolved by consensus
ical subject headings and text terms. The search was con- discussion.
ducted on 6 February 2016 and repeated on 7 April 2016.
One reviewer scanned references of eligible studies to iden-
tify further relevant texts not retrieved by database search.
Data collection and quality assessment
The search strategy was not restricted by time or Full texts were reviewed and data extracted onto a form to
language. highlight characteristics of each study. Extraction was per-
Literature search results were then uploaded into formed by one reviewer and verified by a second reviewer.
Endnote X7 (Thomson Reuters, USA), an electronic ref- The information for data extraction included: year of pub-
erence management software. Duplicates were first identi- lication; location; sample size; eyes tested; age of subjects;
fied using this software and reviewers then manually diseases identified; descriptor of index test; descriptor of
searched the existing list to identify further duplications reference standard; examiner qualification; outcomes mea-
with different citations. The latter were compared by sured; statistical outcome (if relevant); overall outcome of
author names, year and journal of publication, methods study indicating a preference to method (real-time versus
and results in the abstract, prior to extraction. face-to-face) and video/image size and transmission
format.
The revised tool for the quality assessment of diagnos-
Eligibility criteria
tic accuracy studies (QUADAS-2) was applied to the
Studies were included applying the following criteria: (a) included studies to assess the risk of bias and applicability
employed an element of real-time telemedicine (video or to our review question.38 The reference standard test in
audio); (b) conducted in an ophthalmology or optometry our review is classified as face-to-face consultation with an
environment and (c) compared the diagnostic accuracy ophthalmologist. Reviewers customised signalling ques-
and efficacy of a real-time teleophthalmology method tions of the QUADAS-2 tool for assessment of risk of
against a face-to-face consultation. bias. Following customisation and trialling of the
QUADAS-2 tool, the two reviewers (IT and LD) inde-
pendently applied the tool to the selected studies. Where
there was discrepancy in the results of the tool between
Table 1. Search strategy.
reviewers, a third independent reviewer (SB) was recruited
Search Syntax as an arbitrator.
1 ((Telemedicine OR Telehealth OR remote consultation
OR tele OR ehealth OR emedicine OR video con- Data synthesis and analysis
ferencing OR teleconsultation) AND (ophthalmology
OR optometry)) OR (teleophthalmology OR The characteristics of the studies and quality analysis are
teleoptometry) presented in tables and text to aid the presentation of the
systematic narrative synthesis. The review retains studies

Downloaded from jtt.sagepub.com at CORNELL UNIV on October 13, 2016

Tan 3

with any level of bias in analysis, shown in tables recom- evaluated non-selected general ophthalmology presenta-
mended by the QUADAS-2 tool, and discussions of the tions.22,40,45,46 Study characteristics and the description of
principal outcomes of interest occur through text. their real-time method are summarised in Table 2.
Real-time teleophthalmology techniques varied greatly
between the studies. These can be categorised into widely
Results available basic videoconferencing equipment combined
The database search yielded 627 reference results (after with store-and-forward image transmission, intermediate
duplicate extraction) and, of these, 12 studies were telehealth technology utilising a real-time video feed from
included in the final assessment of quality via application examination equipment, and advanced teleophthalmology
of the QUADAS-2 tool. Figure 1 represents the applica- technology. Three studies utilised basic videoconferencing
tion of the preferred reporting items for systematic reviews technology,40,42,45 and intermediate-level technology was
and meta-analysis (PRISMA).39 utilised in eight studies which included live-feed from a
slit-lamp22,44,46–49 or direct ophthalmoscope.43 Two stu-
dies assessed innovative advanced teleophthalmology
Study characteristics of included studies technology. Tanabe et al.50 studied a novel remote oper-
Publication dates spanned from 1997 to 2015. The studies ated slitlamp system and Tan et al.41 utilised an integrated
were conducted in Asia-Pacific (n ¼ 4), North America software package which displayed a shared electronic
(n ¼ 4), the United Kingdom (n ¼ 3) and Europe (n ¼ 1). whiteboard to manipulate images and video between the
Five of the studies40–44 specified ages of participants, ophthalmologist and patient. Methods of image and video
with an inclusive range from four years to 89 years. transmission for the teleophthalmology component were
Ocular conditions evaluated were diverse and covered not explicitly stated in all studies, but those that
corneal and retinal pathologies, strabismus, oculoplastics did showed variation in transmission speed, with
and post-operative review. Four of the included studies 384kbits per second as the most commonly used
Identification

Records identified through Additional records identified

database searching through references search
(n = 1364 ) (n = 6 )

Records (n= 627)after duplicates

removed
Screening

Records screened Records excluded

(n = 627 ) (n = 542)

Full-text articles
excluded
Eligibility

Full-text articles
(n = 73)
assessed for eligibility
(n = 85) • Commentary/ Editorial
(n=20)
• Nil full text (n= 31)
• Nil Real-time element / Store
and forward only (n=12)
• Nil comparison between
methods (n=8)
Studies included in • Nil diagnostic accuracy
Included

outcome (n = 2)
qualitative synthesis
(n = 12)

Figure 1. PRISMA representation of study selection.

Downloaded from jtt.sagepub.com at CORNELL UNIV on October 13, 2016

4 Journal of Telemedicine and Telecare 0(0)

Table 2. Study characteristics.

Sample size
(patient/
Author eyes) Age range Eye conditions Real-time method Transmission

Bar-Sela and 49 (98) NS Complicated emergency Store-and-forward with Images ¼ 200 kB JPEG
Glovinsky, room presentations real-time audio Video clips ¼ 700 kB/sec
200745 conference MPEG-2
Transmission ¼ ADSL; 0.1
Mbit/sec and 1.5Mbit/
sec respectively
Hagman et al., 22 (NS) 7–82 years General conditions Real-time VC. Images ¼ wavelet com-
200440 Two VC units pression
Video ¼ 25 frames/Sec
Transmission ¼ LAN
Tan et al., 30 (30) 21–75 years Chronic blurred vision Combination store-and- NS
201341 including: forward with real-time
maculopathy; media opacity; VC
optic neuropathy and
keratopathy
Tanabe et al., 29 (NS) NS Cataract surgery Real-time VC with novel Digital video transport
201150 Trabeculectomy remote slitlamp system ¼ non-com-
Vitrectomy pressed 30 frames/sec.
Scleral buckling 30 Mbit/sec
Retinal detachment
Nitzkin et al., 6 (NS) NS General Combination store-and- NS
199722 forward with real-time
VC
Cheung et al., 85 (NS) 4–66 years Paediatric strabismus Real-time VC Video transmission ¼ 224
200042 kbit/sec.
30 frames/sec
Marcus et al., 37 (73) 25–65 years HIV HIV – retinopathy vs CMV Real-time video of direct Video transmission ¼ 1.5
199843 32–66 years DM DM – CSMO, NPDR, PDR ophthalmoscopy. MBit/sec
Optic nerve changes Monitoring and viewing
Cataracts ophthalmologist in same
room as patient
Peter et al., NS (48) NS Diabetic macula oedema Real-time video slitlamp Video and image
200648 CSMO and VC transmission ¼ ISDN
128 kbit/sec per line. Up
to 348 kbit/sec
Bremner et al., 6 (NS) NS Neuro-ophthalmology clinic Resident and ophthal- Video transmis-
200249 patients mologist real-time video sion ¼ 384kbit/sec
slitlamp and VC
Rayner et al., 17 (NS) NS Adnexal conditions – con- Real-time VC with video Video
2001 47 genital and aponeurotic slitlamp and digital transmission ¼ ISDN
ptosis camera 384kbit/sec.
Lid swellings, basal cell car- 30 frames/sec
cinoma, phthiscal eye
Dawson et al., 30 (NS) 20–89 years Strabismus Real-time VC with video Video transmis-
200244 slitlamp and digital sion ¼ 384kbit/Sec
camera ISDN.
30 frames/sec
Threlkeld et al., 25 (50) NS Ocular adnexa and anterior Real-time video slitlamp Video
199946 segment controlled by non- transmission ¼ 1.544M-
ophthalmologist bit/s
NS: not specified; VC: video conference; HIV: human immunodeficiency virus; DM: diabetes mellitus; CSMO: clinically significant macula oedema; NPDR:
nonproliferative diabetic maculopathy; PDR: proliferative diabetic maculopathy; CMV: cytomegalovirus; ADSL: asymmetric digital prescriber line; ISDN:
integrated services digital network; LAN: local area network; kB: kilobyte; kbit: kilobit; Mbit: Megabit.

Downloaded from jtt.sagepub.com at CORNELL UNIV on October 13, 2016

Tan 5

transmission44,47–49 and the fastest transmission at Discussion

1.5Mbit per second.43,45,46
Summary of evidence
All included studies assessed the outcomes from exam-
inations conducted by consultant ophthalmologists, Overall, the diagnostic accuracy of teleophthalmology is
excluding a sub-study within Marcus et al.43 whereby comparable to face-to-face consultation but has some
third year ophthalmology residents reviewed a live direct limitations. The efficacy of diagnostic accuracy is affected
ophthalmoscopy video feed operated by a primary care by the quality of information provided to the clinician,
physician. There was greater variation in the ophthalmol- particularly for live video-feed examination. In the face
ogy skills of the operator co-located with the patient in the of an increasingly strained health system, it is important
telemedicine arm. The onsite operator was an ophthal- to redesign service delivery to maximise utilisation of
mologist or ophthalmology fellow in one study,48 ophthal- resources currently available. Real-time teleophthalmol-
mology resident in four studies,40,45,47,49 ophthalmic ogy has the potential to reduce the economic costs and
technician or orthoptist in three studies,41,42,44 and other service coverage limitations currently seen in the trad-
health care professionals or unstated assistants in four itional consultation setting. There may be large initial
studies.22,43,46,50 costs associated with installing the information technol-
In terms of overall diagnostic accuracy, real-time tele- ogy required for effective teleconsultation. However, stu-
ophthalmology was considered superior to face-to-face dies utilising basic videoconferencing technology alone,
consultation in one study41 and was comparable in six such as in Rayner et al.47 and Dawson et al.,44 had similar
studies.40,42–45,47 Results for each study are reported in overall diagnostic accuracy outcomes favouring telehealth
Table 3. when compared to studies with more advanced tele-
ophthalmology equipment.41,50 This highlights that effect-
ive teleconsultation is possible with technology currently
Quality of included studies widely available to health practitioners in remote and
The risk of bias in patient selection was high in 33% of rural locations. The effectiveness of this interaction, how-
studies and 42% were considered unclear. This was due to ever, is largely dependent on internet transmission speed,
limited information regarding the patient recruitment pro- particularly when required for remote examination. In this
cess or patient selection within the methods of the pub- instance, a hybrid of asynchronous image transmission
lished studies. paired with real-time consultation overcomes this
For the index test, 33% were judged to be at high risk limitation.
of bias and 8% were considered unclear. The main con-
cern around the index test assessment was the variability
of the index test methods amongst studies. These were
Limitations
often novel and the subsequent interpretation or conduct On reviewing the QUADAS tool results, we note that the
by the researcher of the index test was associated with largest source of bias is in the selection of patients. In
high risk of bias. more than half of studies there was limited disclosure on
In the reference standard domain, the risk of bias was the process of recruiting study participants, thereby
high in 25% of cases and low in 75%. Such a result reveals increasing the selective bias to patients who would per-
that the majority of the studies conducted a reference form well for the telehealth consultation.
standard examination by a clinician at an appropriately The greatest limitation of this review was the lack of
trained level (ophthalmologist) and the results of the ref- consistent outcome measures across various study designs.
erence test were interpreted without the knowledge of the Our study question was aimed at an overview of real-time
index test. teleophthalmology in its multiple applications, which
The domain of flow and timing also had 25% of the accordingly included a range of studies with different
studies at high risk. This was because the timing between methods and measured outcomes. As a consequence,
the reference and index test was not appropriate, particu- each study reported findings using various statistics,
larly in studies with minutes or days between each test from specificity and sensitivities,41,43,46,48 to self-deter-
with the same examiner.22,45,50 In addition to this, the mined feasibility scores45 and subjective assessment.49
unnecessary exclusion of patients from data analysis had Our study conclusions are therefore based on the perform-
the potential to introduce bias.41 ance of outcome measures specific to each report, with
The overall applicability of the domains pooled specificity and sensitivity information limited to
(patient selection, index test and reference standard) the four studies in which this statistical measure was
was of low concern. Nitzkin et al.22 was the only reported.
study to raise some concern around the applicability of A second limitation of our study was the difficulty in
patient selection. This was due to the unclear nature of accessing full texts of references accepted based on title
patient selection in the study and thus the selection not and/or abstract information. The reviewers utilised library
matching the review question. Table 4 represents a sum- document request facilities and contacting journals and
mary analysis of the quality assessment, QUADAS-2 tool authors directly; however, success was low. This was
results. likely due to the age of the publication and the majority

Downloaded from jtt.sagepub.com at CORNELL UNIV on October 13, 2016

 6

Table 3. Study results.

Reference Real-time telehealth or
Author Study design Eye conditions Telehealth standard face-to-face outcome

Bar-Sela and Feasibility Ocular surface Feasibility mean scores (range): Agreement in Comparable
Glovinsky, Anterior chamber 89 (70–100) diagnosis for
200745 Anterior chamber angle 87 (80–90) all cases
Pupils 95 (90–100) seen with
Lens 90 telehealth.
Posterior pole inc. optic 86 (80–95) Nil statis-
nerve head 90 tical analysis.
Macula 85(80–90)
Hagman et al., Feasibility General conditions Clinician preference for real-time NS Comparable
200440 video equipment in percentage Current technology
Sony SLC-VL10 29% seems sufficient for
Tandberg image 9% real-time telehealth
Sony ¼ Tandberg 62%
Tan et al., 201341 Prospective study Chronic blurred vision: Sensitivity 96%; specificity 100% NA Telemedicine
Visually significant media opa- Sensitivity 100%; specificity 100%
city Sensitivity 100%; specificity 92%
Maculopathy Sensitivity 100%; specificity 100%
Optic neuropathy, any type
Keratopathy
Tanabe et al., NS Cataract surgery NS NS Conventional slitlamp
201150 Trabeculectomy
Vitrectomy
Scleral buckling
Retinal detachment
Nitzkin et al., NS General Kappa coefficient NS Reliability of telemedicine

Downloaded from jtt.sagepub.com at CORNELL UNIV on October 13, 2016

199722 Group 1 – 0.61 compared to face-to-
Group 2 – 0.75 face improves with
experience in ophthal-
mic examination
Cheung et al., Prospective inter- Paediatric strabismus cat- Intraclass correlation coefficient (kappa) 0.79; 0.69 Comparable
200042 observer agree- egory: 0.66; 0.74 (good) (good)
ment studies Horizontal 6 m and 0.33 m 0.28; 0.25 (poor) 0.83; 0.91
fixation 0.79;0.70 (good) (excellent)
Vertical 6 m and 0.33 fixation 0.78;0.65 (good) 0.91; 0.92
Angle of deviation: (excellent)
Horizontal 6 m and 0.33 m 0.87; 0.90
fixation (excellent)
Vertical 6 m and 0.33 fixation
(continued)
Journal of Telemedicine and Telecare 0(0)
 Table 3. Continued
Tan

Reference Real-time telehealth or

Author Study design Eye conditions Telehealth standard face-to-face outcome

Marcus et al., Prospective com- HIV retinopathy Sensitivity 83.3%; specificity 95.2% 100% of tele- Comparable in HIV
199843 parative case DM – CSMO, NPDR, Sensitivity 29%; specificity 100% health for Not comparable in DM,
series PDR Sensitivity 50%; specificity 100% HIV retin- particularly in those
DM – optic nerve Sensitivity 41%; specificity 100% opathy diag- with media opacity
changes nosis ¼ face-
DM – cataracts to-face diag-
nosis with-
out cataract
Peter et al., Prospective pilot Diabetic macula oedema þ Sensitivity 38%; specificity 95% NA Face-to-face
200648 study CSMO (Note: store-and-forward
photographs sensitivity
75%; specificity 95%)
Bremner et al., NS Neuro-ophthalmology clinic NS NS Face-to-face
200249 patients
Rayner et al., NS Adnexal conditions – con- 18% of cases had errors in diagnosis and management Comparable for specific
200147 genital and aponeurotic 58% of cases had full agreement adnexal conditions –
ptosis 24% similar diagnosis with different treatment plan untreated congenital
Lid swellings, basal cell car- and aponeurotic ptosis
cinoma, phthiscal eye
Dawson et al., NS Strabismus 80% full agreement in Comparable for manifest
200244 diagnosis and management strabismus
3.3% partial agreement Telehealth inferior for
16.6% nil agreement latent strabismus and
micro-movements or
non-compliant patients
Threlkeld et al., NS Eyelid mass Sensitivity 100%; specificity 64% NA Face-to-face
199946 Conjunctival pigment Sensitivity 100%; specificity 85%
Posterior synechiae Sensitivity 100%; specificity100%

Downloaded from jtt.sagepub.com at CORNELL UNIV on October 13, 2016

Blepharitis Sensitivity 80%; specificity 0%
Iridotomy Sensitivity 83%; specificity 93%
Pinguecula Sensitivity 70%; specificity 93%
Iris lesion Sensitivity 75%; specificity 93%
Corneal scar Sensitivity 56%; specificity 98%
Chamber inflammation Sensitivity 0%; specificity 100%
Nuclear cataract Sensitivity 57%; specificity 93%
IOL Sensitivity 37%; specificity 100%
Cortical cataract Sensitivity 100%; specificity 91%
Vitreous in chamber Sensitivity 0%; specificity 100%
Chalazion Sensitivity 0%; specificity 100%
Keratitis Sensitivity 0%; specificity 100%
Follicles/papillae Sensitivity 0%; specificity 94%

NA: not applicable; NS: not specified; HIV: human immunodeficiency virus; DM: diabetes mellitus; CSMO: clinically significant macula oedema; NPDR: nonproliferative diabetic maculopathy; PDR: proliferative diabetic
maculopathy; IOL: intraocular lens
7
8 Journal of Telemedicine and Telecare 0(0)

Table 4. QUADAS-2 results.

Risk of bias Applicability concerns

Patient Index Reference Flow & Patient Index Reference

Study selection test standard timing selection test standard

Bar-Sela and Glovinsky, 200745 L L L H L L L

Hagman et al., 200440 L L H L L L L
Tan et al., 201341 H L L H L L L
Tanabe et al., 201150 H H H L L L L
Nitzkin et al., 199722 U H H H U L L
Cheung et al., 200042 L L L L L L L
Marcus et al., 199843 H H L L L L L
Peter et al., 200648 H L L L L L L
Bremner et al., 200249 U L L L L L L
Rayner et al., 200147 U L L L L L L
Dawson et al., 200244 U U L L L L L
Threlkeld et al., 199946 U H L L L L L
L: low risk; H: high risk; U: unclear risk

of these references having been published in magazines confidence.54,55,61 Despite the lack of research specifically in
rather than journals. It is expected that the inclusion of real-time ophthalmology consultations, the experiences in
these references would not have influenced our findings as other disciplines have the potential to be applied to real-
they would be excluded on the absence of face-to-face time teleophthalmology and may provide valuable lessons
reference consultation. Overall, the conclusions of this to improve the real-world application.
review are restricted by the lack of research in real-time
teleophthalmology and the methodological and quality
variation between projects.
Conclusion
Studies on the accuracy of real-time teleconsultation In terms of the application of this review to clinical prac-
are widespread across medical and surgical disciplines tice, we note that teleophthalmology is successful at
including dermatology,51–53 neurology,54–56 psych- detecting gross changes, while subtle changes are deter-
iatry,57–59 otolaryngology,60 neurosurgery,61 rheumatol- mined by the quality of imaging. The difficulty in deter-
ogy,62 oncology63 and burns.64 Outcomes of these mining the overall diagnostic accuracy of real-time
studies have shown lower accuracy in real-time telehealth teleophthalmology is the range of conditions which we
consultations compared to face-to-face consultations; have included in this review. In order to determine
however, there is future promise with further techno- whether the performance of a test, in this case real-time
logical developments. teleophthalmology, is sufficiently accurate for diagnosis,
Psychiatric assessment using telephone consultation consideration must be given to the implications of a
and videoconferencing has been found to be agreeable false positive or false negative result. If the consequence
to face-to-face consultation, as by nature this assessment of a false negative result is serious, a test should have high
is less reliant on the video transmission quality.58 sensitivity. Similarly, in diseases with low prevalence, a
However, in oncology telehealth, diagnosis of malignant high specificity would be desirable.66 It must be remem-
breast lesions requires physical examination and as such it bered, however, that sensitivity and specificity values over-
was noted that diagnostic accuracy for carcinoma was simplify teleconsultation, which is not a diagnostic test.
lower than benign breast conditions when conducted Consultation is a process of diagnosis and management
through telephone consultation.57 which is influenced by the clinical acumen of the clinician.
Improvements in accuracy and confidence in real-time tel- Studies determined to be higher in quality were performed
econsultation is variable across the disciplines; however, there under study conditions and may not be practical to imple-
are examples of improvement in teledermatology, neurology ment into clinical practice.40,43,48
and neurosurgical teleconsultations. Teledermatology studies In terms of diagnostic accuracy, real-time teleophthal-
suggest a hybrid of live interactive consultations combined mology was considered superior to face-to-face consult-
with store-and-forward imaging as a method of improving ation in one study and comparable in six studies.
accuracy,65 which reflects the findings of our review in oph- Teleconsultation is successful in detecting gross changes;
thalmology. Neurosurgical and neurological teleconsultation however, it is limited by the quality of the live-streaming
has indicated that diagnostic accuracy is highly dependent on video feed. Store-and-forward image transmission coupled
the experience of the referring physician; however, advances in with real-time videoconferencing is a suitable alternative
teleradiology assisted in improving diagnostic to overcome poor internet transmission speeds.

Downloaded from jtt.sagepub.com at CORNELL UNIV on October 13, 2016

Tan 9

Declaration of conflicting interests 11. Ausayakhun S, Skalet AH, Jirawison C, et al. Accuracy and
The authors declared no potential conflicts of interest with reliability of telemedicine for diagnosis of cytomegalovirus
respect to the research, authorship, and/or publication of this retinitis. Am J Ophthalmol 2011; 152: 1053–1058.
article. 12. Briggs R, Bailey JE, Eddy C, et al. A methodologic issue for
ophthalmic telemedicine: Image quality and its effect on
diagnostic accuracy and confidence. J Am Optom Assoc
Funding
1998; 69: 601–605.
The authors received no financial support for the research, 13. Chiang MF, Keenan JD, Starren J, et al. Accuracy and reli-
authorship, and/or publication of this article. ability of remote retinopathy of prematurity diagnosis. Arch
Ophthalmol 2006; 124: 322–327.
Guarantor and statement of contributorship 14. Chiang MF, Starren J, Du YE, et al. Remote image based
AT is the guarantor. AT, JM, LD and IT contributed to the retinopathy of prematurity diagnosis: A receiver operating
development of selection criteria. LD and IT developed the characteristic analysis of accuracy. Br J Ophthalmol 2006;
search strategy, independently reviewed results, extracted data, 90: 1292–1296.
assessed quality and drafted the manuscript. SB was a third 15. Chiang MF, Wang L, Busuioc M, et al. Telemedical retin-
reviewer in quality assessment. All authors read and provided opathy of prematurity diagnosis: Accuracy, reliability, and
feedback and approval for the final manuscript. image quality. Arch Ophthalmol 2007; 125: 1531–1538.
16. Chiang MF, Wang L, Kim D, et al. Diagnostic performance
of a telemedicine system for ophthalmology: Advantages in
Supplementary materials accuracy and speed compared to standard care. AMIA
Supplementary materials may be accessed online alongside full Annual Symposium Proceedings 2010; 111–115. Available
text articles, or obtained through correspondence with the at: http://www.ncbi.nlm.nih.gov/pubmed/21346951.
authors. 17. Conlin PR, Asefzadeh B, Pasquale LR, et al. Accuracy of a
technology-assisted eye exam in evaluation of referable dia-
References betic retinopathy and concomitant ocular diseases. Br J
Ophthalmol 2015; 99: 1622–1627.
1. Ryu S. Telemedicine: Opportunities and developments in
18. de Leon AR, Soo A, Bonzo DC, et al. Joint estimation of
member states: Report on the second global survey on ehealth
diagnostic accuracy measures for paired organs–Application
2009 (Global Observatory for eHealth Series, Volume 2).
in ophthalmology. Biom J 2009; 51: 837–850.
Healthc Inform Res 2012; 18: 153–155.
19. Whited JD. Accuracy and reliability of teleophthalmology
2. Rose HA and Grant TW. The use of a travelling ophthalmic
for diagnosing diabetic retinopathy and macular edema: A
technologist in a remote, sparsely populated region. Can J
review of the literature. Diabetes Tech Therapeut 2006; 8:
Ophthalmol 1975; 10: 201–204.
102–111.
3. Wei JC, Valentino DJ, Bell DS, et al. A web-based
20. Williams SL, Wang L, Kane SA, et al. Telemedical diagnosis
telemedicine system for diabetic retinopathy screening using
of retinopathy of prematurity: Accuracy of expert versus
digital fundus photography. Telemed J e Health 2006; 12:
non-expert graders. Br J Ophthalmol 2010; 94: 351–356.
50–57.
21. Al Sabti K, Raizada S, Wani VB, et al. Efficacy and reli-
4. Taylor CR, Merin LM, Salunga AM, et al. Improving dia-
ability of fundus digital camera as a screening tool for dia-
betic retinopathy screening ratios using telemedicine-based
betic retinopathy in Kuwait. J Diabetes Complications 2003;
digital retinal imaging technology: The Vine Hill study.
17: 229–233.
Diabetes Care 2007; 30: 574–578.
22. Nitzkin JL, Zhu N and Marier RL. Reliability of telemedi-
5. Boucher MC. Organisation of seamless and reliable manage-
cine examination. Telemed J 1997; 3: 141–157.
ment of diabetic retinopathy screening through teleophthal-
23. Aoki N, Dunn K, Fukui T, et al. Cost-effectiveness analysis
mology. Can J Diabetes 2009; 33: 284.
of telemedicine to evaluate diabetic retinopathy in a prison
6. Andonegui J, Serrano L, Eguzkiza A, et al. Diabetic retinop-
population. Diabetes Care 2004; 27: 1095–1101.
athy screening using tele-ophthalmology in a primary care
24. Chew SJ, Cheng HM, Lam DS, et al. OphthWeb—Cost-
setting. J Telemed Telecare 2010; 16: 429–432.
effective telemedicine for ophthalmology. Hong Kong Med
7. Kurji K, Kiage D, Rudnisky CJ, et al. Improving diabetic
J 1998; 4: 300–304.
retinopathy screening in Africa: Patient satisfaction with tele-
25. de la Torre-Diez I, Lopez-Coronado M, Vaca C, et al.
ophthalmology versus ophthalmologist-based screening.
Cost-utility and cost-effectiveness studies of telemedicine,
Middle East Afr J Ophthalmol 2013; 20: 56–60.
electronic, and mobile health systems in the literature:
8. Chin EK, Ventura BV, See KY, et al. Nonmydriatic fundus
A systematic review. Telemed J e Health 2015; 21: 81–85.
photography for teleophthalmology diabetic retinopathy
26. Gomez-Ulla F, Alonso F, Aibar B, et al. A comparative cost
screening in rural and urban clinics. Telemed J e Health
analysis of digital fundus imaging and direct fundus exam-
2014; 20: 102–108.
ination for assessment of diabetic retinopathy. Telemed J e
9. Zimmer-Galler IE, Kimura AE and Gupta S. Diabetic retin-
Health 2008; 14: 912–918.
opathy screening and the use of telemedicine. Curr Opin
27. Johnston K, Kennedy C, Murdoch I, et al. The cost-effec-
Ophthalmol 2015; 26: 167–172.
tiveness of technology transfer using telemedicine. Health
10. Crossland L, Askew D, Ware R, et al. Diabetic retinopathy
Pol Plann 2004; 19: 302–309.
screening and monitoring of early stage disease in Australian
28. Kumar S, Tay-Kearney ML, Chaves F, et al. Remote oph-
general practice: Tackling preventable blindness within a
thalmology services: Cost comparison of telemedicine and
chronic care model. J Diabetes Res 2016. DOI: 10.1155/
alternative service delivery options. J Telemed Telecare
2016/8405395.
2006; 12: 19–22.

Downloaded from jtt.sagepub.com at CORNELL UNIV on October 13, 2016

10 Journal of Telemedicine and Telecare 0(0)

29. Lamminen H, Lamminen J, Ruohonen K, et al. A cost study 49. Bremner F, Kennedy C, Rees A, et al. Usefulness of tele-
of teleconsultation for primary-care ophthalmology and consultations in neuro-ophthalmology. J Telemed Telecare
dermatology. J Telemed Telecare 2001; 7: 167–173. 2002; 8: 305–306.
30. Lamminen J, Forsvik H, Vopio V, et al. Teleconsultation: 50. Tanabe N, Go K, Sakurada Y, et al. A remote operating slit
Changes in technology and costs over a 12-year period. lamp microscope system. Development and its utility in
J Telemed Telecare 2011; 17: 412–416. ophthalmologic examinations. Meth Inform Med 2011; 50:
31. Pasquel FJ, Hendrick AM, Ryan M, et al. Cost-effectiveness 427–434.
of different diabetic retinopathy screening modalities. 51. Levin YS and Warshaw EM. Teledermatology: A review of
J Diabetes Sci Technol 2016; 10: 301–307. reliability and accuracy of diagnosis and management.
32. Phan AD, Koczman JJ, Yung CW, et al. Cost analysis of Dermatol Clin 2009; 27: 163–176.
teleretinal screening for diabetic retinopathy in a county hos- 52. Loane MA, Corbett R, Bloomer SE, et al. Diagnostic accur-
pital population. Diabetes Care 2014; 37: e252–e253. acy and clinical management by realtime teledermatology.
33. Rachapelle S, Legood R, Alavi Y, et al. The cost-utility of Results from the Northern Ireland arms of the UK multi-
telemedicine to screen for diabetic retinopathy in India. centre teledermatology trial. J Telemed Telecare 1998; 4:
Ophthalmology 2013; 120: 566–573. 95–100.
34. Rein DB, Wittenborn JS, Zhang X, et al. The cost-effective- 53. Baba M, Seckin D and Kapdagli S. A comparison of tele-
ness of three screening alternatives for people with diabetes dermatology using store-and-forward methodology alone,
with no or early diabetic retinopathy. Health Serv Res 2011; and in combination with web camera videoconferencing.
46: 1534–1561. J Telemed Telecare 2005; 11: 354–360.
35. Richardson DR, Fry RL and Krasnow M. Cost-savings ana- 54. Bergrath S, Reich A, Rossaint R, et al. Feasibility of pre-
lysis of telemedicine use for ophthalmic screening in a rural hospital teleconsultation in acute stroke–A pilot study in
Appalachian health clinic. W Va Med J 2013; 109: 52–55. clinical routine. PloS One 2012; 7: e36796.
36. Kumari Rani P, Raman R, Manikandan M, et al. Patient 55. Wenger TL, Gerdes J, Taub K, et al. Telemedicine for gen-
satisfaction with tele-ophthalmology versus ophthalmolo- etic and neurologic evaluation in the neonatal intensive care
gist-based screening in diabetic retinopathy. J Telemed unit. J Perinatol 2014; 34: 234–240.
Telecare 2006; 12: 159–160. 56. Russell TG, Hoffmann TC, Nelson M, et al. Internet-based
37. Paul PG, Raman R, Rani PK, et al. Patient satisfaction physical assessment of people with Parkinson disease is
levels during teleophthalmology consultation in rural accurate and reliable: A pilot study. J Rehabil Res Dev
South India. Telemed J e Health 2006; 12: 571–578. 2013; 50: 643–650.
38. Whiting PF, Rutjes AWS, Westwood ME, et al. QUADAS- 57. Haghighat S, Yunesian M, Akbari ME, et al. Telephone and
2: A revised tool for the quality assessment of diagnostic face-to-face consultation in breast cancer diagnosis: A com-
accuracy studies. Ann Intern Med 2011; 155: 529–536. parative study. Patient Educ Counsel 2007; 67: 39–43.
39. Liberati A, Altman DG, Tetzlaff J, et al. The PRISMA 58. Singh SP, Arya D and Peters T. Accuracy of telepsychiatric
statement for reporting systematic reviews and meta-ana- assessment of new routine outpatient referrals. BMC
lyses of studies that evaluate health care interventions: Psychiatr 2007; 7: 55. DOI: 10.1186/1471-244X-7-55.
Explanation and elaboration. PLoS Med 2009; 6: e1000100. 59. Muskens EMH, van Weel C, Groenleer W, et al. Psychiatric
40. Hagman J, Hyytinen P and Tuulonen A. A pilot experiment diagnosis by telephone: Is it an opportunity? Soc Psychiatr
using a network camera in ophthalmic teleconsultation. Acta Psychiatr Epidemiol 2014; 49: 1677–1689.
Ophthalmol Scand 2004; 82: 311–312. 60. Ullah R, Gilliland D and Adams D. Otolaryngology consult-
41. Tan JC, Poh EW, Srinivasan S, et al. A pilot trial of tele- ations by real-time telemedicine. Ulster Med J 2002; 71: 26–29.
ophthalmology for diagnosis of chronic blurred vision. 61. Poon WS, Leung CHS, Lam MK, et al. The comparative
J Telemed Telecare 2013; 19: 65–69. impact of video-consultation on neurosurgical health ser-
42. Cheung JC, Dick PT, Kraft SP, et al. Strabismus examin- vices. Int J Med Informat 2001; 62: 175–180.
ation by telemedicine. Ophthalmology 2000; 107: 1999–2005. 62. Leggett P, Graham L, Steele K, et al. Telerheumatology –
43. Marcus DM, Brooks SE, Ulrich LD, et al. Telemedicine Diagnostic accuracy and acceptability to patient, specialist,
diagnosis of eye disorders by direct ophthalmoscopy. A and general practitioner. Br J Gen Pract 2001; 51: 746–748.
pilot study. Ophthalmology 1998; 105: 1907–1914. 63. Stalfors J, Edstrom S, Bjork-Eriksson T, et al. Accuracy of
44. Dawson E, Kennedy C, Bentley C, et al. The role of tele- tele-oncology compared with face-to-face consultation in
medicine in the assessment of strabismus. J Telemed head and neck cancer case conferences. J Telemed Telecare
Telecare 2002; 8: 52–55. 2001; 7: 338–343.
45. Bar-Sela SM and Glovinsky Y. A feasibility study of an inter- 64. Smith AC, Kimble R, Mill J, et al. Diagnostic accuracy of
net-based telemedicine system for consultation in an ophthal- and patient satisfaction with telemedicine for the follow-up
mic emergency room. J Telemed Telecare 2007; 13: 119–124. of paediatric burns patients. J Telemed Telecare 2004; 10:
46. Threlkeld AB, Fahd T, Camp M, et al. Telemedical evalu- 193–198.
ation of ocular adnexa and anterior segment. Am J 65. Levin Y and Warshaw EM. Teledermatology: A review of
Ophthalmol 1999; 127: 464–466. reliability and accuracy of diagnosis and management.
47. Rayner S, Beaconsfield M, Kennedy C, et al. Subspecialty Dermatol Clin 2009; 27: 163–176.
adnexal ophthalmological examination using telemedicine. 66. Saunders LJ, Zhu H, Bunce C, et al. Ophthalmic statistics
J Telemed Telecare 2001; 7(Suppl 1): 29–31. note 5: Diagnostic tests—Sensitivity and specificity. Br J
48. Peter J, Piantadosi J, Piantadosi C, et al. Use of real-time Ophthalmol 2015; 99: 1168–1170.
telemedicine in the detection of diabetic macular oedema:
A pilot study. Clin Exp Ophthalmol 2006; 34: 312–316.

Downloaded from jtt.sagepub.com at CORNELL UNIV on October 13, 2016