CTS Clinical Trial Suite Studies by M&S Technologies

PUBLISHED STUDIES

The First Choice in Vision Testing Systems

M&S Technologies sets the standard in software and product development of cutting-edge technology and vision testing solutions. For over 30 years, our computerized vision testing systems have brought science, accuracy and ease of-use to the eye care industry, putting M&S in a class of its own. Since our inception, M&S has invested millions of dollars in research and development. This has resulted in comprehensive testing features and unsurpassed accuracy in each of our computerized vision testing products. Our tests, algorithms and protocols strictly adhere to the ANSI and ISO standards, are accepted for use in Phase III Trials, and are peer-reviewed, tested and published by prominent industry professionals. The science behind our technology provides our Clinical Trial clients a high level of precision and repeatability from site to site, visit to visit, for reliable results in scientific studies. We will continue to invest our time and resources into new technologies for the Clinical Trial Suite so it remains the most comprehensive product on the market. A product alone cannot ensure complete customer satisfaction, which is why M&S Technologies maintains an in-house staff of Product Engineering and Technical Support personnel. Our professional support team assists our customers and investigator sites with study protocols, as well as implementation and most importantly, the ultimate service. This team, along with our 24-hour replacement warranty, web-based assistance and strong network of distributors and industry partners, fulfills our promise to provide complete customer satisfaction.

TECHNOLOGY TRAINING, INSTALLATION AND SITE CERTIFICATION We offer training and certification services for clinical sites worldwide. The comprehensive program follows specific criteria for initial training, certification and follow-up re-certification based upon the duration of the clinical trial and specific needs of the sponsor. Training includes, but is not limited to, review and understanding of test protocol, verification of device operation, data entry, calibration, luminance settings and test distance. Certification criteria are rigid, and investigators must demonstrate a complete understanding of testing objectives and successfully complete schedules. STELLAR SUPPORT, WARRANTY AND EXTENDED SERVICE • Each trial is assigned a dedicated CTS Technical and Administrative Specialist as a point of contact to assist with study protocols, implementation and most importantly ultimate service. • CTS comes with a software warranty and technical support that lasts the duration of the trial. • Hardware and software support is available 24 hours/day, 7 days a week to accommodate international customers and our software allows us to diagnose, remotely update, or add modules. • Should any hardware issue arise, our “Spare-in-the-Air” Replacement Warranty Program assures that you will be up and running with a replacement unit within 24 hours. (International delivery times will vary based on location, customs, schedules.)

Evaluation of an Electronic Visual acuit for Use in Ophtha

Sanjeev Kasthurirangan,1 Carrie Garufis,1 Jay Rudd,2 Jonathan Solomon3 — 1Johnson & Johnson Vision RESUlTS

INTRODUCTION Ophthalmic clinical trials typically require multiple visual acuity and contrast sensitivity measurements. Contrast sensitivity curve testing used in multi-center FDA clinical trials involve more than 100 subjects to reliably detect clinically significant losses in contrast sensitivity. Computer-based vision testing could help improve workflow and precise detection of clinically significant losses with fewer subjects. Contrast vision tests such as 10% contrast acuity, 25% contrast acuity, and contrast threshold tests for fixed optotype sizes, need careful evaluation before inclusion in clinical trials.

Mesopic contrast sensitivity with the computer-based system was better than the chart system at 1 by 0.35 logunits (p<0.05 in each case, Figure 1A) and with glare 3.0 cpd by 0.16 logunits (p<0.05, spatial frequencies and glare conditions, the mean differences were within 0.13 logunits and not sta (p > 0.05 for all comparisons). The highest contrast sensitivity (i.e. lowest contrast) available in the for 1.5 cpd and 2.08 logunits for 3 cpd, whereas the computer-based system can present up to 2.4 frequencies. This test method difference of 0.40 logunits may explain the measured difference betw FIGURE 1a Computer vs. Chart: Mesopic, No Glare

FIGUR

Computer vs. Cha

The purpose of this clinical study was to evaluate visual acuity and different contrast test methods using a computer-based test system to characterize the clinical effects of refractive error and to identify a robust contrast testing method for future product evaluation.

METHODS Study Design: Prospective, monocular (right eye) measurement study in subjects who have had previous bilateral implantation of a monofocal intraocular lens conducted at two clinical sites (USA). Visual acuity and contrast sensitivity measurements were made with a computer-based system (M&S Technologies, Inc.) and a chart based system (Vector Vision). Endpoints: Repeatability at best refractive correction and the rate of change with induced refractive error (+0.25 D, +0.50 D, and +0.75 D) were evaluated for the following:

Mesopic 100% contrast visual acuity, mesopic 25% low contrast acuity and mesopic 10% low contrast acuity (Computer-based system)

Contrast sensitivity test-retest showed no statistically significant differences. The mean paired diffe 0.13 logunits for the computer based system (Figure 2A & 2B) and 0.11 log units for the chart syst the no-glare and with-glare conditions. Test-retest standard deviation was up to 0.38 logunits for b frequencies and glare conditions. Interestingly, the computer-based system showed a constant sta chart system showed increasing standard deviation with spatial frequency. FIGURE 2a Repeatability: Computer-Based, Mesopic without Glare

FIGUR

Repeatability: Computer-B

Mesopic contrast threshold at 0.30 logMAR (Computerbased system) Mesopic contrast sensitivity curve without glare and with glare (repeatability only, no defocus testing for the glare condition) for spatial frequencies 1.5, 3, 6, 12, and 18 cpd with the computer-based and chart-based systems.

Photopic uncorrected and best corrected visual acuity at 100% contrast were compared between the computer- and chart-based systems. Age (years)

Mean ± SD

Age Group, n (%)

Sex, n (%) Race, n (%)

71.3 ± 5.9

60 – 69

(41.9%)

70 – 79

(48.4%)

> = 80

(9.7%)

Male

(38.7%)

Female

(61.3%)

African American

(16.1%)

White

(83.9%)

TablE 1. Demographics for study subjects (N = 31 subjects) VaRIablE

Mean

SD.

Median

Min.

Max.

Sphere (D)

-0.42

0.56

-0.50

-2.25

0.75

Astigmatism (D)

+0.19

0.29

0.00

0.75

MRSE (D)

-0.32

0.55

-0.25

-2.25

0.75

FIGURE 3a Repeatability: Chart-Based, Mesopic without Glare

FIGUR

Repeatability: Chart-Ba

4229

ty and Contrast Sensitivity Test System almic Clinical Trials

n, Santa Ana, CA, US; 2Clarus Eye Center, Lacey, WA, US; 3Solomon Eye Associates, Bowie, MD, US

1.5 and 3.0 cpd without glare , Figure 1B). For all other atistically significantly different chart system is 1.97 logunits 40 logunits at all spatial ween the two systems.

RESUlTS Mean photopic visual acuity with the computer-based system was slightly better than the chart system: 0.04 logunits for uncorrected distance visual acuity (UCDVA) and 0.03 logunits for best corrected visual acuity (BCDVA) (Table 3). The mesopic contrast acuity tests with the computer-based system showed a mean difference within 0.01 logMAR and repeatability standard deviation within 0.10 logMAR (Table 4). The contrast threshold test with the computer-based system had a mean difference of -0.01 logunits and a repeatability standard deviation of 0.08 logunits (Table 4). All four letter-based contrast tests plotted on the same graph show a roughly linear trend of increasing visual acuity with decreasing contrast (Figure 4).

RE 1b

art: Mesopic, with Glare

erences were within tem (Figure 3A & 3B) across both systems across spatial andard deviation, whereas the

RE 2b

Paired Difference VaRIablE

Mean

Median

UCDVA

-0.04

0.11

-0.02

-0.09

95% CI -0.00

BCDVA

-0.03

0.06

-0.04

-0.05

-0.01

FIGURE 4

TablE 3. Photopic uncorrected and best corrected visual acuity for the computer- and chart-based systems. Paired difference letter Test

Mean

Median

100% (logMAR)

-0.01

0.06

0.00

-0.12

95% CI 0.11

25% (logMAR)

-0.01

0.04

0.00

-0.09

0.07

10% (logMAR)

0.00

0.10

0.02

-0.20

0.21

Sensitivity for 20/40 letter (logunits)

-0.01

0.08

0.00

-0.16

0.15

ased, Mesopic with Glare

0.25

0.50

0.75

1.00

TablE 4. Test-retest repeatability of the mesopic acuity and contrast threshold testing with computer-based testing.

Contrast sensitivity curves did not show a noticeable decrease with defocus (Figure 5) and only 0.0 D vs. 0.75 D was statistically significantly different (p < 0.05) at 6 cpd and 12 cpd (approximately 0.20 logunits). All four letter-based tests showed a statistically significant decrease in performance at 0.50 D and 0.75 D induced defocus, compared to 0.0 D (Table 5). FIGURE 5

0.0 D vs. 0.25 D 0.0 D vs. 0.50 D

Based, Mesopic with Glare

RE 3b

0.0 D vs. 0.75 D

Paired Mean (95% CI)

100% (logMAR)

-0.05 (-0.08, -0.02)

-0.10 (-0.14, -0.06)

-0.17 (-0.22, -0.12)

25% (logMAR)

-0.03 (-0.05, -0.00)

-0.06 (-0.09, -0.03)

-0.13 (-0.17, -0.09)

10% (logMAR)

-0.01 (-0.04, +0.01)

-0.04 (-0.07, -0.02)

-0.10 (-0.14, -0.06)

Sensitivity for 20/40 letter (logunits)

-0.04 (-0.07, +0.00)

-0.10 (-0.15, -0.04)

-0.17 (-0.24, -0.10)

letter Test

TablE 5. Effect of defocus on the letter-based contrast acuity and contrast threshold tests with the computer-based system

CONClUSIONS Contrast sensitivity testing with the two systems was largely equivalent with similar repeatability (0.38 logunits). The computer-based system measured slightly better contrast sensitivity at low spatial frequencies, likely due to the limited range of contrast available in the chart-based system. Visual acuity testing was slightly better (2 letters) with the computer-based system compared to the chart-based system. Mesopic visual acuity (100% contrast, 25% contrast and 10% contrast) and contrast threshold for a fixed letter size measured with the computer-based system showed very good repeatability of approximately 1 letter for the letter-based tests and 0.01 logunit for the contrast threshold test. The letter-based tests showed a clear reduction in performance with induced optical defocus compared to the spatial contrast sensitivity test. Financial Disclosure: Sanjeev Kasthurirangan, Carrie Garufis, Abbott Medical Optics, E, Employment; Jay Rudd, Jonathan Solomon, Abbott Medical Optics, F, Financial Support

Article

Comparison of Backlit and Novel Automated ETDRS Visual Acuity Charts Paul A. Harris, OD, Southern College of Optometry, Memphis, Tennessee Laurel E. Roberts, Southern College of Optometry, Memphis, Tennessee Rachel Grant, OD, Southern College of Optometry, Memphis, Tennessee

ABSTRACT Background: This study was conducted to compare two different methods and presentation systems of testing visual acuity to determine whether they are equivalent. Methods: We compared the results of taking visual acuity (VA) measures with the standard backlit Early Treatment of Diabetic Retinopathy Study (backlit ETDRS) and Automated ETDRS (A-ETDRS) VA charts (M&S Technologies, Inc., Niles, IL) on 111 healthy subjects with corrected visual acuity of 20/20 or better. Testing was done under four conditions— with spectacles, uncorrected, with +1.50 blur over spectacles, and with +3.00 blur over spectacles—to assess correlation of primary outcomes between charts across a wide range of acuity measures. Visual acuity measures were recorded in letter count, logMAR, and standard Snellen measures. Results: Correlations between the backlit ETDRS and the A-ETDRS chart types were 0.93 (uncorrected), 0.60 (with spectacles), 0.76 (+1.50 blur over spectacles), and 0.50 (+3.00 blur over spectacles), with all correlations statistically significant at p< 0.001. Conclusion: This study shows that traditional backlit ETDRS and A-ETDRS charts are functionally equivalent to each other under a variety of testing conditions, mimicking both clinical and research applications. Additional benefits of the automated system over the backlit charts include: the ability to calibrate the system precisely, faster testing and scoring times combined, and less chance for error to enter into the conversion of the raw data into logMAR, letter, or Snellen scores. For all of these reasons, Automated-ETDRS testing is preferred. Keywords: automated testing, backlit screen, digital screen display, Early Treatment of Diabetic Retinopathy Study, electronic vision chart, ETDRS, logMAR, optotype, Snellen, Visual acuity Background Visual acuity is one of the most important tools in determining visual function and has been established as the “gold standard” in prospective clinical trials, especially regarding eye disease and treatment.1,2 The assessment of visual acuity with optotype charts is the most standardized test of visual function. These high-contrast printed charts include black optotypes, letters, or symbols on a white background and are externally illuminated. Optometry & Visual Performance

The charts allow a diverse patient population to be tested. Theoretically, visual acuity testing should give a precise, reproducible, and reliable result that represents the state of macular function. The testing further implies that any acuity changes are related to disease or treatment. However, visual acuity can be influenced and altered by external factors, including but not limited to exam room lighting, contrast, design of the chart, subject motivation, and scoring technique.3,4 87

Volume 6 | Issue 2 | 2018, April

Figure 2. The A-ETDRS opening screen. Figure 1. The backlit ETDRS chart with the A-ETDRS computer screen behind. The backlit chart starts at 20/200, while the A-ETDRS chart starts at 20/100. The sizes of the letters and spacing from 20/100 to 20/10 are exactly the same.

has been recognized to be highly reliable for vision testing1 and has been considered one of the standard tools for measuring acuity in prospective clinical research for more than 30 years.1,8,9 Each Sloan letter on the ETDRS chart (ten in total) has approximately equal legibility or difficulty, and each line has the same overall difficulty.10 Each row contains five letters, with the spacing between each letter being equal to the width of one letter and the space between lines being equal in height to the letters of the next lower line.10 The letter size from row to row changes in equal logarithmic intervals.4,6 The chart itself is non-reflective, white, highimpact polystyrene with the black letters creating a contrast level of approximately 90%. The accompanying light box produces a standardized illumination of 120 cd/m2, which conforms to the ANSI specifications.3,11,12 All other light sources in the room should be turned off to reduce any potential glare sources. Although considered the standard for clinical research, ETDRS and other logMAR charts are not widely used in clinical practice.16 As evidence of this, at Southern College of Optometry, the 90+ clinical testing lanes and the 50+ student practice lanes are equipped with computerbased chart systems, while there are only two ETDRS charts in place for compliance with specific FDA clinical protocols. This ratio is similar in most North American optometry schools. It is thought that the test format, including the length of test administration, unfamiliar scoring,

The Snellen eye chart is the most widely used method of visual acuity measurement in clinical practice, in part due to its ease of use and availability. Snellen’s original chart had a single large letter at the top, and with each successive row, the letters became more numerous and progressively smaller.5 The letters are not equal in their legibility; there is also unequal letter and line spacing.4,6,7 In addition, since its original conception, many variations in size, sequence, chart layout, and design of the optotypes were made; subsequently, there is no broadly accepted “standard” Snellen chart.5 Most commonly, visual acuity measurements are determined under high-contrast conditions, as previously mentioned. Over time, the required contrast level for the chart can be impacted by stains and fading, which may alter reflectivity. Room position and room illumination may also introduce variability.3 ETDRS Standards The Early Treatment of Diabetic Retinopathy Study (ETDRS) chart is based on the previously designed Bailey-Lovie logMAR chart to establish a standardized measurement of visual acuity. The inclusion of administration and scoring protocols serves to improve the precision of visual acuity measurement in the range of poorer visual acuities.5 The ETDRS chart Optometry & Visual Performance

Volume 6 | Issue 2 | 2018, April

user of A-ETDRS. In many clinical studies, subjects spend a great deal of time, under many different conditions, reading the backlit ETDRS charts from top to bottom, over and over. Since there are only three different charts, which must be manually changed, there is a chance that subjects could begin to know some of the letter sequences in those charts. This could lead to an overestimation of their visual acuity, which is not related to the specific testing or experimental condition. Randomization of each “chart” in the A-ETDRS configuration would eliminate memorization from prior exposures, thus increasing the validity of the measure. An additional benefit that would result from the aim being met is increased reliability in the calculation of the letter count, from which the logMAR and/or visual acuity measure is derived. The standard backlit ETDRS charts must be manually scored. This process is highly repetitive and adds time to the process of getting the letter count. The A-ETDRS system immediately provides the user with all of the scores needed, without the need for a separate recording system or the counting or calculation of any of the scores. This should save time and guarantee that the measures reported are indeed the measures obtained. This study also addresses some of the issues raised in discussion of some early attempts to computerize the ETDRS testing process.17 Issues that have been raised include pixelation of the letters on the computer screen, where individual pixels are visible to the naked eye, and anti-aliasing.a These specifically affect the ability cleanly to present letters smaller than 20/20 on older computer monitors. Smaller screens limit the size of the largest letter that can be shown to a subject. Larger computer monitors with smaller pixels, packed much more closely together on the screen in both the vertical and horizontal dimensions (dot pitch), combine to allow for much larger letters than before, while being able to present letters down

and patients memorizing letter sequences, as well as the inherent difficulty in discussing logMAR acuity with patients, contribute to the practical limitations.9,14,15 The standard ETDRS chart is a large, floor-mounted, backlit device that takes up a significant amount of space and requires manual changes amongst the three provided plastic sheets (Figure 1). The Automated ETDRS chart (A-ETDRS; M&S Technologies, Inc., Niles, IL),5 which is part of the Clinical Trial Suite offered by M&S, has the potential to make the test more portable, more difficult to memorize, easier to score, and it may speed up testing time (Figure 2). Computerized Testing Technological advancements have improved the incorporation of technology, such as computer-based displays, in all facets of health care, including electronic vision testing. Various forms of electronic and automated displays exist on the market and continue to gain popularity with patients and practitioners alike. The inevitable trend towards using more computer-based displays for the measurement of visual acuity has specific research advantages that come from computer control of visual displays for measuring visual acuity.5 Computer displays can provide selectable options, such as optotypes, spacing and crowding arrangements, contrast, and color. Research has shown that another advantage of a computerized acuity system is the ability to increase the test-retest repeatability through repetition and averaging of measurements.8,16 Furthermore, a computerbased acuity chart allows random order presentation and automated processing.8 Purpose This study was conducted to compare two different methods and presentation systems of testing visual acuity to determine whether they are equivalent. Should that aim be met, then additional benefits would accrue to the Optometry & Visual Performance

Volume 6 | Issue 2 | 2018, April

to wear their spectacle correction. This made the randomization of the uncorrected conditions on both the backlit and the A-ETDRS charts less time-consuming than if we had our subjects remove their contact lenses and then readapt to them for the next condition. All testing was done binocularly in a room where the only illumination came from the two charts. Both displays were on all the time. Whenever a backlit chart was to be used for testing, one of the three charts was selected based on a randomization table. It should be noted that for each subject, four different measures were made on the backlit chart, but there are only three different charts. Each chart had an equal chance of being used at any time. Although it was time-consuming to change the charts manually, it was done to minimize any chance of a subject memorizing the charts, as well as to simulate formal research protocols. In the cases where the randomization table indicated that the same backlit chart was to be used again, the researcher went through the chart changing routine and simply put the prior chart back in position. This was to encourage the thought in the subjects that the charts were different each time. When the backlit ETDRS charts were used, printed score sheets were available for each of the three charts. The appropriate score sheet was selected, and the subject was asked to read each letter distinctly from the top of the chart. Every letter was marked on the recording form as either correct or incorrect. The total number of letters correct was recorded, and a conversion chart was used to derive the logMAR and Snellen score for that condition. The protocol used to measure visual acuity with these charts followed standard ETDRS research protocol, where for every measurement, letters were read at a speed of one per second, beginning at the top left of the chart and proceeding line by line, left to right, with an opportunity to correct an error only before the next letter was attempted. The

to 20/8 Snellen visual acuity levels. Aliasingb occurs in computer graphics when a screen cannot render as smooth a curve as intended and it appears on the screen as jagged. When viewed extremely closely, what is seen are small steps rather than smooth curves. Anti-aliasing software has been used to attempt to minimize these effects. The typical panel displays used now in these systems do not need anti-aliasing software because of the smaller dots, which are packed much more closely together. The M&S Technologies Smart System II used in this study has a 22-inch digital flat panel screen with a resolution of 1680 x 1050. Lastly, the new control systems, which use a separate tablet with built-in scoring, should allow for faster data collection times. Methods One hundred and eleven (N=111) secondand third-year students from Southern College of Optometry (SCO), with corrected visual acuity of 20/20 or better binocularly, had their visual acuity taken eight separate times, with each of the conditions being randomized. Visual acuity was measured four different times on each of the two different types of charts. The four conditions for each chart included: with spectacles, without spectacles, with +1.50 spheres over spectacles, and with +3.00 spheres over spectacles. For each subject, randomization was across all 8 conditions, and all testing was done on the same day at a single sitting. The randomization table was generated by research randomizer.c All testing was done at 4 meters. The standard ETDRS protocol has the subject wearing a +0.25 DS lens to compensate for this distance. The +0.25 lens was not used in any of our 8 test conditions. We created the two pairs of spectacles for testing, one pair of +1.50 spheres and the other of +3.00 spheres, in frames large enough to allow them to be worn over the subjects’ own spectacles without difficulty. Those subjects who wore contact lenses were asked not to wear their contact lenses on the day of testing, but instead Optometry & Visual Performance

Volume 6 | Issue 2 | 2018, April

of visual acuity measures, range-finding and thresholding. During the range-finding phase, the subject finds the smallest line of letters that they believe they can read completely correctly and proceeds to read them aloud. The operator presses the button on the control software that corresponds to that line of letters. Figure 3 shows the screen from the tablet. The letters shown to the subject on the display screen are also displayed to the operator on the tablet. Figure 4 shows the screen that the subject would see after the operator presses the number 20 on the tablet’s screen, indicating that the subject got all the letters correct on the 20/20 line but made a mistake on the 20/16 line. At this point, the program switches to the thresholding phase. Now the tablet asks the operator to specify how many letters the subject correctly reports on each subsequently smaller line and finally stops either when the subject is unable to get any letters correct on the next smaller line or no more smaller lines exist to be shown. After the A-ETDRS testing is complete, the system saves the results and reports the test results to the main screen, which can be printed. A sample set of data includes the eye (Right, Left, or Both), test distance (4 meters in this protocol), spectacles (on or off), light level (photopic vs. mesopic), and the visual acuity results in three forms (letter score, visual acuity, and logMAR). The letter score is the total number of letters that the subject said correctly, which results in a standard Snellen equivalent and a logMAR score. For example, a letter score of 87 converts to a Snellen VA of 20/20 and a logMAR score of -0.04. Test times were measured for all trials using a stopwatch function on the investigator’s smart phone. Timing started as soon as the A-ETDRS chart was presented and terminated when the program displayed the scores to the computer screen for recording. For the backlit chart, timing was started when the subject said the first letter and finished when they were no

Figure 3. Android tablet with control program showing the lines from 20/50 to 20/16 in the red background area. Pressing the “Up” or the “Down” buttons on the screen changes the display to different parts of the chart.

Figure 4. The A-ETDRS opening screen.

procedures for encouraging letter recognition and the stopping rule are standardized. Training materials for Ophthalmic Clinical Trial Training and Certification are available from the Emmes Corporation.d The researchers did not anticipate any of the subjects triggering the standard protocol for the conditions when visual acuity was worse than 20/200. When this was encountered, the A-ETDRS program returned a standard value of 20/250 and a letter count of 34, and similar results were recorded with the backlit ETDRS chart. The portion of the standardized testing protocol used to change the working distance to one meter was not done. The A-ETDRS uses an Android tablet with the M&S Technologies, Inc. custom control program, which synchronizes with the main Smart System through a Bluetooth connection. Each time the protocol is run, the chart provides a random sample of the 10 ETDRS letters, making memorization of the chart impossible. There are two phases of determining the endpoint Optometry & Visual Performance

Volume 6 | Issue 2 | 2018, April

Table 1 .logMAR scores by condition and chart type. Note: Significance test based on paired t-tests for Automated vs. Backlit on the common

sample within each condition.

mean

95% Confidence Interval (CI)

Difference

Sig

-.16 -.16

.12 .10

.01 .01

-.18, -.14 -.18, -.14

-.001

p= .89

.20 .23

.47 .46

.05 .05

.10, .30 .13, .32

-.028

p= .13

.29 .29

.22 .20

.02 .02

.24, .33 .25, .33

-.005

p= .70

.71 .72

.21 .19

.02 .02

.67, .75 .69, .76

-.010

p= .62

With spectacles (n= 110) Automated Backlit Uncorrected (n= 87) Automated Backlit +1.5 Blur over spectacles (n= 111) Automated Backlit +3.0 Blur over spectacles (n= 102) Automated Backlit

Note: Significance test based on paired t-tests for Automated vs. Backlit on the common sample within each condition.

longer able to get any letters correct. Time to change the plastic test cards in the backlit box was not included in the timing, nor was the time to count and score the subject’s results. Weber Contrast was calculated for each of the targets using measurements collected with the Konica-Minolta LS-110 luminance meter, which measures the amount of reflected or emitted light from an area of 0.33 of a degree. On the backlit box, the white area was 181 cd/m2, while the black was 1.35 cd/m2. This produced a Weber Contrast of 99.25%. On the M&S Technologies A-ETDRS screen, the white was 120 cd/m2, and the black was 0.72 cd/m2, which produced a Weber Contrast of 99.4%. The backlit box was not adjustable in luminance. The M&S system was at its calibrated light value of 120 cd/m2. Both are compliant with ANSI Z80.21-2010 (R2015) and ISO 8597:1994(E) standards.11,12 All subjects gave informed consent after a verbal and written explanation of the experiment, which was approved by the Southern College of Optometry Institutional Review Board in accordance with the Declaration of Helsinki.

There is also a calculated logMAR visual acuity level assigned to this value. LogMAR scores for both Backlit ETDRS and A-ETDRS chart types were compared under each of four testing conditions (uncorrected, with spectacles, +1.50 blur over spectacles, and +3.00 blur over spectacles) using paired t-tests. Findings were also confirmed using nonparametric alternatives (Wilcoxon sign-rank tests), as well as a repeated measures analysis of variance (ANOVA), which yielded the same conclusions. Thus, for ease of presentation, the mean comparisons of each chart type at each testing condition are shown here. BlandAltman plots were used to illustrate agreement between A-ETDRS and Backlit ETDRS chart types. Correlations between A-ETDRS and Backlit ETDRS chart types were examined using Pearson’s r. Results Analyses were conducted using Stata/SE software, version 13.e Table 1 shows means, standard deviations (SDs), standard errors (SEs), and 95% confidence intervals comparing logMAR scores on A-ETDRS and Backlit ETDRS charts for each condition. There were no significant differences between the chart types at any condition. Figure 5 shows mean logMAR values for Automated and Backlit charts graphically, with error bars. The widest standard deviations exist for the uncorrected measures,

Statistical Analysis Measurement of visual acuity with the ETDRS charts yields two different scores. The first is a Letter Score, which is a count of the total number of letters correct from the largest letter until the subject stops getting letters correct. Optometry & Visual Performance

Volume 6 | Issue 2 | 2018, April

Figure 5. Mean logMAR values for Automated and Backlit EDTRS charts Figure 7. Bland-Altman plot for the “uncorrected” condition.

Figure 6. Bland-Altman plot for the “with spectacles” condition.

and here the visual acuities measured trended to be worse with the backlit ETDRS. However, the differences were neither statistically nor clinically significant. Figures 6 to 9 show Bland-Altman plots for each study condition, which plot the difference of the paired chart vs. their average. There were only very few cases where individual values fell outside the range of agreement. Correlations between Automated and Backlit chart types were 0.93 (uncorrected), 0.60 (with spectacles), 0.76 (+1.50 blur over spectacles), and 0.50 (+3.00 blur over spectacles), with all correlations statistically significant at p< 0.001. It was decided to eliminate any data points in the Bland-Altman plots when one or both logMAR values was greater than 1.0, because we had not anticipated that we would have significant numbers of these measures. Indeed, only 9 subjects in the +3.00 blur and 24 subjects Optometry & Visual Performance

Figure 8. Bland-Altman plot for the “+1.50 blur over spectacles” condition.

Figure 9. Bland-Altman plot for the “+3.00 blur over spectacles” condition.

Volume 6 | Issue 2 | 2018, April

in the uncorrected conditions had one or more logMAR measures greater than 1.0. This accounted for the different number of subjects in each direct comparison. Figure 6 shows the Bland-Altman plot for the “with spectacles” condition. Three of the 110 subjects (2.73%) fell outside the 95% limits of agreement. Figure 7 shows the Bland-Altman plot for the “uncorrected” condition. Three of the 87 subjects (3.45%) fell outside the 95% limits of agreement. The number of subjectsfor this condition was the smallest, because 24 of the subjects had either the A-ETDRS or the backlit ETDRS visual acuity worse that 1.0 logMAR and therefore were not included in the analysis. Figure 8 shows the Bland-Altman plot for the “+1.50 blur over spectacles” condition. Four of the 111 subjects (3.6%) fell outside the 95% limits of agreement. Figure 9 shows the Bland-Altman plot for the “+3.00 blur over spectacles” condition. Six of the 102 subjects (5.88%) fell outside the 95% limits of agreement. The N for this condition was reduced to 102 as nine subjects had either the A-ETDRS or the backlit ETDRS visual acuity worse than 1.0 logMAR. The authors independently took the raw scores from the A-ETDRS testing and verified that the computer algorithm indeed yielded the correct Letter and logMAR Scores for each measurement for the first 20 subjects. Timing data for all 444 trials on the A-ETDRS chart across all conditions averaged 21.24 seconds (SD 18.6 seconds), with a range from 3 to 180 seconds. The average time varied across the conditions, with the corrected measures averaging the fastest (24.88 seconds SD 9.9 seconds) The “+3.00 blur over spectacles” condition averaged 39.1 seconds (SD 25.7). Timing data for all 444 trials on the backlit ETDRS averaged 18.7 seconds (SD 11.9 seconds), with a range from 2 to 117 seconds. This was only the time to perform the test and did not include the time to score the results or to change the chart prior to starting each trial (Table 2). Optometry & Visual Performance

Table 2. Timing data for each condition All

With Spectacles

Uncorrected +1.50 Blur over Spectacles

+3.00 Blur over Spectacles

Number

N=444 N=111

N=111

Average

31.2

24.8

26.8

39.1

16.6

18.6

9.9

15.9

15.7

9.0

Low

High

180

118

180

Discussion The four different testing conditions for each chart were chosen to represent both real-world conditions (uncorrected and with spectacles) as well as some research-based conditions (simulated 1.50 D and 3.00 D of uncorrected myopia). We did not anticipate any of our subjects triggering the standard protocol for the conditions when visual acuity was worse than 20/200. When this was encountered, the A-ETDRS program returned a standard value of 20/250 and a letter count of 34. As we reviewed the results, nine subjects in the “+3.00 blur over spectacles” and 24 subjects in the “uncorrected” group had visual acuities worse than 20/200, or logMAR greater than 1.0. In future studies, we will repeat measures in those conditions following the standard protocol, which is to reduce the working distance to one meter and repeat the testing. In that setup, the 20/200-sized letters at four meters are equivalent to 20/800 at one meter. This was not done. Measures where the visual acuity was greater than logMAR 1.0 were removed for analysis. This did not affect any measures in either the “spectacles” or the “+1.50 blur over spectacles” groups. Calibration In formal research settings, having testing instruments able to be calibrated is a must. Many individual systems are used for periods of years. Though the backlit ETDRS systems have been the gold standard for many years, there is no easy way to calibrate them, short of changing bulbs until the measured luminance levels are within standards. Luminance of the bulbs in the units varies, and the plastic sheets 94

Volume 6 | Issue 2 | 2018, April

are prone to yellowing over time, which reduces contrast. The A-ETDRS systems ship with a luminance measuring system, the use of which is integrated into the system. Periodically, as prompted by the software, the measuring system is suspended directly in front of the screen, and the system varies the illumination to reach the exact specified amount of 120 cd/m2. Both the bright and dark luminance measures are taken and adjusted to ensure proper calibration, within very tight tolerances. This is a major advantage of the A-ETDRS system over the backlit ETDRS targets.

not take the time to perform the letter count and corresponding conversion to logMAR while the subject was present. We neglected to record the time it took for scoring the data as we did it during downtime between subject sittings, and we did not add in the time to change the backlit ETDRS charts, which would affect this comparison even more. In a future study, the timing of the scoring and conversion to logMAR values as well as changing the charts should be done for each data point to be comparing like entities. Conclusions The findings of this study show that the traditional and automated ETDRS charts are functionally equivalent to each other under a variety of testing conditions, mimicking both clinical and research applications. Use of the A-ETDRS system by M&S Technologies is faster and less prone to recording errors or calculation errors, can be calibrated regularly, and is very easy. These findings set the stage for the adoption of the A-ETDRS chart by M&S Technologies in any clinical research study or clinical trial setting that calls for ETDRS testing.

Randomized letters Two major advantages accrue to those using A-ETDRS over standard backlit charts. The scoring step is eliminated, from the hard copy made during the testing to however the results are being recorded. The first benefit is that errors are eliminated in the calculation of the score. Others have reported that in their experience, manual recording systems are prone to error.18 A second benefit is that about 30 seconds are saved, which is the typical time it takes to do the actual scoring.19 Time Saving from Automation The new A-ETDRS is faster to use than standard ETDRS testing when scoring time on the standard ETDRS charts was added to the time needed to perform the test. The A-ETDRS system automatically scores the trial and shows all measures (letter score, Snellen visual acuity, and logMAR) immediately after the testing. All 444 A-ETDRS tests averaged 21.2 seconds (SD 18.6 seconds). Laidlaw et al. found an average time for standard EDTRS measures of 56 seconds with adult populations, and on average their computerized system was 7-10 seconds slower.18 Timing for each of the backlit ETDRS trials was recorded in our study, but these did not include the time spent scoring each of the results. We only recorded the raw data for each trial on the matching score sheet for that trial, but we did Optometry & Visual Performance

Acknowledgments Thank you to: Jan Gryczynski, PhD, Founding Partner COG Analytics, Potomac, MD, for his assistance with the statistics in this paper. References

Beck R, Moke P, Turpin A, et al. A computerized method of visual acuity testing: Adaptation of the early treatment of diabetic retinopathy study testing protocol. Am J Ophthalmol 2003;135:194-205. http://bit.ly/2qH31bW

Ferris FL, Bailey I. Standardizing the measurement of visual acuity for clinical research studies: Guidelines from the Eye Care Technology Forum. Ophthalmology 1996;103:181-2. http://bit.ly/2HavdyP

Ehrmann K, Fedtke C, Radić A. Assessment of computer generated vision charts. Cont Lens Anterior Eye 2009;32:13340. http://bit.ly/2qJJt6C

Williams MA, Moutray TN, Jackson AJ. Uniformity of visual acuity measures in published studies. Invest Ophthalmol Vis Sci 2008;49:4321-7. http://bit.ly/2qJCdb7 Volume 6 | Issue 2 | 2018, April

Bailey IL, Lovie-Kitchin JE. Visual acuity testing. From the laboratory to the clinic. Vision Res 2013;90:2-9. http://bit. ly/2HdOhMA

Rosser D, Laidlaw D, Murdoch IE. The development of a “reduced logMAR” visual acuity chart for use in routine clinical practice. Br J Ophthalmol 2001;85:432-6. http://bit. ly/2qHCqLO

16. Shah N, Laidlaw DAH, Shah SP, et al. Computerized repeating and averaging improve the test-retest variability of ETDRS visual acuity measurements: Implications for sensitivity and specificity. Invest Ophthalmol Vis Sci 2011;52:9397-402. http://bit.ly/2HehuHC 17. McClenaghan N, Kimura A, Stark LR. An evaluation of the M&S Technologies Smart System II for visual acuity measurement in Young Visually-Normal Adults. OVS 2007;84:3:218-23. http:// bit.ly/2HehuHC

Ferris FL 3rd, Sperduto RD. Standardized illumination for visual acuity testing in clinical research. Am J Ophthalmol 1982;94:97-8. https://go.nature.com/2qJD37J

Rosser D, Murdoch IE, Fitzke FW, Laidlaw DAH. Improving on ETDRS acuities: Design and results for a computerised thresholding device. Eye 2003;17:701-6. http://bit.ly/2H9qrSu

Ferris FL, Bailey I. Standardizing the measurement of visual acuity for clinical research studies: Guidelines from the Eye Care Technology Forum. Ophthalmology 1996;103(1):181-2. http://bit.ly/2qISQDM

18. Laidlaw DAH, Tailor V, Shah N, Atamian S, Harcourt C. Validation of a computerised logMAR visual acuity measurement system (COMPlog): comparison with ETDRS and the electronic ETDRS testing algorithm in adults and amblyopic children. J Ophthalmol 2008;92:241-4. http://bit.ly/2qHDVJW 19. Bokinni Y, Shah N, Maquire O, Laidlaw DAH. Performance of a computerised visual acuity measurement device in subjects with age-related macular degeneration: Comparison with gold standard ETDRS chart measurements. Eye 2015;29:1085-91. http://bit.ly/2qIJic1

10. Ferris FL 3rd, Kassoff A, Bresnick GH, et al. New visual acuity charts for clinical research. Am J Ophthalmol 1982;94:91-6. http://bit.ly/2H9RQn3

a. https://goo.gl/U3zCun b. https://goo.gl/tbzBN5 c. https://www.randomizer.org d. https://goo.gl/qaYrTu e. https://www.stata.com

11. American National Standards Institute, Inc. American National Standards for Ophthalmics – Instruments – General-Purpose Clinical Visual Acuity Charts, ANSA Z80.21-2010 (R2015), The Vision Council, Alexandria, VA. http://bit.ly/2qEMfds 12. Technical Committee ISO/TC 172, Optics and optical instruments – Visual acuity testing – method of correlating optotypes, International Standard ISO 8597, International Organization of Standardization 1984, Genève, Switzerland. http://bit.ly/2qIwfXO

Correspondence regarding this article should be emailed to Paul A. Harris, OD, MS, at pharris@sco.edu. All statements are the author’s personal opinions and may not reflect the opinions of the representative organizations, ACBO or OEPF, Optometry & Visual Performance, or any institution or organization with which the author may be affiliated. Permission to use reprints of this article must be obtained from the editor. Copyright 2018 Optometric Extension Program Foundation. Online access is available at www.acbo.org.au, www.oepf.org, and www.ovpjournal.org.

13. Kaiser PK. Prospective evaluation of visual acuity assessment: A comparison of Snellen versus ETDRS charts in clinical practice (An AOS Thesis). Trans Am Ophthalmol Soc 2009;107:311-24. http://bit.ly/2qMtYLw 14. Lovie-Kitchin JE. Validity and reliability of visual acuity measurements. Ophthalmic Physiol Opt 1998;8:363-70. http://bit.ly/2HbPFjb

Harris PA, Roberts LE, Grant R. Comparison of backlit and novel automated etdrs visual acuity charts. Optom Vis Perf 2018;6(2):87-96.

15. Kuo HK, Kuo MT, Tiong IS, et al. Visual acuity as measured with Landolt C chart and Early Treatment of Diabetic Retinopathy Study (ETDRS) chart. Graefes Arch Clin Exp Ophthalmol 2011;249:601-5. http://bit.ly/2qKEjY7

Optometry & Visual Performance

Volume 6 | Issue 2 | 2018, April

A Computerized Method of Visual Acuity Testing: Adaptation of the Early Treatment of Diabetic Retinopathy Study Testing Protocol ROY W. BECK, MD, PHD, PAMELA S. MOKE, MSPH, ANDREW H. TURPIN, PHD, FREDERICK L. FERRIS III, MD, JOHN PAUL SANGIOVANNI, SCD, CHRIS A. JOHNSON, PHD, EILEEN E. BIRCH, PHD, DANIELLE L. CHANDLER, MSPH, TERRY A. COX, MD, PHD, R. CLIFFORD BLAIR, PHD, AND RAYMOND T. KRAKER, MSPH

To develop a computerized method of visual acuity testing for clinical research as an alternative to the standard Early Treatment for Diabetic Retinopathy Study (ETDRS) testing protocol, and to evaluate its test-retest reliability and concordance with standard ETDRS testing. ● DESIGN: Test-retest reliability study. ● METHODS: Multicenter setting of a study population of 265 patients at three clinical sites. Visual acuity was measured with both the electronic visual acuity testing algorithm (E-ETDRS) and standard ETDRS protocol (S-ETDRS) twice on one eye of each patient. E-ETDRS testing was conducted using the electronic visual acuity tester (EVA), which utilizes a programmed Palm (Palm, Inc, Santa Clara, California, USA) hand-held device communicating with a personal computer and 17-inch monitor at a test distance of 3 meters. ● RESULTS: For the E-ETDRS protocol, test-retest reliability was high (r 0.99; with 89% and 98% of retests within 0.1 logMAR and 0.2 logMAR of initial tests, respectively) and comparable with that of S-ETDRS testing (r 0.99; with 87% and 98% of retests within 0.1 logMAR and 0.2 logMAR of initial test, respectively). The E-ETDRS and S-ETDRS scores were highly correlated (r 0.96 for initial tests and r 0.97 for repeat tests). Based on estimates of 95% confidence ● PURPOSE:

Accepted for publication Aug 21, 2002. InternetAdvance publication at ajo.com Nov 14, 2002. From the Jaeb Center for Health Research (R.W.B., P.S.M., R.C.B., R.T.K.), Tampa, Florida; Curtin University (A.H.T.), Bentley, Western Australia; the National Eye Institute, Division of Epidemiology and Clinical Research, the National Institutes of Health (F.L.F., J.P.S.G., T.A.C.), Bethesda, Maryland; Discoveries in Sight Research Labs, Devers Eye Institute (C.A.J.), Portland, Oregon; and the Retina Foundation of the Southwest (E.E.B.), Dallas, Texas. This study was supported by the National Eye Institute grant #EY13095. Inquiries to Roy W. Beck, MD, PhD, Jaeb Center for Health Research, 3010 E. 138th Ave., Suite 9, Tampa, FL 33613; fax: (813) 975-8761; e-mail: rbeck@jaeb.org

194

2003 BY

intervals, a change in visual acuity of 0.2 logMAR (10 letters) from a baseline level is unlikely to be related to measurement variability using either the E-ETDRS or the S-ETDRS visual acuity testing protocol. ● CONCLUSIONS: The E-ETDRS protocol has high testretest reliability and good concordance with S-ETDRS testing. The computerized method has advantages over the S-ETDRS testing in electronically capturing the data for each tested letter, requiring only a single distance for testing from 20/12 to 20/800, potentially reducing testing time, and potentially decreasing technician-related bias. (Am J Ophthalmol 2003;135:194 –205. © 2003 by Elsevier Science Inc. All rights reserved.)

ISUAL ACUITY IS A COMMON PRIMARY OUTCOME

measure in clinical research of eye diseases. In multicenter clinical trials, considerable effort is placed on the standardization of acuity testing across sites.1–3 To more easily standardize measurement of visual acuity in clinical trials and to provide a method to directly capture acuity data electronically, we have developed a computerized vision tester called the electronic visual acuity tester (EVA). Previously we reported on the adaptation of the Amblyopia Treatment Study (ATS) visual acuity testing protocol for the EVA.4 This testing protocol was developed to facilitate the standardization of visual acuity testing in clinical trials of pediatric eye disease involving children from 3 to 6 years old.5 We now report on the development and evaluation of a second testing protocol for the EVA, one for the testing of older children and adults. This protocol is based in part on the testing protocol developed for Early Treatment for Diabetic Retinopathy Study (ETDRS),1,6 a protocol which has been the standard for visual acuity testing in most clinical research for more than 15 years. We conducted a study to assess test-retest reliability for both the electronic

ELSEVIER SCIENCE INC. ALL

RIGHTS RESERVED.

0002-9394/03/$30.00 PII S0002-9394(02)01825-1

(E-ETDRS) and standard ETDRS (S-ETDRS) protocols and to compare visual acuity scores obtained with the two protocols.

METHODS The EVA utilizes a programmed Palm hand-held device (Palm, Inc., Santa Clara, California, USA) that communicates with a personal computer running a Linux operating system (IBM Corp., Armonk, New York, USA) (Figure 1). Stimuli are high contrast, black-and-white letters with luminance of 85 to 105 candelas/meter2 and contrast of 98%. Both Sloan and HOTV letter sets are available. Single letters are presented framed with crowding bars that are spaced a letter width around the letter. With a high-resolution (1600 1200) 17-inch monitor, letters can be displayed from 20/800 (1.6 logarithm of the minimum angle of resolution [logMAR]) to 20/12 ( 0.2 logMAR) at a test distance of 3 meters. Letters are rendered and presented on the monitor by manipulating the individual points in a graphical image, known as pixels (picture elements). Letter sizes are determined by translating octave steps (3 logMAR lines) to the number of pixels for a given stroke width, beginning with 3 pixels for a 20/12 letter. Letter size is a close, but not exact, approximation of the logMAR progression of the ETDRS charts (within about 2% of the letter size at each logMAR level). The Palm hand-held device, which is connected to the personal computer through a serial cable, provides instructions for the technician, displays the letter that is being illustrated on the monitor, records the responses, and sends instructions to the personal computer with regard to the sequence of letter presentations. The size of each letter presentation can be either controlled by the technician or determined from a computer program based on the subject’s responses. Both the letter graphics and the Palm applications are written in the C programming language.

● DESCRIPTION OF THE EVA:

● ELECTRONIC ETDRS VISUAL ACUITY TESTING PRO-

The E-ETDRS testing protocol is described in Figure 2. Testing begins with a screening phase to determine an approximate visual acuity threshold, using the letters V, R, K, and D (which have been reported to be of intermediate and comparable identification difficulty6). This phase is followed by threshold testing to determine an upper logMAR level at which 5 of 5 letters are correctly identified and a lower logMAR level at which 0 of 5 letters are correctly identified. The specific letters tested at each logMAR level are the same letters as those on the original-series ETDRS charts for the right and left eyes.6 A letter score to approximate the S-ETDRS score is computed as the number of letters correctly identified during threshold testing, plus 5 letters for each logMAR line above the upper boundary through 20/800. TOCOL:

VOL. 135, NO. 2

A COMPUTERIZED METHOD

FIGURE 1. Electronic visual acuity tester. The electronic visual acuity tester utilizes a programmed Palm (Palm, Inc., Santa Clara, California, USA) hand-held device (bottom) that communicates with a personal computer running a Linux operating system (IBM, Armonk, New York, USA). With a high-resolution (1600 1200) 17-inch monitor (top), single letters can be displayed from 20/800 to 20/12 at a test distance of 3 meters. OF

VISUAL ACUITY TESTING

195

FIGURE 2. Electronic Early Treatment for Diabetic Retinopathy Study (E-ETDRS) visual acuity testing strategy. Testing begins with a screening phase to determine an approximate visual acuity threshold. This phase is followed by threshold testing to determine an upper logMAR level at which 5 of 5 letters are correctly identified and a lower logMAR level at which 0 of 5 are correctly identified.

● STUDY PROTOCOL: The test-retest reliability of both E-ETDRS and S-ETDRS testing were evaluated at three sites. The study population consisted of individuals age 7

196

AMERICAN JOURNAL

years old and older who were being seen as patients at the three sites and who agreed to undergo the testing. The study was approved by the respective Institutional Review OF

OPHTHALMOLOGY

FEBRUARY 2003

TABLE 1. Conversions Between Letter, LogMAR, and Snellen Visual Acuity Scores

TABLE 2. Baseline Characteristics of Subjects Baseline Characteristics*

Letter Score

LogMAR Value

Snellen Equivalent

5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95

1.6 1.5 1.4 1.3 1.2 1.1 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0 0.1 0.2

20/800 20/640 20/500 20/400 20/320 20/250 20/200 20/160 20/125 20/100 20/80 20/63 20/50 20/40 20/32 20/25 20/20 20/15 20/12

Age n (%) 7–17 years 18–50 years 50 years mean ( SD), years Female sex, number (%) Ethnicity, number (%) Caucasian African American Hispanic Asian Other Education, number (%) Less than high school High school College Post-graduate Clinical diagnosis, number (%) Normal Uncorrected refractive error Age-related macular degeneration Diabetic retinopathy Other retinal disease Cataract Optic nerve Cornea Uveitis Glaucoma Amblyopia Other Visual acuity on initial S-ETDRS test, number (%) 20/20 (logMAR 0.00) 20/20 to 20/40 (logMAR 0.02 to 0.28) 20/40 to 20/100 (logMAR 0.30 to 0.70) 20/100 (logMAR 0.70)

LogMAR logarithm of the minimal angle of resolution.

Boards, and was in accordance with the Declaration of Helsinki; informed consent was obtained from patients. Visual acuity was measured with both the E-ETDRS and S-ETDRS protocols twice on one eye of each patient. The E-ETDRS testing protocol, which uses single-letter presentations at a 3-meter test distance, is described above. The S-ETDRS testing protocol uses a 5-letter per line chart at a 4-meter test distance. The patient is asked to read each letter starting at the top of the chart (20/200). If fewer than 20 letters are correctly identified at 4 meters, testing is also done at 1 meter. Testing ends when no letters are correctly identified on a line. For the first 151 patients, the right eye was the study eye; thereafter, the eye with the worse acuity was tested so as to increase the number of eyes with reduced acuity in the study sample. At two sites (at which 128 and 60 patients participated) the testing order was S-ETDRS, E-ETDRS, repeat E-ETDRS, and repeat S-ETDRS. This testing order permitted completion of the initial S-ETDRS testing before any other testing to avoid interfering with other studies for which the S-ETDRS data were being used. At one site at which 77 patients participated, patients were not participating in any other studies using the acuity data; therefore the testing order of the S-ETDRS and E-ETDRS protocols was randomly determined for the initial tests, followed by repeat testing in the same order. The testing was conducted in sequence without a break unless the patient was tired, in which case a short break was given. In VOL. 135, NO. 2

A COMPUTERIZED METHOD

Total Number 265

24 (9) 108 (41) 130 (50) 50 22 155 (59) 201 (77) 40 (15) 11 (4) 9 (3) 1 (.4) 30 (15) 56 (28) 80 (40) 35 (17) 53 (20) 21 (8) 25 (10) 15 (6) 58 (22) 11 (4) 3 (1) 5 (2) 30 (11) 29 (11) 2 (1) 11 (4)

55 (21) 77 (29) 80 (30) 53 (20)

S-ETDRS standard Early Treatment for Diabetic Retinopathy Study. *Missing data: age (3), gender (3), ethnicity (3), education (64), and diagnosis (2).

most cases, a patient’s repeat testing was conducted by the same technician as the one who conducted the initial testing. The S-ETDRS testing was conducted with charts placed in a retroilluminated light box at a test distance of 4 meters (as per the protocol, the test distance was reduced to 1 meter when the letter score at 4 meters was less than 20).6 The E-ETDRS testing was conducted at a test distance of 3 meters. The S-ETDRS letter score was calculated as the number of letters correctly identified at OF

VISUAL ACUITY TESTING

197

4 meters plus 30 when the 4-meter score was 20 and plus the number of letters correctly identified at 1 meter when the 4-meter score was 20. All testing was conducted either with the patient’s current spectacles or without correction. A refraction was not performed for purposes of the study; therefore the measured acuities did not necessarily represent best-corrected visual acuity. Study enrollment continued until there were at least 50 subjects with study eyes with acuity worse than 20/100. The primary diagnosis/cause of visual loss of each eye was recorded as one of the following: normal, uncorrected refractive error, macular disease (age-related macular degeneration, diabetic retinopathy, or other retinal diseases or dystrophies), cataract, optic nerve disease, uveitis, glaucoma, amblyopia, or other.

TABLE 3. Test-Retest Differences With the Electronic ETDRS and Standard ETDRS Protocols

Only patients who had scores for all four tests were included in the analysis (data from 14 patients with incomplete testing were not included). For both the E-ETDRS and S-ETDRS tests, the letter scores were converted to logMAR equivalents using the formula logMAR 1.7 (.02)(letter score). Conversions between letter scores and Snellen scores are given in Table 1. With this conversion, a 5-letter difference in visual acuity is equivalent to a difference of 0.1 logMAR and to one Snellen line. To assess test-retest reliability, frequency distributions of the differences in visual acuity scores between the initial and repeat test score were evaluated and intraclass correlation coefficients were computed. Ninety-five percent confidence intervals (95% CI) for an acuity score were determined based on the standard error of measurement,7 and for a change in an acuity score from a baseline level were determined based on the standard error of the differences. Analyses were replicated in subgroups based on visual acuity and age. McNemar test was used to compare proportions in paired data; Fisher exact test was used to compare proportions between subgroups. Separate assessments were made using identical methods for the S-ETDRS and E-ETDRS tests. To compare E-ETDRS scores with the S-ETDRS scores, frequency distributions of the differences between E-ETDRS and S-ETDRS visual acuity scores for each subject were constructed and intraclass correlation coefficients were computed. Analyses were replicated in subgroups based on visual acuity and age. Similar methods were used to compare E-ETDRS and S-ETDRS scores in a subset that included only eyes with high test-retest reliability (both E-ETDRS and S-ETDRS test-retest scores within five letters). Bland-Altman plots8 were constructed to further assess test-retest reliability and the concordance of S-ETDRS and E-ETDRS according to level of visual acuity. All analyses were conducted using SAS software version 8 (Cary, North Carolina, USA).9 ● DATA ANALYSIS:

198

AMERICAN JOURNAL

E-ETDRS

S-ETDRS

(Number 265)

Absolute Value of Difference in Letters*

n (%)†

Cumulative Percent†

n (%)†

Cumulative Percent†

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

34 (13) 63 (24) 53 (20) 40 (15) 26 (10) 19 (7) 12 (5) 5 (2) 5 (2) 3 (1) 1 (.4) 1 (.4) 1 (.4) 0 (0) 1 (.4) 0 (0) 0 (0) 0 (0) 1 (.4)

13% 37% 57% 72% 82% 89% 93% 95% 97% 98% 98% 99% 99% 99% 100% 100% 100% 100% 100%

44 (17) 71 (27) 40 (15) 27 (10) 28 (11) 21 (8) 12 (5) 12 (5) 2 (1) 2 (1) 1 (.4) 2 (1) 2 (1) 1 (.4) 0 (0) 0 (0) 0 (0) 0 (0) 0 (0)

17% 43% 58% 69% 79% 87% 92% 96% 97% 98% 98% 99% 100% 100% 100% 100% 100% 100% 100%

E-ETDRS electronic Early Treatment for Diabetic Retinopathy Study; S-ETDRS standard ETDRS. *0.1 logMAR 5 letters; 0.2 logMAR 10 letters; 0.3 logMAR 15 letters. † Due to rounding, percentages may not sum to 100 and/or their respective cumulative percentages.

RESULTS The mean age of 265 patients was 50 22 years old; 155 were female (59%), and 201 were Caucasian (77%). Fifty-three (20%) study eyes were considered normal (including corrected refractive error), 21 had an uncorrected refractive error (8%), and 189 had eye disease (72%). Visual acuity was better than 20/40 in 132 eyes (50%), 20/40 to 20/100 in 80 eyes (30%), and worse than 20/100 in 53 eyes (20%) (Table 2).

● DESCRIPTION OF SUBJECTS:

● TEST-RETEST RELIABILITY ASSESSMENTS: The distributions of the test-retest differences for both the E-ETDRS and S-ETDRS testing appeared similar (Table 3). For both methods, the correlation between the initial and the retest visual acuity scores was 0.99. For the E-ETDRS testing, 89% of the retest scores were within 0.1 logMAR of the initial test score and 98% were within 0.2 logMAR; whereas for the S-ETDRS testing, 87% of the retest scores were within 0.1 logMAR of the initial test score and 98% were within 0.2 logMAR (Table 4 and Figure 3). Test-retest reliability was high across the range of visual acuity. With both testing methods, more than 90% of OF

OPHTHALMOLOGY

FEBRUARY 2003

TABLE 4. Test-Retest Differences With the Electronic ETDRS and Standard ETDRS Protocols According to Visual Acuity*

All Patients (Number 265)

Visual Acuity 20/40 (Number 132) E-ETDRS

Visual Acuity 20/40 to 20/100 (Number 80)

S-ETDRS

E-ETDRS

S-ETDRS

Visual Acuity 20/100 (Number 53)

E-ETDRS

S-ETDRS

E-ETDRS

S-ETDRS

0.1 logMAR

235 (89)

231 (87)

126 (95)

115 (87)

68 (85)

72 (90)

41 (77)

44 (83)

0.2 logMAR

261 (98)

260 (98)

132 (100)

131 (99)

79 (99)

78 (98)

50 (94)

51 (96)

0.3 logMAR

264 (100)

265 (100)

132 (100)

80 (100)

52 (98)

53 (100)

0.4 logMAR

265 (100)

132 (100)

80 (100)

53 (100)

Absolute value of difference† number (%)

Difference between repeat and initial scores in letters‡ mean SD (95% CI) Intraclass

0.77 3.64

1.10 3.50

0.29 2.80

0.77 3.38

1.15 3.65

1.50 3.33

1.42 5.09

1.34 4.00

(0.33 to 1.21)

(0.68 to 1.53)

( 0.19 to 0.77)

(0.18 to 1.35)

(0.34 to 1.96)

(0.76 to 2.24)

(0.01 to 2.82)

(0.24 to 2.44)

.99

.94

.86

.91

.87

.93

.95

.10

.08

.09

.08

.14

.11

.14

.11

.13

.11

.20

.15

.12

.09

.11

.10

.17

.13

correlation coefficient 95% CI (two-sided) for acuity score, logMAR: halfwidth 95% CI for change between two acuity scores, logMAR Half-width of two-sided CI Width of onesided CI

CI confidence interval; E-ETDRS electronic Early Treatment for Diabetic Retinopathy Study; logMAR logarithm of the minimal angle of resolution; S-ETDRS standard ETDRS; SD standard deviation. *Visual acuity based on letter score from initial standard ETDRS test. Visual acuity 20/40 logMAR 0.28; 20/40 –20/100 logMAR 0.30 to 0.70; and 20/100 logMAR 0.70. † 0.1 logMAR 5 letters; 0.2 logMAR 10 letters; 0.3 logMAR 15 letters; 0.4 logMAR 20 letters. ‡ Positive value means repeat score was higher (better).

retests were within 0.2 logMAR of the initial test even when acuity was worse than 20/100. When acuity was better than 20/40, the proportion of retests within 0.1 logMAR of the initial test was higher with the E-ETDRS method than with the S-ETDRS method (P .02; Table 4 and Figure 4). Table 4 also provides the 95% CI for both methods for an individual acuity score as well as for a change in acuity. Test-retest reliability was high irrespective of age (Table 5). With both testing methods, more than 95% of retests were within 0.2 logMAR of the initial test even in patients 65 years old or older. ● COMPARISON OF E-ETDRS AND S-ETDRS SCORES:

Comparing each patient’s E-ETDRS score and S-ETDRS score, there was no tendency for scores on one test to be higher than on the other (Table 6). The correlation VOL. 135, NO. 2

A COMPUTERIZED METHOD

between the E-ETDRS and S-ETDRS acuity scores was 0.96 for the initial tests and 0.97 for the retests. The E-ETDRS and S-ETDRS scores differed by 0.1 logMAR on 74% of the initial tests and 79% of the repeat tests and differed by 0.2 logMAR on 94% and 96% of tests, respectively (Table 6 and Figure 5). Agreement between the E-ETDRS and S-ETDRS scores was lower in patients with acuity worse than 20/100 compared with patients with acuity of 20/100 or better (for agreement within 0.2 logMAR, P .05 for initial tests and P .01 for repeat tests; Table 6 and Figure 6), but agreement was similar when comparing patients 65 years old or older with patients younger than 65 years old (Table 7). With the analysis limited to the 207 patients with high test-retest reliability (test-retest scores within 0.1 logMAR with both the E-ETDRS and S-ETDRS tests), the OF

VISUAL ACUITY TESTING

199

FIGURE 4. Bland Altman8 plot of test-retest difference versus average visual acuity score (number 265). A positive difference indicates the repeat score was better than the initial score. The dotted lines separate the test-retest scores that were within 10 letters of each other from those in which the difference was greater than 10 letters.

FIGURE 3. Distribution of test-retest differences in acuity scores (number 265). A positive difference indicates the repeat score was better than the initial score.

scores on the two tests were similar to the overall results (data not shown).

E-ETDRS and S-ETDRS scores differed by 0.1 logMAR on 81% of the initial tests and 84% of the repeat tests, and differed by 0.2 logMAR on 95% and 99% of tests, respectively.

DISCUSSION

Although there were some differences in the characteristics of the patients when comparing the three clinical sites (as was expected, due to the different types of patients each had access to for inclusion in the study), there were no meaningful differences in the results comparing the centers (data not shown). At the one site at which the testing order of S-ETDRS and E-ETDRS was determined at random, the test-retest reliability results and the results of the comparison of the ● DIFFERENCES AMONG CENTERS:

200

AMERICAN JOURNAL

WE FOUND THAT OUR COMPUTERIZED VISUAL ACUITY

testing protocol has test-retest reliability comparable to the standard ETDRS chart testing protocol on which it was based. With both testing methods, close to 90% of patients tested within 0.1 logMAR (5 letters) of the initial score on the retest, and more than 95% tested within 0.2 logMAR (10 letters) on the retest. Test-retest reliability did not vary with age, but, not unexpectedly, variability was slightly greater in patients with poor acuity. Still, about OF

OPHTHALMOLOGY

FEBRUARY 2003

TABLE 5. Test-Retest Differences With the E-ETDRS and S-ETDRS Protocols According to Age* Age 40– 65 Years Old Age 40 Years Old (Number 86) Absolute Value of

(Number 94)

Age 65 Years Old (Number 82)

Difference†

E-ETDRS n (%)

S-ETDRS n (%)

E-ETDRS n (%)

S-ETDRS n (%)

E-ETDRS n (%)

S-ETDRS n (%)

0.1 0.2 0.3 0.4

80 (93) 84 (98) 86 (100) 86 (100)

72 (84) 83 (97) 86 (100) 86 (100)

79 (84) 93 (99) 94 (100) 94 (100)

84 (89) 93 (99) 94 (100) 94 (100)

74 (90) 81 (99) 81 (99) 82 (100)

72 (88) 81 (99) 82 (100) 82 (100)

logMAR logMAR logMAR logMAR

E-ETDRS electronic Early Treatment for Diabetic Retinopathy Study; logMAR logarithm of the minimal angle of resolution; S-ETDRS standard ETDRS. *Three patients are missing data on age. † 0.1 logMAR 5 letters; 0.2 logMAR 10 letters; 0.3 logMAR 15 letters; 0.4 logMAR 20 letters.

TABLE 6. Differences Between E-ETDRS and S-ETDRS According to Visual Acuity*

Initial test Absolute value of difference† number (%) 0.1 logMAR 0.2 logMAR 0.3 logMAR 0.4 logMAR Difference between E-ETDRS and S-ETDRS scores, letters mean SD (95% CI)‡ Intraclass correlation coefficient Repeat test Absolute value of difference† number (%) 0.1 logMAR 0.2 logMAR 0.3 logMAR 0.4 logMAR Difference between E-ETDRS and S-ETDRS scores, letters mean SD (95% CI)‡ Intraclass correlation coefficient

All Patients (Number 265)

Visual Acuity 20/40 (Number 132)

Visual Acuity 20/40 to 20/100 (Number 80)

Visual Acuity 20/100 (Number 53)

196 (74) 248 (94) 261 (98) 262 (99)

108 (82) 127 (96) 131 (99) 131 (99)

58 (73) 75 (94) 80 (100) 80 (100)

30 (57) 46 (87) 50 (94) 51 (96)

0.14 5.94 ( 0.86 to 0.58) .96

0.48 5.09 ( 1.35 to 0.40) .77

0.45 5.25 ( 0.72 to 1.62) .76

0.19 8.44 ( 2.52 to 2.14) .81

210 (79) 255 (96) 260 (98) 263 (99)

117 (89) 130 (98) 131 (99) 131 (99)

64 (80) 78 (98) 79 (99) 80 (100)

29 (55) 47 (89) 50 (94) 52 (98)

0.47 5.14 ( 1.09 to 0.15) .97

0.95 4.28 ( 1.69 to 0.22) .82

0.10 4.64 ( 0.93 to 1.13) .85

0.11 7.36 ( 2.14 to 1.91) .86

CI confidence interval; E-ETDRS electronic Early Treatment for Diabetic Retinopathy Study; logMAR logarithm of the minimum angle of resolution; SD standard deviation; S-ETDRS standard ETDRS; SD standard deviation. *Visual acuity based on letter score from initial standard ETDRS test: 20/40 logMAR 0.28; 20/40 –20/100 logMAR 0.30 to 0.70; and 20/100 logMAR 0.70. † 0.1 logMAR 5 letters; 0.2 logMAR 10 letters; 0.3 logMAR 15 letters; 0.4 logMAR 20 letters. ‡ Negative value means S-ETDRS letter score was higher (better).

80% of patients with poor acuity (worse than 20/100) had retest scores within 0.1 logMAR (5 letters) of the initial test scores, and about 95% retested within 0.2 logMAR of the initial score. Based on our estimates of 95% CI, we found that a change in acuity of 0.2 logMAR (10 letters) VOL. 135, NO. 2

A COMPUTERIZED METHOD

from a baseline level is unlikely to be related to measurement variability using either the E-ETDRS or the S-ETDRS visual acuity testing protocol. We found agreement between the E-ETDRS and S-ETDRS scores to be high, suggesting strong concurrent OF

VISUAL ACUITY TESTING

201

FIGURE 5. Distribution of differences between acuity scores on electronic Early Treatment for Diabetic Retinopathy Study (ETDRS) and standard ETDRS testing (number 265). A positive difference indicates the electronic ETDRS score was better than the standard ETDRS score.

validity. The differences in visual acuity scores, however, were greater in comparing the two testing methods than in the repeating of each test. This is to be expected because the two testing strategies and their methods of computing the visual acuity score differ, thus adding a second source 202

AMERICAN JOURNAL

of variation in addition to the test-retest variability. This is particularly true for the testing of patients with low vision, for whom the S-ETDRS score combines the results of testing at 4 meters and 1 meter, whereas the E-ETDRS testing is at a single distance. The E-ETDRS score was OF

OPHTHALMOLOGY

FEBRUARY 2003

FIGURE 6. Bland Altman8 plot of difference between acuity scores on electronic Early Treatment for Diabetic Retinopathy Study (ETDRS) and standard ETDRS vs average visual acuity score (number 265). A positive difference indicates the electronic ETDRS score was better than the standard ETDRS score. The dotted lines separate the E-ETDRS: S-ETDRS scores that were within 10 letters of each other from those in which the difference was greater than 10 letters.

developed to be similar to the S-ETDRS letter score and to maintain the advantages of letter scoring over line scoring.6,10 –13 However, some variation undoubtedly is due to differences in the scoring methods and to the effect of using single-letter presentations with the E-ETDRS procedure and line presentations with the S-ETDRS VOL. 135, NO. 2

A COMPUTERIZED METHOD

procedure.14 –16 There was no tendency for the EETDRS and S-ETDRS scores to be higher or lower than the other. For both procedures, retest scores on average were about one letter (or less) higher than the initial test scores, indicative of an inconsequential learning effect. Most prior OF

VISUAL ACUITY TESTING

203

TABLE 7. Differences Between E-ETDRS and S-ETDRS According to Age Absolute Value of Difference*

0.1 0.2 0.3 0.4

logMAR logMAR logMAR logMAR

Age 40 Years Old (Number 86)

Age 40– 65 Years Old (Number 94)

Age 65 Years Old (Number 82)

Initial Test n (%)

Repeat Test n (%)

Initial Test n (%)

Repeat Test n (%)

Initial Test n (%)

Repeat Test n (%)

61 (71) 82 (95) 86 (100) 86 (100)

73 (85) 84 (98) 86 (100) 86 (100)

73 (78) 87 (93) 93 (99) 93 (99)

74 (79) 91 (97) 92 (98) 93 (99)

60 (73) 77 (94) 79 (96) 80 (98)

62 (76) 77 (94) 79 (96) 81 (99)

E-ETDRS electronic Early Treatment for Diabetic Retinopathy Study; logMAR logarithm of the minimal angle of resolution; S-ETDRS standard ETDRS. *0.1 logMAR 5 letters; 0.2 logMAR 10 letters; 0.3 logMAR 15 letters; 0.4 logMAR 20 letters.

near-normal vision. Only the study of Blackhurst and associates2 included a substantial number of eyes with visual impairment. In that study, visual acuity was tested twice using the ETDRS charts in 89 eyes with acuity of 20/100 or better and in 75 eyes with acuity worse than 20/100. Test-retest reliability was higher in the eyes in the better vision group. The percentages of eyes with retest scores within 0.1 logMAR and 0.2 logMAR of the initial scores were 92% and 98%, respectively, in the better vision group and 80% and 89%, respectively, in the worse vision group. The intraclass correlation coefficient was 0.95 in each group. In summary, the E-ETDRS protocol has high test-retest reliability and good concordance with S-ETDRS testing. Although we expect that this testing method will be useful for pediatric patients as young as 6 or 7 years of age, we had too few children in the study to be able to determine this and, therefore, will be conducting a separate test-retest reliability study in a pediatric population. For a clinical trial, the potential advantages of using a computerized method of testing over manual testing include better standardization of the testing procedure across multiple sites, less training required for the technicians administering the test, reduction of potential bias by limiting the role of the technician in the testing procedure, the ability to test visual acuity from 20/800 to 20/12 at a single test distance, the ability to directly capture the testing data electronically without the need to manually record every response on a score sheet and to calculate the score automatically, and the need for fewer optotype presentations when visual acuity is good. Disadvantages include the cost of purchasing and maintaining the equipment, the inability to measure 20/10 visual acuity at the 3-meter test distance (although this would be possible if the test distance were increased to 4 meters), and the fact that letter size is a close approximation, but not an exact logMAR progression, due to the method by which the letters are rendered on the personal computer. Despite the potential advantages of the E-ETDRS protocol, incorporating this new procedure into ongoing clinical studies that are using the S-ETDRS protocol to assess change in visual

studies2,3,6,11 have also reported that the learning effect is minimal using S-ETDRS or similar testing. Camparini and colleagues17 reported the results of an adaptive strategy for visual acuity chart testing called ETDRS-fast. The procedure involves a determination of approximate threshold followed by the testing of all 5 letters on a line. The approximate threshold is determined by showing one letter per line on the chart starting at 20/200 until a letter is missed and then showing multiple letters per line until the lowest line with 4 or 5 correct is identified. Smaller lines are then tested as per the standard ETDRS testing protocol until 0 or 1 letter is correct on a line. A letter score is determined, similar to our scoring method, by assuming that all letters larger than the largest tested line would have been correctly identified and none of the letters smaller than the smallest tested line would have been correctly identified. Both test-retest reliability (r 0.96; 97% of retests within 0.1 logMAR of initial test) and the correlation of the ETDRS-fast score with the standard ETDRS score (r 0.95) were found to be high in a study of 57 patients with acuities in the range of 20/100 to 20/10, most of whom had acuity of 20/40 or better. With the E-ETDRS testing method, the number of letter presentations in our study averaged about 25 letters (for the screening phase and threshold testing combined) when visual acuity was better than 20/40 and about 30 letters when acuity was 20/40 or worse. Compared with S-ETDRS testing, the number of letter presentations with the E-ETDRS method usually will be fewer for acuities better than 20/63. The better the acuity, the larger the difference in number of letters tested will be. For instance, with acuity of 20/20, about 55 letters will be tested with S-ETDRS testing compared with about 25 letters with E-ETDRS testing. For very poor acuity, however, the E-ETDRS procedure may test more letters than will the S-ETDRS procedure, depending on whether S-ETDRS testing is also done at 1 meter. Several other studies have demonstrated that the ETDRS testing and similar optotype testing with letter scoring have high test-retest reliability.2,6,11,13 Most of the studies have predominately included eyes with normal or 204

AMERICAN JOURNAL

OPHTHALMOLOGY

FEBRUARY 2003

acuity seems inappropriate. However, for new clinical studies, the E-ETDRS method is a viable alternative that should be considered. The E-ETDRS protocol can be adapted for use on commercially-available computerized visual acuity testers using different operating systems. However, for use with standard office personal computers, considerable modifications would be needed to provide the necessary pixel resolution. Information on the required technical specifications can be obtained by contacting the authors. Future work is indicated to determine whether the E-ETDRS testing algorithm can be modified to further reduce the number of letter presentations by altering the stopping rules and/or reducing the number of presentations at each logMAR level.

5. 6. 7. 8.

ACKNOWLEDGMENTS

The following research staff participated in the study at the clinical sites: R. Mercer, L. Goodman, D. Koutsandras, T. LaReau, R. Nashwinter, J. P. Rowan, G. Foster, W. R. O’Donnell (National Eye Institute, Division of Epidemiology and Clinical Research, National Institutes of Health, Bethesda, MD); K. G. Locke, C. E. Wilson (Retina Foundation of the Southwest, Dallas, TX); and J. Thompson, T. Smith, C. Blachly, K. Novitsky (Discoveries in Sight Research Labs, Devers Eye Institute, Portland, OR).

REFERENCES 1. Early Treatment Diabetic Retinopathy Study Research Group. Early treatment diabetic retinopathy study design and baseline patient characteristics. ETDRS report number 7. Ophthalmology 1991;98:741–756. 2. Blackhurst DW, Maguire MG, Macular Photocoagulation Study Group. Reproducibility of refraction and visual acuity measurement under a standard protocol. Retina 1989;9:163– 169. 3. Elliott DB, Sheridan M. The use of accurate visual acuity

VOL. 135, NO. 2

A COMPUTERIZED METHOD

9. 10. 11. 12. 13. 14. 15. 16. 17.

measurements in clinical anti-cataract formulation trials. Ophthalmic Physiol Opt 1988;8:397–401. Moke PS, Turpin AH, Beck RW, et al. Computerized method of visual acuity testing: adaptation of the amblyopia treatment study visual acuity testing protocol. Am J Ophthamol 2001;132:903–909. Holmes JM, Beck RW, Repka MX, et al. The amblyopia treatment study visual acuity testing protocol. Arch Ophthalmol 2001;119:1345–1353. Ferris FL, Kassoff A, Bresnick GH, Bailey I. New visual acuity charts for clinical research. Am J Opthalmol 1982;94: 91–96. Crocker L, Algina J. Introduction to classical and modern test theory. New York: Holt, Rinehart and Winston, 1986: 150 –151. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986;1:307–310. SAS. SAS/STAT user’s guide, version 8. Cary, NC: SAS Institute, Inc, 1999. Vanden Bosch ME, Wall M. Visual acuity scored by the letter-by-letter or probit methods has lower retest variability than the line assignment method. Eye 1997;11:411–417. Lovie-Kitchin JE. Validity and reliability of visual acuity measurements. Ophthalmic Physiol Opt 1988;8:363–370. Bailey IL, Bullimore MA, Raasch TW, Taylor HR. Clinical grading and the effects of scaling. Invest Ophthalmol Vis Sci 1991;32:422–432. Arditi A, Cagenello R. On the statistical reliability of letter-chart visual acuity measurements. Invest Ophthalmol Vis Sci 1993;34:120 –129. Stager DR, Everett ME, Birch EE. Comparison of crowding bar and linear optotype acuity in amblyopia. Am Orthopt J 1990;40:51–56. Flom MC, Weymouth FW, Kahneman D. Visual resolution and contour interaction. J Opt Soc Am 1963;53:1026 –1032. Stuart JA, Burian HM. A study of separation difficulty: its relationship to visual acuity in normal and amblyopic eyes. Am J Ophthalmol 1962;53:471–477. Camparini M, Cassinari P, Ferrigno L, Macaluso C. ETDRSfast: implementing psychophysical adaptive methods to standardized visual acuity measurement with ETDRS charts. Invest Ophthalmol Vis Sci 2001;42:1226 –1231.

VISUAL ACUITY TESTING

205

C L I N I C A L A N D E X P E R I M E N TA L

RESEARCH

Validation of an automated-ETDRS near and intermediate visual acuity measurement Clin Exp Optom 2019

Yi Pang OD PhD Lauren Sparschu BS Elyse Nylin BS Department of Dean’s Education, Illinois College of Optometry, Chicago, Illinois, USA E-mail: ypang@ico.edu

Submitted: 27 February 2019 Revised: 20 September 2019 Accepted for publication: 15 October 2019

DOI:10.1111/cxo.13018

Background: The aim of this study was to determine the repeatability of an automated-ETDRS (Early Treatment Diabetic Retinopathy Study) near and intermediate visual acuity measurement in subjects with normal visual acuity and subjects with reduced visual acuity. The agreement of automated-ETDRS with gold standard chart-based visual acuity measurement was also studied. Methods: Fifty-one subjects were tested (aged 23 to 91 years; 33 subjects with normal visual acuity: 6/7.5 or better; 18 subjects with reduced visual acuity: 6/9 to 6/30). Near and intermediate visual acuity of one eye from each subject was measured with an automated tablet-computer system (M&S Technologies, Inc.) and Precision Vision paper chart in a random sequence. Subjects were retested one week later. Repeatability was evaluated using the 95 per cent limits of agreement (LoA) between the two visits. Results: Average difference between automated-ETDRS near visual acuity and near visual acuity by paper chart was 0.02 0.10 logMAR (p > 0.05). Agreement of near visual acuity between automated-ETDRS and paper chart was good, with 95 per cent LoA of 0.19 logMAR. Furthermore, automated-ETDRS near visual acuity showed good repeatability (95 per cent LoA of 0.20). Mean difference between automated-ETDRS intermediate visual acuity and intermediate visual acuity by paper chart was 0.02 0.10 logMAR (p > 0.05). Agreement of intermediate visual acuity between automated-ETDRS and paper chart was good, with 95 per cent LoA of 0.20 logMAR. In addition, automated-ETDRS intermediate visual acuity had good repeatability (95 per cent LoA of 0.16). Conclusion: Automated-ETDRS near and intermediate visual acuity measurement showed good repeatability and agreement with the gold standard chart-based visual acuity measurement. The findings of this study indicate the automated visual acuity measurement system may have potential for use in both patient care and clinical trials.

Key words: computer, intermediate vision, near vision, tablet, visual acuity

Visual acuity is the most common and primary measurement of visual function. In addition, visual acuity is one of most common outcome measurements in clinical research.1–5 The Early Treatment Diabetic Retinopathy Study (ETDRS) chart is designed for use in clinical studies where accurate and repeatable visual acuity measurements are required.6,7 Although many types of visual acuity charts are used, the ETDRS chart is accepted worldwide as the gold standard for accurate visual acuity measurement. With advances in technology, automated visual acuity tests using electronic devices have been developed and studied.8–11 The advantages of automated visual acuity testing over paper chart testing include use of a standardised protocol, reduced risk of patients memorising the letters, and limiting human error in counting letters and calculating visual

acuity. Thus, many clinical trials use automated visual acuity testing as the gold standard to measure treatment outcomes.3–5 The purpose of this study was to determine the repeatability of an automatedETDRS near and intermediate visual acuity measurement in subjects with normal visual acuity and subjects with reduced visual acuity. In addition, agreement of automatedETDRS with gold standard chart-based visual acuity measurement was also studied.

Methods Study population and data collection Both the study protocol and informed consent forms were approved by the Institutional Review Board of the Illinois College of

Optometry. In accordance with the guidelines of the Declaration of Helsinki, written informed consent was obtained from each subject.

Test procedure A total of 51 sequential subjects who underwent a comprehensive eye examination at the Illinois Eye Institute, an urban eye clinic, were enrolled into the study. Thirty-three subjects had normal distance visual acuity of 6/7.5 or better, with habitual refractive correction. The remaining 18 subjects had reduced habitual distance visual acuity (from 6/9 to 6/30) due to cataract, glaucoma, degenerative myopia, retinal disease, diabetic retinopathy, and/or uncorrected refractive error. Patients with distance visual acuity worse than 20/100 were excluded from this study.

Clinical and Experimental Optometry 2019

Automated-ETDRS visual acuity Pang, Sparschu and Nylin

Subjects wore their habitual refractive correction for all tests. Right eye visual acuity of each subject was measured with an automated tablet-computer system (M&S Technologies, Inc., Niles, IL, USA) and with a paper ETDRS chart (Precision Vision, Woodstock, IL, USA) in a random sequence. Left eye visual acuity was measured in two subjects whose right eye vision was worse than the inclusion limit. Near visual acuity at 40 cm was measured first, followed by intermediate visual acuity at 67 cm. Both paper and automated-ETDRS tests were measured using high-contrast letters. Visual acuity with the paper ETDRS chart was tested at the luminance of 100–110 cd/m2. For paper ETDRS chart, subjects were asked to read the smallest line they could read. Subjects were asked to read the line below the smallest line that they have read. The test was terminated if no letter in the line was correctly read. Subjects were given at least a one-minute break between each test. A retest was performed one week ( 3 days) after the first visit. Forty-eight subjects (94 per cent) completed the retest.

Phase II: threshold

Automated tablet-computer visual acuity test algorithms

Data analysis

ETDRS letters were displayed on a computer screen for subjects to read. The examiner viewed a tablet that displayed the correct answers (Figure S1). The computer screen was auto-calibrated to the luminance level of 85 cd/m2 ( 5 per cent) after 15 minutes warm up. A Datacolor Spyder 5 colorimeter as part of the automated-ETDRS system was used to confirm the luminance level of the computer screen. Glare was controlled by using a 90o metal display tray that the computer was set into; thus, a consistent angle for the screen display was maintained. The resolution of the 13.3 inch computer screen was 3,200 × 1,800, which resulted in a resolution of 276 pixels per inch, a pixel size of 0.79 minutes of arc at 40 cm, and a pixel size of 0.47 minutes of arc at 67 cm. Filtered optotypes were used for both near and intermediate automated-ETDRS test. Two phases were used to determine the visual acuity endpoint.

Phase I: initial threshold An ETDRS chart (from 6/60 to 6/3) was displayed on the computer screen and subjects were instructed to read the smallest line for which they could read all five letters. The examiner submitted the visual acuity level at which subjects correctly read all letters.

Clinical and Experimental Optometry 2019

ETDRS letters at the initial threshold visual acuity level the examiner submitted were displayed on the computer screen as well as smaller size lines of the chart. A blue dot was shown next to the visual acuity level one line below the submitted letter size and subjects were instructed to read the line next to the blue dot. The examiner submitted the correct number of letters that subjects read and then continued to instruct subjects to read letters of decreasing size (0.1 logMAR steps). The test stopped when subjects were unable to correctly read any letters on a line or there were no smaller lines available to be tested. This ending point protocol has been utilised in previous studies.12–14 The system automatically calculated the logMAR visual acuity using the correct letters that subjects read. The test results were displayed on the computer after the completion of the measurement with the following parameters: eye tested, test distance, letter score for visual acuity, logMAR visual acuity, and Snellen visual acuity equivalent.

Visual acuity was converted to logMAR for analysis. The Kolmogorov–Smirnov test was used to check the normality of distributions for all parameters. Near visual acuity and intermediate visual acuity tested with the paper chart were not normally distributed. Wilcoxon signed rank test was performed to determine the difference in visual acuity tested by paper chart and automatedETDRS. Test–retest reliability of the automatedETDRS was evaluated using the Bland– Altman 95 per cent limits of agreement (LoA) method. Agreement between the automated-ETDRS and the gold standard paper chart was also determined by the Bland–Altman method. Power calculation was performed after data of 51 subjects were collected. Based on the standard deviation of difference between paper and automated-ETDRS visual acuity test, there was 93.8 per cent power to detect difference of 0.05 logMAR (0.5 logMAR line or 2.5 letters) between the two tests. All data were analysed using Statistical Package for Social Sciences (IBM SPSS version 21.0; SPSS Inc., IBM UK, Portsmouth, UK) and MedCalc version 12.2.1 (MedCalc Software, Ostend, Belgium). A p-value of < 0.05 was considered statistically significant.

Results Table 1 lists demographic characteristics of the subjects. The average difference between the automated-ETDRS near visual acuity and near visual acuity by paper chart was 0.02 0.10 logMAR (one letter difference) without statistical significance (Figure 1A). The mean difference between the test and retest measurements for both automated-ETDRS near visual acuity and near visual acuity by paper chart A was 0.02 logMAR (one letter) with no statistical significance. Agreement of near visual acuity between the automatedETDRS and paper chart (Figure 1A) was good, with 95 per cent LoA of 0.19 logMAR. Furthermore, the automated-ETDRS near visual acuity showed good repeatability (95 per cent LoA of 0.20, Figure 2A) between the two sessions (one week apart), slightly better than that of near visual acuity by paper (95 per cent LoA of 0.24, Figure 3A). Mean difference between the automatedETDRS intermediate visual acuity and intermediate visual acuity by paper chart was 0.02 0.10 logMAR (one letter difference) without statistical significance (Figure 1B). Average differences between the test and retest measurements were 0.02 0.12 logMAR and 0.01 0.10 logMAR for automated-ETDRS intermediate visual acuity and the intermediate visual acuity by paper, respectively. Agreement of intermediate visual acuity between the automated-ETDRS and paper chart was good, with 95 per cent LoA of 0.22 logMAR (Figure 1B). In addition, the automated-

n (%) Visual acuity 6/7.5 or better

33 (64.7)

6/9 to 6/30 Gender

18 (35.3)

Female

42 (82.4)

Male Race

9 (17.6)

Non-Hispanic Black

22 (43.1)

Hispanic/Latino Non-Hispanic White

8 (15.7) 16 (31.4)

Asian Age (years) Range Mean (SD)

5 (9.8) 22.6–91.1 46.7 (17.5)

Table 1. Demographic characteristics of the subjects (n = 51)

Automated-ETDRS visual acuity Pang, Sparschu and Nylin

Figure 1. Agreement between automated-Early Treatment Diabetic Retinopathy Study visual acuity (A-ETDRS-VA) test and paper chart visual acuity (P-VA) test. The difference between the first administration of each test was plotted against the mean for the two tests. Exact confidence intervals were calculated and plotted.32 A: Near visual acuity test (NVA). B: Intermediate visual acuity test (IVA).

ETDRS intermediate visual acuity had good repeatability (95 per cent LoA of 0.16) between the two sessions (Figure 2B), with a slightly better repeatability than that of intermediate visual acuity by paper (95 per cent LoA of 0.20, Figure 3B).

Repeatability of automated-ETDRS near visual acuity and intermediate visual acuity was not significantly different between subjects with normal and reduced visual acuity using independent-samples t-test (all p values > 0.05).

Discussion Repeatability of visual acuity has been extensively studied.11,15–24 Repeatability of distance visual acuity (95 per cent LoA) has been reported at rates varying from 0.07

Figure 2. Repeatability of automated-Early Treatment Diabetic Retinopathy Study visual acuity (A-ETDRS-VA) two measurements, one week apart. The difference between the first and second administrations of A-ETDRS-VA was plotted against the mean of the two measurements. Exact confidence intervals were calculated and plotted.32 A: Near visual acuity test (NVA). B: Intermediate visual acuity test (IVA).

Clinical and Experimental Optometry 2019

Automated-ETDRS visual acuity Pang, Sparschu and Nylin

Figure 3. Repeatability of paper chart visual acuity (P-VA) two measurements, one week apart. The difference between the first and second administrations of P-VA was plotted against the mean of the two measurements. Exact confidence intervals were calculated and plotted.32 A: Near visual acuity test (NVA). B: Intermediate visual acuity test (IVA). to 0.25 logMAR.15–17,20,22,25 Many factors could contribute to test–retest variability, including use of different visual acuity charts, subject age (children and older individuals showing higher variability),26,27 optical defocus (higher variability),25 and ocular abnormalities (higher variability).20,26 Despite numerous reports on repeatability of distance visual acuity, fewer studies of near visual acuity11,23,24 have been conducted. Aslam et al.11 tested near visual acuity at two times, 15–20 minutes apart, in 78 subjects with or without eye pathologies, using an automated computer tablet-based visual acuity system. They reported the repeatability was 0.17 logMAR, which is comparable to the current study. These authors also found that the agreement between their automated near visual acuity and paper near chart (near Landolt C) was 0.31 logMAR, which is worse than the present study. Cho and Woo23 measured near visual acuity in 55 young optometry students with normal vision and reported that the repeatability of near visual acuity using a paper chart (Waterloo Four-Contrast logMAR Visual Acuity Chart) was 0.06 logMAR. Lam et al. measured near visual acuity in 55 young optometry students with visual acuity 6/6 or better and stated that the repeatability of near visual acuity was 0.06 logMAR for a PolyU paper chart and 0.12 logMAR for the Precision paper charts.24 The repeatability

Clinical and Experimental Optometry 2019

of near visual acuity in the present study is lower than in both studies by Cho and Woo and Lam et al.23,24 The two studies referred to above tested young adults with normal visual acuity.23,24 Both factors, young adult and normal visual acuity, have been reported to be associated with a better visual acuity repeatability,25–27 which could contribute to the disparities between those studies and the present work. In addition, subjects in both studies were optometry students, who might understand the test procedure better and provide more reliable responses. Other factors could affect test–retest repeatability including termination rules of visual acuity tests. Carkeet28 suggested that a termination rule of four mistakes or more on a line should be used for ETDRS charts and single-letter scoring. Shah et al.29 tested vision in 50 normal observers and determined that test–retest variability was smaller for a termination rule of four or five mistakes ( 0.14 logMAR) compared to the rule of three mistakes or less. The present study used a termination rule of five mistakes to minimise the test–retest variability as suggested by previous authors.28,29 With the extensive use of computers, intermediate visual acuity becomes more critical in the lives of many individuals. However, only one study of intermediate visual acuity was identified after a thorough literature search. Tsilimbaris et al.30 simulated presbyopia in

34 healthy young emmetropes or myopes without ocular abnormalities. They reported the mean intermediate visual acuity was 0.01 0.09; however, they did not study repeatability of intermediate visual acuity. In the present study, both the repeatability of intermediate visual acuity using automatedETDRS and agreement of automated-ETDRS with paper charts were comparable to that of near visual acuity. There are some limitations of the present study. First, the letter sizes in the automated-ETDRS system were limited by the screen size of the monitor; thus, the range of testing was limited to 6/3 to 6/60. Although a short test distance can be utilised in clinical practice to measure visual acuity worse than 6/60, a shorter test distance may introduce other variables including change of refractive correction. Second, the resolution of the computer screen in the present study was 3,200 × 1,800, which resulted in a resolution of 276 pixels per inch and a pixel size of 0.092 mm, which is lower than suggested by Carkeet and Lister31 in their study. This may potentially impact ETDRS letter resolution of smaller size. In summary, the present study determined the repeatability of automated-ETDRS near and intermediate visual acuity measurement on a wide age range of individuals with normal or reduced vision. Two-session repeatability was measured rather than one-session repeatability. Because visual

Automated-ETDRS visual acuity Pang, Sparschu and Nylin

acuity is measured over time on different visits in both patient care and clinical trials, it is critical to establish two-session visual acuity repeatability. The present study revealed that the automated-ETDRS near and intermediate visual acuity measurements had good repeatability and agreement with the gold standard paper chart, which indicates this automated visual acuity measurement system may have potential for use in both patient care and clinical trials.

10.

11.

12.

ACKNOWLEDGEMENTS The authors would like to thank M&S Technologies, Inc. for providing the automated tabletcomputer visual acuity system for this study. The authors also would like to thank Dr Kelly Frantz in helping edit the manuscript. REFERENCES 1.

Early treatment diabetic retinopathy study design and baseline patient characteristics. ETDRS report number 7. Ophthalmology 1991; 98: 741–756. Photocoagulation for diabetic macular edema. Early treatment diabetic retinopathy study report number 1. Early treatment diabetic retinopathy study research group. Arch Ophthalmol 1985; 103: 1796–1806. Pediatric Eye Disease Investigator Group. A randomized trial of atropine vs. patching for treatment of moderate amblyopia in children. Arch Ophthalmol 2002; 120: 268–278. Holmes JM, Edwards AR, Beck RW et al. A randomized pilot study of near activities versus non-near activities during patching therapy for amblyopia. J AAPOS 2005; 9: 129–136. Holmes JM, Kraker RT, Beck RW et al. A randomized trial of prescribed patching regimens for treatment of severe amblyopia in children. Ophthalmology 2003; 110: 2075–2087. Told R, Baratsits M, Garhofer G et al. Early treatment diabetic retinopathy study (ETDRS) visual acuity. Ophthalmologe 2013; 110: 960–965. Koenig S, Tonagel F, Schiefer U et al. Assessing visual acuity across five disease types: ETDRS charts are faster with clinical outcome comparable to Landolt Cs. Graefes Arch Clin Exp Ophthalmol 2014; 252: 1093–1099. Zhang ZT, Zhang SC, Huang XG et al. A pilot trial of the iPad tablet computer as a portable device for

13.

14.

15.

16.

17.

visual acuity testing. J Telemed Telecare 2013; 19: 55–59. Bokinni Y, Shah N, Maguire O et al. Performance of a computerised visual acuity measurement device in subjects with age-related macular degeneration: comparison with gold standard ETDRS chart measurements. Eye (Lond) 2015; 29: 1085–1091. Black JM, Jacobs RJ, Phillips G et al. An assessment of the iPad as a testing platform for distance visual acuity in adults. BMJ Open 2013; 3: e002730. Aslam TM, Parry NR, Murray IJ et al. Development and testing of an automated computer tablet-based method for self-testing of high and low contrast near visual acuity in ophthalmic patients. Graefes Arch Clin Exp Ophthalmol 2016; 254: 891–899. Ravikumar A, Sarver EJ, Applegate RA. Change in visual acuity is highly correlated with change in six image quality metrics independent of wavefront error and/or pupil diameter. J Vis 2012; 12: 11. Ravikumar A, Marsack JD, Bedell HE et al. Change in visual acuity is well correlated with change in imagequality metrics for both normal and keratoconic wavefront errors. J Vis 2013; 13: 28. Applegate RA, Marsack JD, Thibos LN. Metrics of retinal image quality predict visual performance in eyes with 20/17 or better visual acuity. Optom Vis Sci 2006; 83: 635–640. Camparini M, Cassinari P, Ferrigno L et al. ETDRS-fast: implementing psychophysical adaptive methods to standardized visual acuity measurement with ETDRS charts. Invest Ophthalmol Vis Sci 2001; 42: 1226–1231. Reeves BC, Wood JM, Hill AR. Reliability of high- and low-contrast letter charts. Ophthalmic Physiol Opt 1993; 13: 17–26. Kheterpal S, Jones HS, Auld R et al. Reliability of visual acuity in children with reduced vision. Ophthalmic Physiol Opt 1996; 16: 447–449.

18. Barrio A, Antona B, Puell MC. Repeatability of mesopic visual acuity measurements using high- and lowcontrast ETDRS letter charts. Graefes Arch Clin Exp Ophthalmol 2015; 253: 791–795. 19. Raasch TW, Bailey IL, Bullimore MA. Repeatability of visual acuity measurement. Optom Vis Sci 1998; 75: 342–348. 20. Blackhurst DW, Maguire MG. Reproducibility of refraction and visual acuity measurement under a standard protocol. The macular photocoagulation study group. Retina 1989; 9: 163–169. 21. Lovie-Kitchin JE. Validity and reliability of visual acuity measurements. Ophthalmic Physiol Opt 1988; 8: 363–370. 22. Manny RE, Hussein M, Gwiazda J et al. Repeatability of ETDRS visual acuity in children. Invest Ophthalmol Vis Sci 2003; 44: 3294–3300. 23. Cho P, Woo GC. Repeatability of the Waterloo fourcontrast LogMAR visual acuity chart and near vision

24.

25.

26.

27.

28.

29.

30.

31.

32.

test card on a group of normal young adults. Ophthalmic Physiol Opt 2004; 24: 427–435. Lam AK, Tong C, Tse J et al. Repeatability of near visual acuity measurement at high and low contrast. Clin Exp Optom 2008; 91: 447–452. Rosser DA, Murdoch IE, Cousens SN. The effect of optical defocus on the test-retest variability of visual acuity measurements. Invest Ophthalmol Vis Sci 2004; 45: 1076–1079. Dougherty BE, Flom RE, Bullimore MA. An evaluation of the Mars letter contrast sensitivity test. Optom Vis Sci 2005; 82: 970–975. Elliott DB, Yang KC, Whitaker D. Visual acuity changes throughout adulthood in normal, healthy eyes: seeing beyond 6/6. Optom Vis Sci 1995; 72: 186–191. Carkeet A. Modeling logMAR visual acuity scores: effects of termination rules and alternative forcedchoice options. Optom Vis Sci 2001; 78: 529–538. Shah N, Dakin SC, Whitaker HL et al. Effect of scoring and termination rules on test-retest variability of a novel high-pass letter acuity chart. Invest Ophthalmol Vis Sci 2014; 55: 1386–1392. Tsilimbaris MK, Plainis S, Tontos C et al. Normative distance, intermediate and near visual acuity in simulated presbyopia. Invest Ophthalmol Vis Sci 2011; 52: 2833. Carkeet A, Lister LJ. Computer monitor pixellation and Sloan letter visual acuity measurement. Ophthalmic Physiol Opt 2018; 38: 144–151. Carkeet A. Exact parametric confidence intervals for bland-Altman limits of agreement. Optom Vis Sci 2015; 92: e71–e80.

Supporting information Additional supporting information may be found in the online version of this article at the publisher’s website: Figure S1. Automated-ETDRS (Early Treatment Diabetic Retinopathy Study) measurement. A: Automated-ETDRS measurement at near in a subject. B: Automated-ETDRS letters viewed by subjects. C: AutomatedETDRS letters viewed by the examiner. D: An example of automated-ETDRS at 6/6 level viewed by the examiner.

Clinical and Experimental Optometry 2019

ORIGINAL ARTICLE

An Evaluation of the M&S Technologies Smart System II for Visual Acuity Measurement in Young Visually-Normal Adults NEIL MCCLENAGHAN, BS, AYUMI KIMURA, BA, and LAWRENCE R. STARK, PhD, FAAO Southern California College of Optometry, Fullerton, California

ABSTRACT Purpose. To compare visual acuity measures obtained with the M&S Technologies Smart System II (SSII) and the revised Early Treatment of Diabetic Retinopathy Study (ETDRS) charts in terms of accuracy and test–retest repeatability. Methods. Monocular visual acuities were taken in 57 young, visually normal adults on two separate visits in which both the SSII system and the ETDRS charts were tested in random order by two masked examiners. The eye to be tested throughout was chosen randomly at the initial visit. Measurements were made through an optimal phoropter correction, determined by a noncycloplegic refraction for a 10-foot distance. Both charts were presented at 10 feet, and were matched closely for luminance. Results. The mean visual acuity in the group was 0.16 log minimum angle of resolution (MAR) for the ETDRS chart and 0.18 log MAR for the SSII, a small but statistically significant difference. A 95% confidence interval for the mean difference in visual acuity between the two charts was 0.033 log MAR to 0.003 log MAR. The test–retest repeatability was not significantly different in the two tests. The 95% limits of agreement for test–retest repeatability were 0.13 log MAR to 0.17 log MAR for the SSII and 0.12 log MAR to 0.13 log MAR for the ETDRS charts. Conclusions. The SSII can provide an accurate (mean difference 0.033 log MAR) and repeatable alternative to the ETDRS charts for visual acuity measurement in young, visually normal, well-corrected individuals. (Optom Vis Sci 2007;84:218–223) Key Words: visual acuity, M&S Technologies Smart System II

isual acuity (VA) is the result of the culmination of many critical elements of the visual process. Despite the importance in evaluating visual function with visual acuity,1 this measurement is often subject to error. Conventional Snellen wall charts in particular have been the topic of many criticisms. The standard for visual acuity measurement is considered to be a chart based on Bailey–Lovie principles.2 We used one such chart series in the present study: the revised Early Treatment of Diabetic Retinopathy Study (ETDRS) charts.3,4 Some of the important features of the Bailey–Lovie inspired charts include: letters that are of approximately equal legibility; equal letters per line to control task difficulty and contour interactions; and a regular progression in letter size between lines. The M&S Technologies Smart System II (SSII; Park Ridge, IL) is a new and a more technologically advanced computer generated visual acuity test (VAT). The SSII comprises a computer processor, 17-inch flat LCD screen monitor, and an interactive keypad con-

troller. The 17-inch flat LCD screen monitor is wall-mounted and manufactured to M&S Technologies’ specifications, with high resolution and a 450:1 nominal contrast ratio.5 The examiner uses the keypad to access directly each primary acuity test, specific optotypes, randomization options, and to increase or decrease the size of the optotype on demand.5 The system allows the examiner a higher amount of control over the variables in visual acuity testing.6 In particular, the ability to present random letter sequences precludes patients from using memorization to pass each line. In addition, the SSII can be calibrated for lane lengths of 6 to 22 feet. Upon visual inspection of an SSII unit, various characteristics were noted which could potentially lead to less accurate or less repeatable VA readings. These included: (1) an unequal number of letters per line with some charts. For example, some lines have five letters, while others have six; (2) letters on each line of varying legibility with some charts. For example, the letter set includes most of the alphabet, instead of a restricted number of letters with

Optometry and Vision Science, Vol. 84, No. 3, March 2007

Evaluation of Visual Acuity Measurement in Young Visually-Normal Adults—McClenaghan et al.

approximately equal legibility, such as in the Sloan set or the British standard set; (3) a slightly uneven progression of letter sizes between lines. For example, the progression follows the 0.1 log minimum angle of resolution (MAR) increment of the ETDRS charts except for the substitution of the nominal 0.2 log MAR line (20/12.5 Snellen equivalent) with a 20/12 Snellen equivalent line; and (4) variable letter quality for 20/20 equivalent letters and smaller when the SSII was calibrated for a 10-foot distance. In particular, on inspection with a loupe, we noted some degree of anti-aliasing for these letters (which could reduce the local contrast of parts of each letter), and observed that the 20/10 equivalent letters were subject to pixilation. In the above list, item (4) was of most interest because if the small letters are not rendered sufficiently well, then legibility might suffer, causing the SSII to underestimate acuity in those individuals with normal visual acuity. In contrast, the ETDRS charts have an equal number of letters per line (five letters), the letters are of approximately equal legibility,4 there is a regular progression of letter size between lines (that is, a difference of 0.1 log MAR per line), and the printed letters are not subject to the rendering limitations of an LCD display. There have been no published studies of high-contrast acuity testing with the SSII. However, one study on low-contrast testing with the SSII suggested the procedure was simple to administer, especially for inexperienced participants.6 The purpose of the current study then is to compare the SSII and ETDRS in terms of the accuracy and repeatability of high-contrast distance visual acuity measures in young, well-corrected, visually normal adults.

METHODS Participants Volunteers were considered for inclusion in the study if they met the following criteria: aged 18 years or older; no current ocular pathology or history of significant ocular pathology; no history of amblyopia, strabismus, or presence of significant uncorrected anisometropia or astigmatism at any time up to 7 years of age; no history of cataract, keratoconus, ocular surgery, or significant eye injuries; no history of systemic diseases with ocular effects; and no current use of medications known to have visual side-effects. Thus, the study is designed to investigate acuity in visually normal adults. There were 60 volunteers, but one volunteer did not meet all the inclusion criteria. Two participants did not come for the second visit, so only complete data for 57 participants were available for analysis. Participants were optometry students of the College. The average age of the participants was 25.7 years with a range of 22.1 to 33.4 years. There were 28 males and 29 females. Best spherical ocular refraction for the right eye ranged from 10.31 to 4.19 D, with an average of 3.26 D. Best spherical ocular refraction for the left eye ranged from 8.52 to 5.22 D, with an average of 2.94 D. The average best-corrected VA (measured just after subjective refraction with the Smart System II) for the right eye was 0.20 log MAR with a range of 0.30 to 0.08 log MAR. For the left eye it was 0.21 log MAR, with a range of 0.30 to 0.02 log MAR. Participants gave informed consent to participate in the study. The protocol was approved by the SCCO Institutional Review Board and followed the tenets of the Declaration of Helsinki.

219

Sample Size Estimates Sample size estimates7 were made for Student’s t-test to compare mean VA by the two VATs. For these estimates, a “clinically important” change in visual acuity was taken as 0.025 log MAR, as such a decrement is considered unacceptable by a significant number of individuals.8 An a priori estimate of the standard deviation of visual acuity measures was taken as 0.03 log MAR.9 For 90% power with a 5% significance level, the required sample size was estimated at 24. For calculations of Bland–Altman limits of agreement, it was necessary to have sufficient power to keep the 95% confidence intervals for those limits within acceptable bounds. Once again, 0.025 log MAR was taken as a clinically important bound. An a priori estimate of the standard deviation for differences between repeated VA measures was taken as 0.056 log MAR.10 These values were then used in formulae provided by Bland and Altman11 to yield a sample size estimate of 58. Other statistical tests in this study (see Statistical Analysis) do not have published methods for estimating sample sizes. In summary, the largest required sample size was estimated as 58, and so this was set as the target sample size of the study.

Apparatus The equipment included ETDRS letter charts (“Revised” chart 1 and “Revised” chart 2; 10-foot design) produced by Precision Vision (La Salle, IL),3,4 and the Smart System II PC-Plus (SSII) by M&S Technologies (Park Ridge, IL). Both charts were set at a test distance of 10 feet. This distance was used because many practices cannot afford the space needed for a 4-m or 6-m test lane. The two systems were matched for luminance using the SSII software, room lighting and a luminance probe. Mean luminance was 141.6 cd m 2 for the SSII and 135.1 cd m 2 for ETDRS, while room illumination was identical in both cases. The SSII was calibrated to a 10-foot distance. The option to “calibrate for short lanes” was not used. The screen size was calibrated using the SSII software. The default letter set was used, which includes all the capital Roman letters except I, J, M, Q, W, X, and Y. The SSII was set to present letters randomly. A default Snellen distance increment was used corresponding to denominators in the Snellen equivalent fraction 20/x of 40, 32, 25, 20, 16, 12, and 10 feet. (By way of comparison, the Precision Vision ETDRS charts have a denominator progression of 40, 32, 25, 20, 16, 12.5, and 10 feet for the Snellen equivalent fraction 20/x.) The default number of letters per line were utilized; namely, five letters for rows with denominators of 40 to 25 feet, and six letters for rows with denominators of 20 to 10 feet, all denominators being for the Snellen equivalent fraction 20/x.

Procedures Data were collected in two visits conducted on different days separated by 1 to 3 days. On the first visit, a brief case history was conducted. Best-corrected vision was determined after a noncycloplegic subjective refraction in a phoropter for a 10-foot test distance. This test distance was to help assure maximal acuity for the distance of the chart. A “maximum positive power for best visual acuity” criterion was used and visual acuities with this cor-

Optometry and Vision Science, Vol. 84, No. 3, March 2007

220

Evaluation of Visual Acuity Measurement in Young Visually-Normal Adults—McClenaghan et al.

rection were recorded using the Smart System II. Next, visual acuities were measured by each procedure through the phoropter with the previously obtained subjective refraction in place. The second visit involved a visual acuity measurement by each procedure under identical test conditions. The order of testing with the ETDRS or SSII was randomized across participants in both the first and second visits. All randomized test orders were obtained with the RAND() function, a pseudorandom number generator in the Microsoft Excel program. The examiners were masked by having one examiner perform one test and the other perform the second test, without communication. Visual acuities were calculated with letter-by-letter scoring. Each correct letter was awarded 0.02 log MAR in lines that had five letters per row, and 0.0167 log MAR in lines that had six letters per row. This study followed recommendations to use a termination rule of stopping after four or more mistakes on a line for letter-byletter scoring.12 Although such a rule implies different effective thresholds for five-letter and six-letter lines, simulations of various line-by-line termination rules suggest that any effects on measured visual acuity would be minor.12 All visual acuity measurements were obtained monocularly. Left or right eye at random for each participant were selected at the initial visit. The tested eye was the right eye in 20 participants and the left eye in 37 participants.

Statistical Analysis The assumptions of Student’s t-test were investigated routinely before its use. The normality of each data distribution was investigated with the Kolmogorov–Smirnov test,13 and homogeneity of variances with the Brown–Forsythe test.14 To assess in part the accuracy of the SSII (see Results), principal axis regression was used.15 This is an alternative to linear regression when there is no true independent variable. (In the current case, we wished to regress SSII visual acuity against ETDRS visual acuity, but neither of these is an independent variable.) Before principal axis regression, plots were investigated visually for outliers. We calculated both the slope and intercept of the best fit line, and a 95% CI for the slope of the best fit line.15 Agreement between tests, and the test–retest repeatability of each test were assessed with Bland–Altman analysis.11 Before analyses, normality of the distributions was assessed with the Kolmogorov– Smirnov test.13 As the Bland–Altman limits of agreement are only estimates of their respective population parameters, 95% CIs for the limits of agreement were also calculated.11 To make a formal comparison of test–retest repeatability with the two VATs, it was necessary to choose a suitable parameter and design a custom statistic. The difference in measured visual acuity between first and second visits (d) was used as a parameter for comparing the test–retest repeatability of the two VATs. Differences in repeatability between VATs could manifest as a difference in average d (for example, a VAT shows a systematic drift between visits), or a difference in the variance of d (for example, one VAT is inherently more variable, but the mean acuity is constant), or a combination of both. Accordingly, the Cramér–von Mises W2 statistic was used as an overall measure of any difference in the empirical distribution functions (also known as cumulative distribution functions) between VATs.16 A probability value was calcu-

lated by a randomization procedure17 with random enumeration (n 500).18

RESULTS Differences Between Two Examiners There were no significant differences in measured visual acuities between the two examiners on either visit for either VAT, by t-test, where d denotes the mean Examiner A acuity reading less the mean Examiner B acuity reading (visit 1, ETDRS, d 0.017 log MAR, t 0.81, p 0.42; visit 1, SSII, d 0.030, t 1.30, p 0.20; visit 2, ETDRS, d 0.010, t 0.56, p 0.58; visit 2, SSII, d 0.027, t 1.33, p 0.19).

Accuracy The average visual acuity in the group was significantly different between VATs (t 2.46, p 0.017), being 0.16 log MAR for ETDRS and 0.18 log MAR for the SSII charts. These values correspond to a visual acuity difference of about one log MAR letter more on SSII in comparison to the ETDRS. A 95% confidence interval for the difference in VA between the two systems was 0.033 to 0.003 log MAR. Thus, we can be reasonably certain that the true difference between charts is less than about 1.65 letters.

Homogeneity of Accuracy Across the Chart One effect we anticipated is that accuracy could be poorer in certain parts of the chart due to design issues such as poor pixel rendition of the very smallest lines on the SSII. To investigate this issue, mean visual acuity for the SSII over both visits was plotted as a function of mean visual acuity for the ETDRS chart over both visits, and principal axis regression15 used to determine the best fitting line to the data (Fig. 1). If the difference in acuity between instruments is uniform across all letter sizes, then this plot should have a slope of 1.00. The best-fit line for SSII visual acuity as a function of ETDRS visual acuity had a slope of 1.1097 and an intercept of 0.001. The 95% confidence interval for the slope was 0.82 to 1.52. The best-fit line has a slope close to 1.0 and an intercept close to zero. This suggests negligible differences between the two systems.

Agreement Between the Two Tests A Bland–Altman analysis was used to present visually and quantitatively the level of agreement between the two instruments (Fig. 2).11 The 95% limits of agreement between the two VATs were 0.129 and 0.092 log MAR. The 95% CIs for the limits of agreement had a largest expected range of 0.155 and 0.118, and a smallest expected range of 0.104 and 0.067.

Test–Retest Repeatability A Bland–Altman analysis was used separately for each instrument to establish 95% limits of agreement for visual acuity testretest repeatability. The Bland–Altman plot for the SSII is shown in Fig. 3. The 95% limits of agreement for repeatability for the SSII

Optometry and Vision Science, Vol. 84, No. 3, March 2007

Evaluation of Visual Acuity Measurement in Young Visually-Normal Adults—McClenaghan et al.

FIGURE 1. Mean SSII log MAR visual acuity over both visits as a function of mean ETDRS log MAR visual acuity over both visits. The solid line indicates the best-fit line to the data by principal axis regression. The dashed line indicates the unit-slope line corresponding to no difference between the two charts.

221

FIGURE 3. Bland–Altman plot for test–retest repeatability of the SSII. The average visual acuity (log MAR) of the two visits is plotted on the abscissa and the difference between the two visual acuity measurements is plotted on the ordinate. The solid line indicates the mean difference between the two visits and dashed lines indicate the 95% limits of agreement for the difference in visual acuity between visits.

Comparison of Test–Retest Repeatability The difference in measured visual acuity between first and second visits (d) was used as a parameter for comparing the test-retest repeatability of the two VATs. Differences in repeatability between VATs could manifest as a difference in average d, a difference in the variance of d, or a combination of both. There was no significant difference in repeatability between the two VATs (p 0.19; randomization test with Cramér–von Mises W2).

DISCUSSION

FIGURE 2. Bland–Altman plot for a comparison of the SSII and ETDRS charts. The average visual acuity (log MAR) of the two charts is plotted on the abscissa and the difference between the two visual acuity measurements is plotted on the ordinate. The solid line indicates the mean difference between the two charts and dashed lines indicate the 95% limits of agreement for the difference in visual acuity between charts.

were 0.125 and 0.166 log MAR. In comparison, the Bland– Altman plot for the ETDRS is shown in Fig. 4. The 95% limits of agreement for repeatability for the ETDRS charts were 0.121 and 0.132 log MAR. For the SSII, 95% CIs for the limits of agreement had a largest expected range of 0.158 and 0.199, and a smallest expected range of 0.091 and 0.132. For the ETDRS, 95% CIs for the limits of agreement had a largest expected range of 0.150 and 0.161, and a smallest expected range of 0.092 and 0.103.

A small statistically significant difference in mean visual acuity was found between the two VATs. Mean visual acuity was on average 0.02 log MAR better on the SSII than the ETDRS charts. In addition, the 95% confidence interval demonstrated that this difference is unlikely to be more than 0.033 log MAR. If 0.025 log MAR is taken as a difference in acuity noticeable to many individuals,8 then the differences between VATs are most likely negligible, but possibly small and noticeable. By way of comparison, mean differences of up to 0.09 log MAR have been found between common test charts based on the Bailey–Lovie principles.19 These differences could be of importance in situations where a patient is tested on two or more VATs and clinical decisions were necessary based on the measured VAs. The limits of agreement for test–retest repeatability of the SSII were equivalent to a fluctuation of a little over plus or minus one line ( 0.125 to 0.166 log MAR). Upon initial inspection, this fluctuation seemed high. However, similar test–retest repeatability was found with the ETDRS. Further statistical analysis demonstrated no significant difference between the repeatability of the SSII and the ETDRS systems. In addition, these levels of repeatability are typical of commonly used log MAR testing charts.19,20

Optometry and Vision Science, Vol. 84, No. 3, March 2007

222

Evaluation of Visual Acuity Measurement in Young Visually-Normal Adults—McClenaghan et al.

When setting up the SSII, we noted that selection of the “calibrate for short lanes” option within the software led to large and noticeable errors in the rendition of letter sizes smaller than 20/20 equivalent at distances of 10 feet and less. In the current protocol we did not use the “calibrate for short lanes” option. In comparing the SSII and the ETDRS, we found the two systems to be virtually equivalent in accuracy and repeatability in a clinical setting. If anything, we found the SSII to carry an advantage in terms of flexibility and convenience of target presentation.

ACKNOWLEDGEMENTS We thank the California Chapter of the American Academy of Optometry for their support of this project by a Student OD Project Research Grant. Received March 24, 2006; accepted November 13, 2006.

REFERENCES

FIGURE 4. Bland–Altman plot for test–retest repeatability of the ETDRS charts. The average visual acuity (log MAR) of the two visits is plotted on the abscissa and the difference between the two visual acuity measurements is plotted on the ordinate. The solid line indicates the mean difference between the two visits and dashed lines indicate the 95% limits of agreement for the difference in visual acuity between visits.

Putting aside variables which the examiner may easily control (such as most aspects of chart design21), there are not many suggestions for why visual acuity measurement variability is so high. Some possibilities are too few letters per line on available charts,21 small differences in legibility of letters between lines on different chart versions,22 memorization,23 and small uncorrected refractive errors.10,24 More research is needed on the temporal properties of visual acuity. Based on certain characteristics of the SSII unit mentioned in the Introduction, one might expect visual acuities to be more accurate and repeatable with the ETDRS chart. However, it was found that clinically they are virtually equivalent in terms of accuracy and repeatability. The issues of pixilation and anti-aliasing of letters smaller than 20/20 Snellen equivalent, noticeable under close inspection of the SSII, may not be important for the current recognition task at threshold. Letter recognition depends on a narrow band of spatial frequencies, which for small letters tends to be centered on spatial frequencies that are low in terms of cycles per letter.25 Thus, it may be that pixilation of the current LCD display is irrelevant to the task provided the low spatial frequencies of the letter strokes are adequately represented by the display. The SSII, as configured for the current study, also differed from the ETDRS chart by an unequal number of letters per line (some rows had six rather than five letters), use of a larger and nonstandard letter set, and slight differences in size progression (a 20/12 rather than 20/12.5 equivalent line). The current results show that these small differences were not critical to the measured acuity. However, our results should not be construed to support a return to pre Bailey–Lovie2 chart designs.

1. Lennie P, Van Hemel SB, eds. Visual Impairments: Determining Eligibility for Social Security Benefits. Washington, DC: National Academy Press; 2002. 2. Bailey IL, Lovie JE. New design principles for visual acuity letter charts. Am J Optom Physiol Opt 1976;53:740–5. 3. Ferris FL 3rd, Kassoff A, Bresnick GH, Bailey I. New visual acuity charts for clinical research. Am J Ophthalmol 1982;94:91–6. 4. Ferris FL 3rd, Freidlin V, Kassoff A, Green SB, Milton RC. Relative letter and position difficulty on visual acuity charts from the Early Treatment Diabetic Retinopathy Study. Am J Ophthalmol 1993; 116:735–40. 5. Meszaros L, Trokel S. Integrated system solves many acuity testing problems: visual assessment device allows for fingertip control, superior vision evaluation of patients. Ophthalmol Times 2004; March 1; 41. Available at: http://www.ophthalmologytimes.com/ ophthalmologytimes/article/articleDetail.jsp?id 88657. Accessed December 28, 2006. 6. Khanani AM, Brown SM, Xu KT. Normal values for a clinical test of letter-recognition contrast thresholds. J Cataract Refract Surg 2004; 30:2377–82. 7. Kraemer HC, Thiemann S. How Many Subjects? Statistical Power Analysis in Research. Newbury Park, CA: Sage Publications; 1987. 8. Miller AD, Kris MJ, Griffiths AC. Effect of small focal errors on vision. Optom Vis Sci 1997;74:521–6. 9. Brown B, Lovie-Kitchin J. Repeated visual acuity measurement: establishing the patient’s own criterion for change. Optom Vis Sci 1993;70:45–53. 10. Rosser DA, Murdoch IE, Cousens SN. The effect of optical defocus on the test-retest variability of visual acuity measurements. Invest Ophthalmol Vis Sci 2004;45:1076–9. 11. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement [revised]. Lancet 1986;1:307–10. Available at: http://www-users.york.ac.uk/ mb55/ meas/ba.htm. Accessed December 28, 2006. 12. Carkeet A. Modeling logMAR visual acuity scores: effects of termination rules and alternative forced-choice options. Optom Vis Sci 2001;78:529–38. 13. Siegel S, Castellan NJ Jr. Nonparametric Statistics for the Behavioral Sciences, 2nd ed. Boston: McGraw–Hill; 1988. 14. Brown MB, Forsythe AB. Robust tests for the equality of variances. J Am Stat Assoc 1974;69:364–67. 15. Sokal RR, Rohlf FJ. Biometry: The Principles and Practice of Statistics in Biological Research, 2nd ed. San Francisco: Freeman; 1981. 16. Stephens MA. Tests based on EDF statistics. In: D’Agostino RB,

Optometry and Vision Science, Vol. 84, No. 3, March 2007

Evaluation of Visual Acuity Measurement in Young Visually-Normal Adults—McClenaghan et al.

17. 18. 19.

20.

21. 22.

Stephens MA, eds. Goodness-of-Fit Techniques. New York: M. Dekker; 1986:97–194. Edgington ES. Randomization Tests, 3rd ed. New York: M. Dekker; 1995. Manly BFJ. Randomization and Monte Carlo Methods in Biology. London: Chapman and Hall; 1991. Hazel CA, Elliott DB. The dependency of logMAR visual acuity measurements on chart design and scoring rule. Optom Vis Sci 2002; 79:788–92. Lovie-Kitchin JE, Brown B. Repeatability and intercorrelations of standard vision tests as a function of age. Optom Vis Sci 2000;77: 412–20. Raasch TW, Bailey IL, Bullimore MA. Repeatability of visual acuity measurement. Optom Vis Sci 1998;75:342–8. McMonnies CW, Ho A. Letter legibility and chart equivalence. Ophthalmic Physiol Opt 2000;20:142–52.

223

23. McMonnies CW. Chart memory and visual acuity measurement. Clin Exp Optom 2001;84:26–34. 24. Carkeet A, Lee L, Kerr JR, Keung MM. The slope of the psychometric function for Bailey-Lovie letter charts: defocus effects and implications for modeling letter-by-letter scores. Optom Vis Sci 2001;78: 113–21. 25. Majaj NJ, Pelli DG, Kurshan P, Palomares M. The role of spatial frequency channels in letter identification. Vision Res 2002;42: 1165–84.

Optometry and Vision Science, Vol. 84, No. 3, March 2007

Lawrence R. Stark Southern California College of Optometry 2575 Yorba Linda Blvd Fullerton, California 92831 e-mail: lstark@scco.edu

ARTICLE

Relationship between contrast sensitivity and spherical aberration Comparison of 7 contrast sensitivity tests with natural and artificial pupils in healthy eyes Kim W. van Gaalen, MSc, Nomdo M. Jansonius, MD, PhD, Steven A. Koopmans, MD, PhD, Thom Terwee, MSc, Aart C. Kooijman, PhD

PURPOSE: To find a contrast sensitivity test that can be used clinically to evaluate interventions aimed at minimizing spherical aberration and determine the circumstances under which these tests should be performed. SETTING: Laboratory of Experimental Ophthalmology, University of Groningen, Groningen, The Netherlands. METHODS: Contrast sensitivity tests were performed using 2 experimental designs. Design 1 was with a natural pupil under mesopic and photopic conditions. Design 2 was with a 5.0 mm artificial pupil after cycloplegia under photopic conditions only. Two computerized tests (vertical sine-modulated gratings [VSG] and Holladay circular sine-modulated patterns [HACSS]) and 5 chart tests (Pelli-Robson, acuity-measuring letter charts at low contrast [2.5% and 10%], VectorVision, and edge contrast sensitivity) were used. Spherical aberration was assessed with a Hartmann-Shack wavefront analyzer. RESULTS: Forty-nine healthy subjects aged 20 to 35 years (n Z 24) and 55 to 70 years (n Z 25) participated. Design 2 showed a significant relationship between contrast sensitivity and spherical aberration with the HACSS at 3 cycles per degree (cpd) (P Z .03) and 6 cpd (P Z .01) and with the VSG at 6 cpd (P Z .01). Design 1 yielded no significant relationships. CONCLUSIONS: Using an artificial pupil, a relationship between contrast sensitivity and spherical aberration was established with the VSG and HACSS tests but not with the chart tests. No test showed a relationship using natural pupils under either lighting condition. Chart tests are unsuitable for uncovering contrast sensitivity differences related to differences in spherical aberration, as typically found in healthy phakic eyes. J Cataract Refract Surg 2009; 35:47–56 Q 2009 ASCRS and ESCRS

In recent years, cataract surgery has consisted of replacing the cataractous lens with a spherical intraocular lens (IOL). The young human lens, however, is anything but spherical and actually improves the optics of the eye by compensating for the spherical aberrations of the cornea. Hence, optical performance after cataract surgery can be less than perfect. In an attempt to further improve the optical performance of the pseudophakic eye, IOLs with optical properties more similar to those of the clear young human lens have been designed. Several studies1–11 have found improved optical performance after the cataractous lens is replaced with a so-called aspherical Q 2009 ASCRS and ESCRS Published by Elsevier Inc.

IOL compared with the performance with a spherical IOL. The results of cataract surgery with implantation of aspherical IOLs that aim to minimize spherical aberrations can be evaluated using a Hartmann-Shack wavefront sensor and dynamic skiascopy. These techniques measure the optical aberrations of the eye precisely and objectively. The principles associated with these techniques were explained by Liang et al.12 and Cervino et al.13 The advantage of these methods is the objectivity, and the disadvantages are the high cost and that the apparatus does not measure visual performance directly. 0886-3350/09/$dsee front matter

doi:10.1016/j.jcrs.2008.09.016 ' " ( %# # " $ & "# $( ' " ( ( # & " "% "( " ! "# %# ( $ " %# # ' $ %$ ! " ## !(" $ ) # & " " $# " # "&

CONTRAST SENSITIVITY AND SPHERICAL ABERRATION

In a clinical setting, contrast sensitivity testing with chart tests could be a simple and inexpensive method of directly evaluating visual performance after cataract surgery. Many contrast sensitivity chart tests are commercially available, including edge contrast sensitivity tests and letter contrast sensitivity tests such as the Pelli-Robson14 and the Mars,15 which display single-size optotypes with decreasing contrast (Kooijman AC, et al. IOVS 1994; 35:ARVO Abstract 550). A different approach to contrast sensitivity testing with letter charts is the use of charts that present optotypes at a fixed low contrast with a range of sizes similar to that of visual acuity charts.16 Traditionally, contrast sensitivity is measured with gratings at a range of spatial frequencies.17 These gratings can be generated by a computer and displayed on a monitor or presented with chart tests such as the VectorVision.18 Computer-driven tests allow continuous controllable contrast levels that enable precise assessment of the threshold at a wide range of spatial frequencies, which results in a complete contrast sensitivity function. The disadvantages of computerdriven tests are the long testing time and the relatively high cost of the equipment. Recent studies that attempted to evaluate visual performance after the implantation of aspherical IOLs with contrast sensitivity measurements yielded conflicting results.1–11,19–22 Therefore, we thought that a systematic inventory of the ability of contrast sensitivity tests to uncover the effects of differences in spherical aberration was long overdue. In this study, we selected 2 computerized tests and 5 chart tests. We assessed the Submitted: July 14, 2008. Final revision submitted: September 16, 2008. Accepted: September 21, 2008. From the Laboratory of Experimental Ophthalmology (van Gaalen, Kooijman) and Department of Ophthalmology (Jansonius, Koopmans), University Medical Center Groningen, University of Groningen, and Advanced Medical Optics Groningen BV (Terwee), Groningen, the Netherlands. Dr. Terwee is an employee of Advanced Medical Optics Groningen BV, The Netherlands. No other author has a financial or proprietary interest in any material or method mentioned. Presented at the annual meeting of the Association for Research in Vision and Ophthalmology, Fort Lauderdale, Florida, USA, May 2007. Supported by SenterNovem, The Hague, The Netherlands, and Dutch National Grant ISO43081. Corresponding author: K.W. van Gaalen, MSc, Department of Ophthalmology, University Medical Center Groningen, University of Groningen, Postbus 30.001, 9700 RB, Groningen, The Netherlands. E-mail: k.van.gaalen@ohk.umcg.nl.

ability of these tests to show a relationship between contrast sensitivity and spherical aberration, the latter measured with a wavefront sensor. We also explored the conditions under which contrast sensitivity measurements should be performed for this purpose. SUBJECTS AND METHODS Subjects and Wavefront Analysis The study adhered to the tenets of the Declaration of Helsinki and was approved by the Medical Ethical Committee of the University Medical Center Groningen. The study was registered in the ISRCTN register (trial ISCRTN66724598) and in the Dutch trial registers (trial 812). Measurements were obtained from healthy subjects in 2 age groups (20 to 35 years and 55 to 70 years). Before inclusion in the study, subjects gave their written informed consent. Eyes with a refractive error of more than G2.00 diopters (D) spherical equivalent were excluded, as were eyes with a cylindrical correction of more than 1.50 D or with a cylindrical correction that deviated more than 20 degrees from the horizontal or vertical axis. Only the dominant eye was tested. The best corrected visual acuity (BCVA) in that eye had to be at least 0.8 (20/25). Visual acuity after optimal subjective refraction was determined with an Early Treatment Diabetic Retinopathy Study (ETDRS) chart. No subject had a known history of ocular pathology or surgery. To further document the absence of ocular pathology, corneal topography was performed (Orbscan II version 3.12, Bausch & Lomb, Inc.) and a dilated fundus photograph of the papillomacular region was taken and evaluated by an ophthalmologist. Wavefront aberrations were measured with a wavefront analyzer (WASCA version 1.26.3, Asclepion Meditec) in standardized Optical Society of America values (micrometers).23 The Zernike term Z(4,0) was used as a measure of the spherical aberration in the eye. Initially, wavefront aberrations were measured in 35 subjects in each age group to estimate the Gaussian spherical aberration distribution in that group. Next, subjects were selected from each age group to obtain roughly equal numbers of subjects in 4 subgroups ( 2 SD to 1 SD; 1 SD to mean; mean to 1 SD; 1 SD to 2 SD). In other words, the aim was a uniform distribution around the mean spherical aberration. This resulted in 24 younger subjects and 25 older subjects. This selection was performed to improve the observation of a potential effect of spherical aberration on contrast sensitivity. The spherical aberration was measured with a natural pupil (experimental design 1) and with an artificial pupil (experimental design 2). In experimental design 2, contrast sensitivity measurements started 30 minutes after cycloplegia and iridoplegia were obtained with cyclopentolate 1%.

Contrast Sensitivity Tests Contrast sensitivity was tested using 2 computerized tests and 5 chart tests. Contrast sensitivity was measured at several spatial frequencies for both computerized tests and 1 chart test. The order of the tests was randomized. Tests were performed with best spectacle correction in a trial frame. Each test was performed at the recommended viewing distance, and the refractive correction of the subjects was corrected for that viewing distance at the beginning of each test.

J CATARACT REFRACT SURG - VOL 35, JANUARY 2009

' " ( %# # " $ & "# $( ' " ( ( # & " "% "( " ! "# %# ( $ " %# # ' $ %$ ! " ## !(" $ ) # & " " $# " # "&

CONTRAST SENSITIVITY AND SPHERICAL ABERRATION

Figure 1. The contrast sensitivity tests used in the study. A: Vertical sine-modulated gratings. B: Holladay circular sine-modulated patterns. C: Pelli-Robson. D: Early Treatment Diabetic Retinopathy Study 2.5% and 10.0%. E: Groningen Edge Contrast Chart. F: VectorVision.

The first computerized test, vertical sine-modulated gratings (VSG) (VSG 2/3 version 4.02, Cambridge Research Systems), generates vertical sine-modulated gratings (3 cycles per degree [cpd] and 6 cpd) on a cathode ray tube (Barco CCID7351B, Video & Communications n.v.) (Figure 1, A). This test was viewed at a distance of 2 m. The von Békésy tracking method was used to assess the contrast threshold. In this method, the contrast changes continuously. First, it increases until the subject observes the pattern. On pressing a button, the contrast starts to decrease until the subject can no longer see the pattern. The contrast levels at which the gratings are reported as appearing and disappearing are recorded. The speed of change of contrast was set at 0.3 log/second. Six upper reversals and lower reversals were measured. The first, highest, and lowest values of both the upper and lower reversals were excluded. The remaining upper and lower reversals were averaged, resulting in the contrast threshold.24 The grating pattern contrast is expressed in Michelson contrast: Michelson contrast Z ðLmax Lmin Þ=ðLmax þ Lmin Þ

ð1Þ

where Lmax is the maximum luminance of the bright bars and Lmin the minimum luminance of the dark bars. The order in which the spatial frequencies were tested was randomized. Contrast sensitivity was defined as the inverse of the measured contrast threshold. The other computerized test was the Holladay automated contrast sensitivity testing system (HACSS) (M&S Technologies) (Figure 1, B). The circular sine-modulated patterns with spatial frequencies of 3 cpd and 6 cpd were used. The test begins with 50% contrast, starting at the highest spatial frequency. The subject indicates whether the displayed stimulus is a circular pattern or a blank disk. Throughout the test, several blank disks are shown at the same mean luminance level to check reliability. After each correct answer, the contrast of the stimulus decreases in steps of 0.3 log units. Near the threshold, contrast decreases by 0.1 log units. When an incorrect answer is given, contrast increases by 0.3 log units (after the second incorrect answer by 0.2 log units) and decreases by 0.1 log unit until the next incorrect response. The contrast threshold corresponds to the lowest contrast level at which the subject can correctly identify 2 of 3 circular patterns. The contrast sensitivity is based on

Michelson contrast (equation 1). This test was performed at the recommended viewing distance of 4 m. The Pelli-Robson contrast sensitivity chart test (Clement Clarke International Ltd.) displays Sloan letters of constant size (Figure 1, C). The chart consists of 8 rows, each with 2 triplets. Contrast decreases from 1 triplet to the next in steps of 0.15 log units. Contrast sensitivity is expressed as the inverse of the Weber contrast: Weber contrast Z ðLmax Lmin Þ=Lmax

ð2Þ

where Lmax is the luminance of the background and Lmin the luminance of the letters. The test was performed at a viewing distance of 3 m, which corresponds to a spatial frequency of approximately 3 cpd.25 The maximum log contrast sensitivity (logCS) that can be tested is 2.20. When a subject makes 2 mistakes within 1 triplet,14 the test is terminated and then scored by letter.26 Two other chart contrast sensitivity tests were ETDRS-like optotype charts developed in the Laboratory of Experimental Ophthalmology, University of Groningen, with low contrast (2.5% and 10.0%) optotypes in Weber contrast (equation 2) (Figure 1, D). These charts measure resolutiondthe smallest optotype that can be seendat a given fixed low contrast. The visual acuity rate (VAR) values were used for statistical evaluation.27,28 These tests are performed at a viewing distance of 1 m. When the subject makes the first mistake, the test is terminated and the last correct answer noted. Contrast sensitivity is scored by letter with a maximum VAR of 85 and a minimum VAR of 35. Edge contrast sensitivity was measured using a test developed in the Laboratory of Experimental Ophthalmology. This test, the Groningen Edge Contrast Chart (GECKO) Figure 1, E), presents 16 circular targets (diameter 74.0 mm) divided into halves with different reflection values (Kooijman AC, et al. IOVS 1994; 35:ARVO Abstract 550). The contrast between the halves decreases in steps of 0.15 log units. The orientation of the separation line has a tilt C15 degrees, 0 degrees, or 15 degrees in the vertical or the horizontal direction. The subject has to indicate the darkest half and the orientation of the separation line. When the subject makes the first mistake, the test is terminated and the last correct answer noted. The minimum contrast for this chart is 0.01 in Michelson contrast (equation 1), corresponding to

J CATARACT REFRACT SURG - VOL 35, JANUARY 2009

' " ( %# # " $ & "# $( ' " ( ( # & " "% "( " ! "# %# ( $ " %# # ' $ %$ ! " ## !(" $ ) # & " " $# " # "&

CONTRAST SENSITIVITY AND SPHERICAL ABERRATION

a maximum logCS of 2.0. This test was performed at a viewing distance of 3 m. The VectorVision contrast sensitivity chart test (VectorVision) (Figure 1, F) presents targets with sine-wave gratings at various spatial frequencies (3, 6, 12, and 18 cpd). These targets are presented on a double row of targets; 1 of the 2 vertically aligned targets is blank and the other, modulated. The subject has to indicate in which target the grating is present. When the subject makes a mistake, the testing of that particular spatial frequency is terminated and the last correct answer noted. The contrast, expressed in Michelson contrast (equation 1), decreases in steps of 0.15 log units. The minimum contrast for this chart corresponds to a logCS of 2.3. This test was performed at a viewing distance of 2.5 m.

Experimental Designs Experimental Design 1: Measurements with Natural Pupil In this experiment, subjects were asked to perform

2 computerized contrast sensitivity tests and 5 chart tests monocularly with the dominant eye. The order of the tests was randomized. Contrast sensitivity tests were performed at the optimum refractive state for their viewing distance under mesopic conditions (3 candelas [cd]/m2) and photopic conditions (85 or 160 cd/m2) without iridoplegia or cycloplegia. The mesopic condition was achieved by placing a neutral density filter in front of the eye. For the photopic condition, the GECKO, VectorVision, VSG, and HACSS were performed at a mean luminance of 85 cd/m2 and the Pelli-Robson and low-contrast ETDRS at a luminance of 160 cd/m2 of the white background. The luminance of the targets and the background was measured with a Minolta CS-100A chroma meter (Minolta Camera Co. Ltd.). A digital photo of the eye, with a ruler beneath the eye, was taken with both luminance conditions to measure the apparent pupil size (approximately 12% larger than the physical pupil size29 as used in the WASCA software). Spherical aberration was measured and calculated for the individually measured photopic and mesopic physical pupil sizes using the WASCA software. Unless otherwise stated, all pupil sizes reported here refer to the apparent pupil size.

Experimental Design 2: Measurements with Artificial Pupil In this experiment, subjects were asked to perform

the 2 computerized tests, the VSG and the HACSS at 3 cpd and 6 cpd, and the Pelli-Robson test monocularly with the dominant eye. The remaining tests were discarded because of the results in experimental design 1 (see Results). The order of the tests was randomized. Cycloplegia and iridoplegia in the tested eye were obtained with 2 drops of cyclopentolate 1%. After 30 minutes, the pupil size was measured and the spherical aberration and contrast sensitivity measurements were begun. Contrast sensitivity tests were performed with optimum refraction for the viewing distance under photopic conditions (see experimental design 1). Measurements were performed with a 5.0 mm artificial pupil in the trial frame in front of the eye. The spherical aberration measurements were normalized to this 5.0 mm apparent pupil size and to the size of the maximally dilated pupil.

Statistical Analysis The main outcome variable for all tests, except the lowcontrast ETDRS-like optotype charts, was the logCS value. The low-contrast ETDRS-like optotype chart results were

expressed in VAR.28 Statistical analyses were performed using the contrast sensitivity values and spherical aberration values of each subject separately. The nonparametric Mann-Whitney U test for independent samples was used to calculate the difference between the measured contrast sensitivity and spherical aberration values obtained from both age groups. The nonparametric Wilcoxon signed rank test for dependent samples was used to calculate the difference between the measured contrast sensitivity obtained under both lighting conditions. The relationship between the contrast sensitivity values and the absolute spherical aberration values (aiming at a linear relationship) was calculated with linear regression analysis. To confirm a normal distribution of the residuals, a nonparametric Kolmogorov-Smirnov Z test was performed. The means are presented with their standard deviation. A P value less than 0.05 was considered statistically significant.

RESULTS Measurements were obtained from 49 healthy subjects in 2 age groups. The mean age in the 20 to 35 year group (younger group; n Z 24) was 25 years and in the 55 to 70 year group (older group; n Z 25), 60 years. All 49 subjects participated in experimental design 1. In experimental design 2, 37 subjects agreed to participate; 17 were in the younger group (mean age 26 years) and 20, in the older group (mean age 62 years). Pupil Size and Spherical Aberration The mean BCVA was 107 VAR (range 103 to 113 VAR; 0.1 to 0.3 logMAR; 20/16 to 20/10 Snellen) in the younger group and 107 VAR (range 100 to 113 VAR; 0.0 to 0.3 logMAR; 20/20 to 20/10 Snellen) in the older group. There was a statistically significant difference in mean natural pupil size between the mesopic condition and photopic condition in the younger group (4.78 G 0.60 mm and 4.03 G 0.64 mm, respectively; P Z .000) and the older group (3.89 G 0.53 mm and 3.21 G 0.53 mm, respectively; P Z .000). The difference between the younger and older groups was statistically significant under mesopic conditions and photopic conditions (both P Z .000). There was no difference in mean spherical aberration with a natural pupil between the younger group and older group under mesopic conditions (0.016 G 0.072 mm and 0.012 G 0.045 mm, respectively; P Z .650) or under photopic conditions (0.002 G 0.036 mm and 0.007 G 0.035 mm, respectively; P Z .749). However, when spherical aberration was measured with an artificial pupil, the older group had statistically significantly greater mean spherical aberration (0.054 G 0.050 mm) than the younger group (0.024 G 0.043 mm (P Z .045). Figure 2 shows spherical aberration as a function of age measured during experimental design 2 after cycloplegia and iridoplegia with a 5.0 mm pupil.

J CATARACT REFRACT SURG - VOL 35, JANUARY 2009

' " ( %# # " $ & "# $( ' " ( ( # & " "% "( " ! "# %# ( $ " %# # ' $ %$ ! " ## !(" $ ) # & " " $# " # "&

CONTRAST SENSITIVITY AND SPHERICAL ABERRATION

Figure 3 shows the relationship between spherical aberration and pupil size with the natural pupil under mesopic and photopic conditions and after cycloplegia and iridoplegia. As expected, spherical aberration increased with increased pupil size. Spherical aberration values showed a large variation for pupils larger than 5.0 mm. Contrast Sensitivity Measurements There was a ceiling effect in the GECKO, the 10.0% ETDRS-like optotype chart, and the VectorVision test at 3 and 6 cpd because some subjects were able to read the most difficult targets in the charts (Table 1). In contrast, under mesopic conditions, a floor effect appeared in the 2.5% ETDRS-like optotype chart because some subjects were unable to perceive any optotype (Table 1). Therefore, the results obtained using these tests were not evaluated further. Table 2 shows the mean contrast sensitivity values measured with a natural pupil at the optimum refractive state of the eye in the remaining tests under mesopic conditions and photopic conditions. Due to technical problems, 3 subjects in both age groups did not perform the VSG test. Under mesopic conditions, the mean contrast sensitivity was statistically significantly lower than under photopic conditions in both age groups in all tests shown in Table 2 (P!.015). Furthermore, there was a statistically significant age-related decline in contrast sensitivity for HACSS measurements under both lighting conditions (P!.05). Table 3 shows the mean contrast sensitivity values measured with an artificial pupil of 5.0 mm. The difference in contrast sensitivity between the younger group and older group was statistically significant for all tests except the Pelli-Robson. Contrast Sensitivity Versus Spherical Aberration Figure 4 shows the relationship between contrast sensitivity assessed after cycloplegia and iridoplegia with a 5.0 mm artificial pupil and the absolute value of the corresponding spherical aberration (n Z 37).

Figure 2. Spherical aberration as a function of age measured after cycloplegia and iridoplegia with a 5.0 mm artificial pupil (SA Z spherical aberration).

A significant slope in the regression line was present in the results obtained with the HACSS at 3 cpd (P Z .03) and at 6 cpd (P Z .01) and with the VSG at 6 cpd (P Z .01). With a natural pupil, there was no significant relationship between contrast sensitivity and spherical aberration for any test. The wavefront data used in the natural pupil situation were based on the actual pupil size of each individual subject. When these data were normalized to a 5.0 mm pupil in all subjects, no statistically significant relationships were found except the HACSS under mesopic conditions at 3 cpd (r2 Z 0.12; P Z .033). DISCUSSION In this study, we assessed the relationship between contrast sensitivity and spherical aberration using different contrast sensitivity tests with artificial and natural pupils. The experiments were performed in individuals who were emmetropic or near emmetropic and who were selected for their spherical aberration. Spherical aberration increases with age. With a 5.0 mm artificial pupil, a significant relationship between Figure 3. Spherical aberration as a function of pupil size. A: Natural pupil; the solid circles represent the mesopic condition and the open circles, the photopic condition. B: Artificial pupil; the solid circles represent spherical aberration measurements calculated for maximum dilated pupils and the open circles, spherical aberration calculations for an apparent 5.0 mm pupil (SA Z spherical aberration).

J CATARACT REFRACT SURG - VOL 35, JANUARY 2009

' " ( %# # " $ & "# $( ' " ( ( # & " "% "( " ! "# %# ( $ " %# # ' $ %$ ! " ## !(" $ ) # & " " $# " # "&

CONTRAST SENSITIVITY AND SPHERICAL ABERRATION

Table 1. Ceiling and floor effects in contrast sensitivity chart tests. Number of Subjects Mesopic

Photopic

Test

Effect*

Younger (n Z 24)

Older (nZ25)

Younger (n Z 24)

Older (nZ25)

2.5% ETDRS-like optotype chart 10.0% ETDRS-like optotype chart GECKO VectorVision 3 cpd VectorVision 6 cpd

Floor Ceiling Ceiling Ceiling Ceiling

6 0 11 0 3

16 1 4 1 1

0 7 16 5 9

1 3 11 5 3

cpd Z cycles per degree; ETDRS Z Early Treatment Diabetic Retinopathy Study; GECKO Z Groningen Edge Contrast Chart * A ceiling effect means the test is too easy; all responses are correct. A floor effect means the test is too difficult; no correct response is assessed by the test

contrast sensitivity and spherical aberration was found for contrast sensitivity measurements performed with the HACSS at 3 and 6 cpd and the VSG at 6 cpd. No relationship was found between contrast sensitivity and spherical aberration when contrast sensitivity was measured with a natural pupil under mesopic or photopic conditions. In the present study, only primary spherical aberration, represented by the Zernike term Z(4,0), was used. The Z(4,0) term is reported to be the most important higher-order aberration term; other higher-order terms,30,31 including secondary spherical aberration Z(6,0),31 are much smaller. When replacing the absolute Z(4,0) by O(Z(4,0)2 C Z(6,0)2) in our data, the same relationships were found. Similarly, including coma did not improve the relationship between contrast sensitivity and the total amount of aberration. For those reasons, we confined our analyses to Z(4,0).

There are several possible explanations for the absence of a significant relationship in the natural pupil condition. Under the photopic condition, the pupils are small and the spherical aberration in eyes with a small pupil is nearly zero. This small spherical aberration barely affects retinal image quality, thus decreasing any possible relationship between spherical aberration and contrast sensitivity. Because a natural pupil was used, the pupil size, and thus the resulting retinal illumination, varied. Retinal illumination has a strong effect on the shape of the contrast sensitivity function,32,33 especially at lower retinal illuminations. Thus, the variation in the retinal illumination in the test conditions might influence the assessed contrast sensitivity. Under photopic conditions, the luminance of the stimulus and its direct surroundings was between 85 cd/m2 and 160 cd/m2. The pupil diameters of the subjects varied between 2.0 mm and 6.7 mm.

Table 2. Mean logCS with a natural pupil measured with different contrast sensitivity tests by age group. Mean LogCS G SD (Range) Lighting and Test Mesopic HACSS at 3 cpd HACSS at 6 cpd VSG at 3 cpd* VSG at 6 cpd† Pelli-Robson Photopic HACSS at 3 cpd HACSS at 6 cpd VSG at 3 cpd* VSG at 6 cpd† Pelli-Robson

Younger (n Z 24)

Older (n Z 25)

P Value

1.90 G 0.17 (1.70–2.40) 1.80 G 0.16 (1.60–2.40) 2.14 G 0.20 (1.77–2.51) 1.68 G 0.24 (1.08–2.06) 1.49 G 0.11 (1.35–1.65)

1.69 G 0.13 (1.49–2.00) 1.62 G 0.17 (1.40–2.10) 2.01 G 0.25 (1.70–2.74) 1.75 G 0.27 (1.40–2.40) 1.39 G 0.12 (1.20–1.55)

.00 .00 .04 .67 .01

1.94 G 0.14 (1.75–2.40) 1.96 G 0.11 (1.80–2.40) 2.35 G 0.18 (1.90–2.76) 2.10 G 0.19 (1.67–2.35) 1.66 G 0.06 (1.50–1.75)

1.81 G 0.13 (1.50–2.00) 1.87 G 0.12 (1.60–2.10) 2.42 G 0.27 (1.72–3.03) 2.20 G 0.27 (1.72–2.72) 1.67 G 0.07 (1.50–1.80)

.00 .02 .32 .28 .85

cpd Z cycles per degree; HACSS Z Holladay circular sine-modulated patterns; LogCS Z log contrast sensitivity; VSG Z vertical sine-modulated gratings * In the younger group, n Z 21; in the older group, n Z 22 † In the younger group, n Z 17; in the older group, n Z 20

J CATARACT REFRACT SURG - VOL 35, JANUARY 2009

' " ( %# # " $ & "# $( ' " ( ( # & " "% "( " ! "# %# ( $ " %# # ' $ %$ ! " ## !(" $ ) # & " " $# " # "&

CONTRAST SENSITIVITY AND SPHERICAL ABERRATION

Table 3. Mean logCS with an artificial pupil measured under photopic conditions with different contrast sensitivity tests by age group. Mean LogCS G SD (Range) Test HACSS at 3 cpd HACSS at 6 cpd VSG at 3 cpd VSG at 6 cpd Pelli-Robson

Younger (nZ17)

Older (nZ20)

P Value

1.76 G 0.19 (1.45–2.10) 1.93 G 0.13 (1.70–2.15) 2.31 G 0.13 (2.12–2.64) 2.23 G 0.15 (1.89–2.49) 1.63 G 0.09 (1.40–1.80)

1.61 G 0.27 (1.20–2.40) 1.73 G 0.23 (1.35–2.40) 2.18 G 0.26 (1.69–2.74) 2.01 G 0.24 (1.62–2.51) 1.58 G 0.11 (1.30–1.80)

.02 .00 .04 .00 .05

cpd Z cycles per degree; HACSS Z Holladay circular sine-modulated patterns; LogCS Z log contrast sensitivity; VSG Z vertical sine-modulated gratings

Hence, the resulting retinal illuminations ranged from 85 (2.0/2)2 p Z 267 trolands to 160 (6.7/2)2 p Z 5638 trolands. Under the mesopic condition, the luminance of the contrast tests was 3 cd/m2 and the measured pupil diameter varied between 3.0 mm and 7.0 mm. The resulting retinal illuminations ranged from 3 (3/2)2 p Z 21 trolands to 3 (7.1/2)2 p

Z 119 trolands. Van Nes et al.32 and Van Nes33 found that contrast sensitivity increases monotonically from 0.0009 trolands to 90 trolands and stabilizes at higher retinal illuminance; the peak of the contrast sensitivity function increased by 0.2 log units between 9 trolands and 90 trolands. This implies that the influence on contrast sensitivity of the variation in retinal illumination

Figure 4. Log contrast sensitivity as a function of absolute spherical aberration measured with different contrast sensitivity tests with a 5.0 mm artificial pupil. Solid circles represent the young group, open circles the old group. A: Pelli-Robson. B: Vertical sine-modulated gratings at 3 cpd. C: Vertical sine-modulated gratings at 6 cpd. D: Holladay circular sine-modulated patterns at 3 cpd. E: Holladay circular sine-modulated patterns at 6 cpd (logCS Z log contrast sensitivity; SA Z spherical aberration).

J CATARACT REFRACT SURG - VOL 35, JANUARY 2009

' " ( %# # " $ & "# $( ' " ( ( # & " "% "( " ! "# %# ( $ " %# # ' $ %$ ! " ## !(" $ ) # & " " $# " # "&

CONTRAST SENSITIVITY AND SPHERICAL ABERRATION

can be ignored under photopic conditions but not under mesopic conditions. Under mesopic conditions, the beneficial effect of a small pupil on contrast sensitivity through lowering the spherical aberration is counteracted by the adverse effect of the lower retinal illumination. In addition, the variation in retinal illumination under mesopic conditions and the resulting variation in contrast sensitivity might add too much variation to produce a statistically significant effect of spherical aberration on contrast sensitivity. Thus, with a natural pupil, the small spherical aberration value for small pupils could explain the absence of a clear relationship between spherical aberration and contrast sensitivity under photopic conditions. The dependence of contrast sensitivity on retinal illumination could explain this absence under mesopic conditions. By also measuring contrast sensitivity with an artificial pupil in front of a dilated pupil, we were able to combine a larger pupil size and thus, in general, a larger spherical aberration with a high and constant retinal illumination. This condition eliminates the influence of retinal illumination on contrast sensitivity, increasing the influence of spherical aberration on contrast sensitivity variation. In this study, a significant relationship between contrast sensitivity and spherical aberration was measured in 2 of the 3 contrast sensitivity tests. Another phenomenon that could reduce the influence of spherical aberration on contrast sensitivity is the Stiles-Crawford effect, which describes the directional sensitivity of the retina as a difference in sensitivity between light that enters the eye through the center of the pupil as opposed to through the periphery.34 Peripheral rays, whose refraction is the most influenced by spherical aberration, are less effective in stimulating the retina than the central rays. The spherical aberration causes a widened distribution of peripheral light rays around the ideal image of a point source at the retina, but its influence on the perceived image is decreased by the Stiles-Crawford effect. Spherical aberration, measured in this study using a wavefront analyzer, is not influenced by the StilesCrawford effect. Olsen35 found that the Stiles-Crawford effect reduced the distance between the effective focus and paraxial focus in pupils larger than 4.0 mm, thereby minimizing the effect of spherical aberration, which could explain the moderate relationship between contrast sensitivity and spherical aberration in the present study. In this study, both age groups were plotted together in Figure 4 to establish a relationship between contrast sensitivity and spherical aberration. Therefore, age-related changes in contrast sensitivity due to causes other than a change in spherical aberration could be a confounder in this analysis. Both optical factors36–39 and neural changes40–44 have been reported to

contribute to age-related changes in contrast sensitivity. Controversy exists regarding the primary cause of this loss in visual performance. Age-related measurements have been performed in young, middleaged, and older subjects. No significant difference between the young and the middle-aged groups was found; however, the older subjects had significantly lower contrast sensitivity.36,40,45–47 Several studies have compared the visual performance of phakic eyes and pseudophakic eyes. The visual performance of subjects with a spherical IOL was comparable to that of age-related phakic subjects but worse than that of younger subjects.48,49 Furthermore, implantation of an aspherical IOL resulted in a higher visual performance than implantation of a spherical IOL.1–11 These findings suggest that the lens is the primary cause of loss in visual performance with age. In the present study, we were unable to distinguish optical from neural factors. However, this does not explain the absence of a relationship between contrast sensitivity and spherical aberration in our study. To conclude, in this study the influence of spherical aberration on contrast sensitivity in phakic subjects with clear lenses could only be established with computer tests and by using cycloplegia and an artificial pupil. Chart contrast sensitivity tests are not suitable. Unfortunately, computer tests are difficult to perform in a clinical setting due to the long testing times and high cost. Therefore, contrast sensitivity testing is not an easy-to-apply tool for the assessment of spherical aberration and the changes in it. REFERENCES 1. Bellucci R, Scialdone A, Buratto L, Morselli S, Chierego C, Criscuoli A, Moretti G, Piers P. Visual acuity and contrast sensitivity comparison between Tecnis and AcrySof SA60AT intraocular lenses: a multicenter randomized study. J Cataract Refract Surg 2005; 31:712–717; errata, 1857 2. Denoyer A, Le Lez M-L, Majzoub S, Pisella P-J. Quality of vision after cataract surgery after Tecnis Z9000 intraocular lens implantation: effect of contrast sensitivity and wavefront aberration improvements on the quality of daily vision. J Cataract Refract Surg 2007; 33:210–216 3. Holladay JT, Piers PA, Koranyi G, van der MM, Norrby NES. A new intraocular lens design to reduce spherical aberration of pseudophakic eyes. J Refract Surg 2002; 18:683–691 4. Mester U, Dillinger P, Anterist N. Impact of a modified optic design on visual function: clinical comparative study. J Cataract Refract Surg 2003; 29:652–660 5. Packer M, Fine IH, Hoffman RS, Piers PA. Prospective randomized trial of an anterior surface modified prolate intraocular lens. J Refract Surg 2002; 18:692–696 6. Packer M, Fine IH, Hoffman RS, Piers PA. Improved functional vision with a modified prolate intraocular lens. J Cataract Refract Surg 2004; 30:986–992 7. Piers PA, Fernandez EJ, Manzanera S, Norrby S, Artal P. Adaptive optics simulation of intraocular lenses with modified spherical aberration. Invest Ophthalmol Vis Sci 2004; 45:4601–4610.

J CATARACT REFRACT SURG - VOL 35, JANUARY 2009

' " ( %# # " $ & "# $( ' " ( ( # & " "% "( " ! "# %# ( $ " %# # ' $ %$ ! " ## !(" $ ) # & " " $# " # "&

CONTRAST SENSITIVITY AND SPHERICAL ABERRATION

10.

11.

12.

13.

14. 15.

16.

17.

18.

19.

20.

21.

22.

23.

Available at: http://www.iovs.org/cgi/reprint/45/12/4601. Accessed October 2, 2008 Kershner RM. Retinal image contrast and functional visual performance with aspheric, silicone, and acrylic intraocular lenses; prospective evaluation. J Cataract Refract Surg 2003; 29:1684–1694 Tzelikis PF, Akaishi L, Trindade FC, Boteon JE. Ocular aberrations and contrast sensitivity after cataract surgery with AcrySof IQ intraocular lens implantation; clinical comparative study. J Cataract Refract Surg 2007; 33:1918–1924 Sandoval HP, Fernández de Castro LE, Vroman DT, Solomon KD. Comparison of visual outcomes, photopic contrast sensitivity, wavefront analysis, and patient satisfaction following cataract extraction and IOL implantation: aspheric vs spherical acrylic lenses. Eye 2007, E-pub ahead of print Bellucci R, Morselli S. Optimizing higher-order aberrations with intraocular lens technology. Curr Opin Ophthalmol 2007; 18:67–73 Liang J, Grimm B, Goelz S, Bille JF. Objective measurement of wave aberrations of the human eye with the use of a Hartmann-Shack wave-front sensor. J Opt Soc Am A 1994; 11:1949–1957 Cerviño A, Hosking SL, Montés-Micó R. Comparison of higher order aberrations measured by NIDEK OPD-scan dynamic skiascopy and Zeiss WASCA Hartmann-Shack aberrometers. J Refract Surg 2008; 24(8):790–796 Pelli DG, Robson JG, Wilkins AJ. The design of a new letter chart for measuring contrast sensitivity. Clin Vis Sci 1987/88; 2:187–199 Arditi A. Improving the design of the letter contrast sensitivity test. Invest Ophthalmol Vis Sci 2005; 46:2225–2229. Available at: http://www.iovs.org/cgi/reprint/46/6/2225. Accessed October 2, 2008 Brown B, Lovie-Kitchin JE. High and low contrast acuity and clinical contrast sensitivity tested in a normal population. Optom Vis Sci 1989; 66:467–473 Campbell FW, Robson JG. Application of Fourier analysis to the visibility of gratings. J Physiol 1968; 197:551–566. Available at: http://jp.physoc.org/cgi/reprint/197/3/551. Accessed October 4, 2008 Pomerance GN, Evans DW. Test-retest reliability of the CSV1000 contrast test and its relationship to glaucoma therapy. Invest Ophthalmol Vis Sci 1994; 35:3357–3361. Available at: http://www.iovs.org/cgi/reprint/35/9/3357. Accessed October 3, 2008 Kasper T, Bühren J, Kohnen T. Visual performance of aspherical and spherical intraocular lenses: intraindividual comparison of visual acuity, contrast sensitivity, and higher-order aberrations. J Cataract Refract Surg 2006; 32:2022–2029 Muñoz G, Albarrán-Diego C, Montés-Micó R, Rodrı́guezGalietero A, Alió JL. Spherical aberration and contrast sensitivity after cataract surgery with the Tecnis Z9000 intraocular lens. J Cataract Refract Surg 2006; 32:1320–1327 Johansson B, Sundelin S, Wikberg-Matsson A, Unsbo P, Behndig A. Visual and optical performance of the Akreos Adapt Advanced Optics and Tecnis Z9000 intraocular lenses; Swedish multicenter study. J Cataract Refract Surg 2007; 33: 1565–1572 Kurz S, Krummenauer F, Thieme H, Dick HB. Contrast sensitivity after implantation of a spherical versus an aspherical intraocular lens in biaxial microincision cataract surgery. J Cataract Refract Surg 2007; 33:393–400 Thibos LN, Applegate RA, Schwiegerling JT, Webb R. Standards for reporting the optical aberrations of eyes; VSIA Standards Taskforce Members. J Refract Surg 2002; 18:S652–S660

24. Nio YK, Jansonius NM, Lamers P, Mager A, Zeinstra J, Kooijman AC. Influence of the rate of contrast change on the quality of contrast sensitivity assessment: a comparison of three psychophysical methods. Ophthalmic Physiol Opt 2005; 25:18–26 25. Maaranen T, Mäntyjärvi M. Contrast sensitivity in patients recovered from central serous chorioretinopathy. Int Ophthalmol 1999; 23:31–35 26. Elliott DB, Bullimore MA, Bailey IL. Improving the reliability of the Pelli-Robson contrast sensitivity test. Clin Vis Sci 1991; 6:471–475 27. Bailey IL, Lovie JE. New design principles for visual acuity letter charts. Am J Optom Physiol Opt 1976; 53:740–745 28. Colenbrander MC. Visual acuity, visual field and physical ability. Ophthalmologica 1975; 171:100–108 29. Kooijman AC. Light distribution on the retina of a wide-angle theoretical eye. J Opt Soc Am 1983; 73:1544–1550 30. Porter J, Guirao A, Cox IG, Williams DR. Monochromatic aberrations of the human eye in a large population. J Opt Soc Am A Opt Image Sci Vis 2001; 18:1793–1803 31. Salmon TO, van de Pol C. Normal-eye Zernike coefficients and root-mean-square wavefront errors. J Refract Surg 2006; 32:2064–2074 32. van Nes FL, Koenderink JJ, Nas H, Bouman MA. Spatiotemporal modulation transfer in the human eye. J Opt Soc Am 1967; 57:1082–1088 33. van Nes FL. Experimental studies in spatiotemporal contrast transfer by the human eye. Doctoral Thesis, University of Utrecht, Utrecht, The Netherlands, 1968 34. Stiles WS, Crawford BH. The luminous efficiency of rays entering the pupil at different points. Proc R Soc B 1933; 112:428–450 35. Olsen T. On the Stiles-Crawford effect and ocular imagery. Acta Ophthalmol (Copenh) 1993; 71:85–88 36. Nio YK, Jansonius NM, Fidler V, Geraghty E, Norrby S, Kooijman AC. Age-related changes of defocus-specific contrast sensitivity in healthy subjects. Ophthalmic Physiol Opt 2000; 20:323–334 37. Hemenger RP. Intraocular light scatter in normal vision loss with age. Appl Opt 1984; 23:1972–1974 38. Liang J, Williams DR, Miller DT. Supernormal vision and highresolution retinal imaging through adaptive optics. J Opt Soc Am A 1997; 14:2884–2892 39. Sturr JF, Church KL, Taub HA. Temporal summation functions for detection of sine-wave gratings in young and older adults. Vision Res 1988; 28:1247–1253 40. Morrison JD, McGrath C. Assessment of the optical contributions to the age-related deterioration in vision. Q J Exp Physiol 1985; 70:249–269. Available at: http://ep.physoc.org/cgi/reprint/70/2/249. Accessed October 4, 2008 41. Elliott DB. Contrast sensitivity decline with ageing: a neural or optical phenomenon? Ophthalmic Physiol Opt 1987; 7:415–419 42. Jay JL, Mammo RB, Allan D. Effect of age on visual acuity after cataract extraction. Br J Ophthalmol 1987; 71:112–115. Available at: http://www.pubmedcentral.nih.gov/picrender.fcgi? artid=1041100&;blobtype=pdf. Accessed October 4, 2008 43. Owsley C, Gardner T, Sekuler R, Lieberman H. Role of the crystalline lens in the spatial vision loss of the elderly. Invest Ophthalmol Vis Sci 1985; 26:1165–1170. Available at: http://www. iovs.org/cgi/reprint/26/8/1165. Accessed October 4, 2008 44. Weale RA. Senile changes in visual acuity. Trans Ophthalmol Soc U K 1975; 95:36–38 45. McGrath C, Morrison JD. The effects of age on spatial frequency perception in human subjects. Q J Exp Physiol 1981; 66:253–261. Available at: http://ep.physoc.org/cgi/reprint/66/3/253. Accessed October 4, 2008

J CATARACT REFRACT SURG - VOL 35, JANUARY 2009

' " ( %# # " $ & "# $( ' " ( ( # & " "% "( " ! "# %# ( $ " %# # ' $ %$ ! " ## !(" $ ) # & " " $# " # "&

CONTRAST SENSITIVITY AND SPHERICAL ABERRATION

46. Derefeldt G, Lennerstrand G, Lundh B. Age variations in normal human contrast sensitivity. Acta Ophthalmol (Copenh) 1979; 57:679–690 47. Ross JE, Clarke DD, Bron AJ. Effect of age on contrast sensitivity function: uniocular and binocular findings. Br J Ophthalmol 1985; 69:51–56. Available at: http://www.pubmedcentral.nih. gov/picrender.fcgi?artidZ1040522&;blobtypeZpdf. Accessed October 4, 2008

48. Guirao A, Redondo M, Geraghty E, Piers P, Norrby S, Artal P. Corneal optical aberrations and retinal image quality in patients in whom monofocal intraocular lenses were implanted. Arch Ophthalmol 2002; 120:1143–1151 49. Navarro R, Ferro M, Artal P, Miranda I. Modulation transfer functions of eyes implanted with intraocular lenses. Appl Opt 1993; 32:6359–6367

J CATARACT REFRACT SURG - VOL 35, JANUARY 2009

' " ( %# # " $ & "# $( ' " ( ( # & " "% "( " ! "# %# ( $ " %# # ' $ %$ ! " ## !(" $ ) # & " " $# " # "&

ORIGINAL INVESTIGATION

Validation of an Automated Early Treatment Diabetic Retinopathy Study Low-contrast Letter Acuity Test Yi Pang, OD, PhD,1* Lauren Sparschu, OD,1 Elyse Nylin, BA,1 and Jingyun Wang, PhD2

SIGNIFICANCE: Automated low-contrast letter acuity (LCLA) has several advantages: consistent luminance, reduced chance of individuals memorizing test letters, and convenient and accurate visual acuity reporting functions. Although automated LCLA might report slightly worse acuity than Sloan LCLA chart, considering its advantages, it may be a viable alternative to Sloan LCLA chart in clinical practice and research. PURPOSE: The purpose of this study was to determine the repeatability of an automated LCLA measurement and its agreement with the Sloan LCLA chart test in normal participants and reduced-vision participants. METHODS: Adult participants (n = 49) were measured with both automated Early Treatment Diabetic Retinopathy Study and Sloan LCLA tests, including normal and reduced-vision groups. Low-contrast letter acuity at two contrast levels (2.5 and 10%) was measured at 3 m in a random sequence with both LCLA tests. To test repeatability, participants were retested 1 week later. Repeatability of the two tests between two visits and agreement between automated and Sloan LCLA tests were evaluated using 95% limits of agreement. RESULTS: In terms of the 95% limits of agreement, the repeatability of both tests was as follows: automated LCLA at 2.5%, ±0.26; automated LCLA at 10%, ±0.22; Sloan LCLA at 2.5%, ±0.23, and Sloan LCLA at 10%, ±0.16. The agreement of the two tests was as follows: ±0.19 at 2.5% and ±0.24 at 10%. The automated LCLA at 2.5 and 10% levels was generally reported one-half to one logMAR line lower than Sloan LCLA (mean differences, −0.04 at 2.5% and −0.13 at 10%; paired t test, P < .05). CONCLUSIONS: The automated LCLA test shows fairly good test-retest repeatability at both 2.5 and 10% contrast levels. The agreement between the automated and the Sloan low-contrast letter acuity tests was comparable with test-retest agreement. Although the automated LCLA test reports slightly worse acuity than the Sloan LCLA test, it could be an appropriate alternative to the Sloan LCLA test.

Author Afﬁliations: 1 Illinois College of Optometry, Chicago, Illinois 2 Salus University Pennsylvania College of Optometry, Elkins Park, Pennsylvania *ypang@ico.edu

Low-contrast letter acuity, as a visual function test, has been identified as an important component in the profile of many types of patients.1–4 For instance, low-contrast letter acuity is widely used as a benchmark of visual dysfunction for patients with multiple sclerosis.1 According to an increasing body of evidence, in these patients, low-contrast letter acuity is a physiologically meaningful test because decreased low-contrast letter acuity scores have been correlated with retinal thinning in optical coherence tomography imaging, the MRI lesion volume, and reduced responses in multifocal electroretinography.1 Furthermore, low-contrast letter acuity is more sensitive to some diseases than is high-contrast visual acuity.5 Reduced low-contrast letter acuity and vision-specific quality of life are evident many years after acute optic neuritis, even when high-contrast visual acuity has recovered.5 The Sloan low-contrast letter acuity chart test (Precision Vision, LaSalle, IL) is the criterion standard to measure low-contrast letter acuity, which has excellent interrater reliability, with high intraclass correlation across all contrast levels.6 With recent advances in technology, computerized tests have been used to measure visual function. If test distance, luminance, and contrast of the test screen are carefully calibrated and external glare is limited, computerized equipment can generate similar results compared with criterion-standard charts. Several commercially www.optvissci.com

available computer-based contrast sensitivity tests have become available. The Freiburg Visual Acuity and Contrast Test is widely used.7 Kollbaum et al.8 validated an iPad test of letter contrast sensitivity; however, this test did not decrease letter size to measure lowcontrast letter acuity. Unfortunately, there have been few studies to validate repeatability of computerized low-contrast letter acuity tests. Implementation of a low-contrast letter acuity test on an electronic platform has several advantages: (1) specific contrast levels, (2) randomization of target presentation, and (3) automatic scoring for printing, which facilitates documentation and communication, thereby saving precious clinic time. The commercially available automated low-contrast letter acuity test (M&S Technologies, Inc., Niles, IL) is one of the new computerized tests. To overcome the common issue of variable monitor screen brightness in computerized tests, this automated test includes self-calibration of luminance. System calibration of this automated test is set in meters at virtually any distance and is adjustable and precise to within 1 cm. The system is calibrated for both distance-to-participant and pixels per inch so that optotypes precisely follow ANSI Z80.21-2010 (R2015) and SO 8596:2009 regarding size, spacing between optotypes, and spacing between lines. Background luminance is accurately calibrated to 85 cd/m2 for standardized ANSI and ISO testing (M&S

Optom Vis Sci 2020; Vol 97(5)

370

Automated Low-contrast Letter Acuity Test — Pang et al.

Technologies, http://www.mstech-eyes.com/products/detail/ automated-etdrs-defocus-curve). Letter contrast system luminance can be automatically calibrated and ambient room conditions measured with the optional integrated luminance meter. Sloan low-contrast letter acuity charts have a standardized format based on the Early Treatment Diabetic Retinopathy Study visual acuity charts, which is often the standard used in clinical trials; thus, it was chosen as the criterion-standard low-contrast letter acuity test for comparison. The purpose of this study was to determine the repeatability of an Early Treatment Diabetic Retinopathy Study– automated low-contrast letter acuity test on a tablet-computer system and its agreement with the criterion-standard chart-based test, for instance, Sloan low-contrast letter acuity chart, in normally sighted participants and reduced-vision participants.

METHODS This research protocol and the informed consent form were approved by the institutional review board of Illinois College of Optometry, Chicago, IL.

Participants We enrolled adult participants from the greater Chicago area. Demographic characteristics of the participants are shown in Table 1. Informed consent was obtained from all participants. Best-corrected visual acuity was ranged from 20/25 to 20/100 each eye in the participants.

Low-contrast Letter Acuity Tests To validate the commercially available automated low-contrast letter acuity test, we used the Sloan low-contrast letter acuity chart (low-contrast Sloan letter chart; Precision Vision) as the criterion standard. Because 10% contrast is considered the level to be used as adaptation and 2.5% is more sensitive to visual deficits, we

TABLE 1. Demographic characteristics of the participants (n = 49) No. participants (%) Visual acuity 20/25 or better

32 (65.3)

20/30 to 20/100

17 (34.7)

Sex Female

41 (83.7)

Male

8 (16.3)

Race Black

20 (40.8)

Hispanic

8 (16.3)

White

16 (32.7)

Asian

5 (10.2)

Age (y) Range

22.6–91.1

Mean (SD)

46.7 (17.5)

SD = standard deviation.

www.optvissci.com

chose these two contrast levels for repeatability tests. A contrast of 100% is the level of high-contrast visual acuity.

Automated Low-contrast Letter Acuity Test The system includes a laptop computer with a high-resolution 13-inch display and a wireless control tablet for the examiner (Fig. 1) We used the built-in automated contrast sensitivity function system. Fixed contrast with decreasing size system was used to estimate low-contrast letter acuity at 2.5 and 10% contrast levels. The test included 10 Sloan letters. The computer screen was autocalibrated to the luminance level of 85 cd/m2 with a photometer for all tests. To determine the end point of automated lowcontrast letter acuity, two phases were used. (1) Phase I: to determine initial threshold, an Early Treatment Diabetic Retinopathy Study chart (from 20/200 to 20/10) at either 10 or 2.5% contrast level was displayed on the computer screen, and participants were instructed to read the smallest line in which they could read all five letters. The examiner submitted the lowest visual acuity level at which participants read all letters correctly. (2) Phase II: to determine threshold, the Early Treatment Diabetic Retinopathy Study letters at the visual acuity level that examiner submitted were displayed on the computer screen, as well as the remaining smaller letters of the chart; in this phase, participants were tested with a change of 0.1 logMAR acuity and a termination rule of five mistakes. The purpose of this procedure was to ensure a relatively efficient (thus, participants were less fatigued) and at the same time a more accurate result. A blue dot (Fig. 1B) was shown next to the visual acuity level one line below the submitted visual acuity size, and participants were instructed to read the line next to the blue dot. The examiner submitted the correct number of letters participants read and then continued to instruct participants to read letters with decreasing size, in 0.1 logMAR steps. The test stopped when the participants were unable to read any letters correctly or when no smaller lines were available to be tested. The system automatically calculated the logMAR visual acuity using the correct letters that participants read. The end point of five total mistakes has been used in previous studies with visual acuity outcomes.9–11 After completion of the measurement, the test outcomes were displayed on the computer with the following parameters: which eye, test distance, visual acuity letter score, logMAR visual acuity, and Snellen visual acuity equivalent.

Sloan Low-contrast Letter Acuity Charts The charts measure 14 14 inches and were mounted on a retroilluminated cabinet. Sloan low-contrast letter acuity charts were standardized according to the Early Treatment Diabetic Retinopathy Study visual acuity charts with five letters per line. Sloan low-contrast letter acuity per contrast level is given as number of letters identified correctly (maximum of 60 letters).

Procedures All tests were administered monocularly and at 3 m in the same room with habitual refractive correction. Two contrast levels (2.5 and 10%) of one eye from each participant were measured in a random sequence with the two low-contrast letter acuity tests. A random number generator in Excel was utilized to provide the random test sequence of the automated and Sloan measurements. Twenty-four participants were measured with the automated tests first and then the Sloan tests. The remaining 25 participants were measured with the Sloan tests followed by the automated tests. The same random

Optom Vis Sci 2020; Vol 97(5)

371

Automated Low-contrast Letter Acuity Test — Pang et al.

FIGURE 1. Automated ETDRS low-contrast letter acuity measurement viewed by the participants (phase I [A], phase II [B] with a blue dot) and the examiner (phase I [C], phase II [D]). ETDRS = Early Treatment Diabetic Retinopathy Study.

sequence for each participant was used for both test-retest measurements. After 1 week (±3 days), all participants were retested with the same procedure, with the exception of four participants who did not return.

Power Calculation Prior data indicate that the difference in the response of matched pairs is normally distributed with a standard deviation of 0.2. We were planning a study with 35 participants. If the true difference in the mean response of matched pairs was 0.1, we would be able to reject the null hypothesis that this response difference was zero with a power of 0.82. Type I error probability associated with this test was .05. We collected data from a total of 49

participants for comparison of the automated and Sloan tests; for repeatability, we collected data from 45 participants.

Statistical Analysis The repeatability between two administrations of the automated and Sloan low-contrast letter acuity tests, as well as agreement between the two low-contrast letter acuity tests in both groups, was evaluated using the 95% limits of agreement, which corresponds to ±1.96 standard deviation of the differences between administrations or tests. The difference between the scores for each administration or test was calculated for each participant. The distribution of these differences was analyzed by calculating the mean, standard deviation, and the 95% limits of agreement. The breadth of these limits of agreement indicates the repeatability of the test.

TABLE 2. Mean (±SD) scores for the first and second administration of each LCLA test in all participants Repeatability: all participants in both groups 2.5% Contrast level (n = 35) Test

Retest

Mean difference

t Test and P value

10% Contrast level (n = 45) 95% LoA

Test

Retest

Mean difference

t Test and P value

95% LoA

Automated 0.74 (0.19) 0.74 (0.20) −0. 002 (0.13) t = −0.10; P = .92 ±0.26 0.51 (0.34) 0.49 (0.35) 0.02 (0.11) t = 1.23; P = .22 ±0.22 Sloan

0.69 (0.15) 0.65 (0.14)

0.04 (0.12)

t = 2.27; P = .03 ±0.23 0.38 (0.30) 0.38 (0.30) 0.01 (0.08) t = 0.64; P = .52 ±0.16

The mean difference and 95% LoA are also shown. Paired t test was used to compare two administrations of each test. LCLA = low-contrast letter acuity; LoA = limit of agreement; SD = standard deviation.

www.optvissci.com

Optom Vis Sci 2020; Vol 97(5)

372

Automated Low-contrast Letter Acuity Test — Pang et al.

The narrower the limits of agreement, the more repeatable the test.12 A paired t test was used to compare test and retest in both groups at both 2.5 and 10% contrast levels. Bonferroni correction was applied because low-contrast letter acuity was tested at two contrast levels; thus, P < .03 indicates statistical significance.

RESULTS Forty-nine participants across a range of acuities were tested. For retest, four participants did not return. Table 1 shows the basic characteristics of these participants. Note that 10 participants with poor high-contrast visual acuity were not able to identify 2.5% contrast for both tests.

Table 2 shows results from all participants. In terms of the 95% limits of agreement, the repeatability of both tests was as follows: automated low-contrast letter acuity at 2.5%, ±0.26; automated low-contrast letter acuity at 10%, ±0.22; Sloan low-contrast letter acuity at 2.5%, ±0.23; and Sloan low-contrast letter acuity at 10%, ±0.16. 1. Repeatability of automated low-contrast letter acuity test (Figs. 2A, B): at both 2.5 and 10% contrast levels, the automated low-contrast letter acuity of the retest was not statistically different from that of the first administration (Table 2). 2. Repeatability of the Sloan low-contrast letter acuity test (Figs. 2C, D): at both 2.5 and 10% contrast levels, the

FIGURE 2. Repeatability of the Sloan low-contrast letter acuity (Sloan-LCLA) test and the automated low-contrast letter acuity (automated-LCLA) test is shown in Bland-Altman plots at 2.5 and 10% contrast levels. The LCLA difference between the second and the first administrations (second minus first) of each test is plotted against the mean of two tests. (A) Automated-LCLA at 2.5%. (B) Automated-LCLA at 10%. (C) Sloan-LCLA at 2.5%. (D) Sloan-LCLA at 10%. From the top to the middle and the bottom, three lines show the upper 95% LoA, bias, and the lower 95% LoA, respectively.

www.optvissci.com

Optom Vis Sci 2020; Vol 97(5)

373

Automated Low-contrast Letter Acuity Test — Pang et al.

Sloan low-contrast letter acuity of the retest was not statistically different from that of the first administration (Table 2). 3. Agreement of the automated and Sloan low-contrast letter acuity tests: agreement between the automated low-contrast letter acuity and Sloan low-contrast letter acuity at 2.5 and 10% levels is shown in Figs. 3A and B.

The mean (±standard deviation) differences between the automated and Sloan low-contrast letter acuity at 2.5 and 10% contrast levels were −0.04 (±0.10) logMAR and −0.13 (±0.12), respectively, with statistical significance in all participants (paired t test, t = −2.19, P = .005; t = −7.06, P < .001). In terms of the 95% limits of agreement, the agreements between two low-contrast letter acuity tests were ±0.19 at 2.5% contrast level and ±0.24 at 10% contrast level (Table 3). Specifically, Fig. 3 shows that more data were below zero at both contrast levels, which indicates that the automated lowcontrast letter acuity reported one-half to one logMAR line higher values (worse acuity) than the Sloan low-contrast letter acuity.

DISCUSSION The automated low-contrast letter acuity test showed fairly good test-retest repeatability in adult participants across the range of acuities at both 2.5 and 10% contrast levels. In addition, the agreement between the automated and the Sloan low-contrast letter acuity tests was similar to test-retest agreement. According to the 95% limits of agreement range of repeatability, there was one outlier data point on both the Sloan and automated tests at 2.5% level, and it was from a 42-year-old participant with a high-contrast visual acuity of 20/15; the large difference between

test and retest for this participant may be due to his lack of engagement with contrast tests at the second visit. The Sloan low-contrast visual acuity score of the retest was slightly but significantly better than the first test, which might represent a potential learning effect and improved familiarity with the Sloan low-contrast letter acuity chart, but not with the automated low-contrast letter acuity system. According to the 95% limits of agreement, the agreement between automated and Sloan low-contrast letter acuity charts was comparable with the test-retest repeatability of the automated and Sloan low-contrast letter acuity charts, which indicates that the agreement between automated and Sloan low-contrast letter acuity charts was as good as could be expected based on testretest repeatability. There was one outlier; this participant had 20/100+ high-contrast visual acuity. Previous studies have reported that vision tests in reduced-vision participants are likely to be less repeatable than in individuals with normal vision.8,13,14 Compared with the Sloan test, the automated low-contrast letter acuity test in this study often reported worse acuity at both 2.5 and 10% contrast levels. One factor could be the luminance of the test. The automated low-contrast letter acuity was always calibrated at 85 cd/m2. On the other hand, the built-in luminance of the Sloan test was often higher than 85 cd/m2; we found that it averaged 108 to 128 cd/m2. Unfortunately, we were not able to adjust the luminance of the Sloan test because of the built-in system. It is unclear why the built-in luminance of the Sloan test has larger variance. Our findings are opposite to the contrast sensitivity test results of Kollbaum et al.8 In their study, both iPad and Freiburg computerized tests yielded better contrast sensitivity function than did the Pelli-Robson test.8 The difference may be due to the nature of low-contrast letter acuity tests, with different study conditions and different tests. Comparing low-contrast letter acuity with ordinary low-contrast chart testing, limits of agreement in our study were similar to those

FIGURE 3. Bland-Altman plots to demonstrate agreement between Sloan low-contrast letter acuity (Sloan-LCLA) and the automated low-contrast letter acuity (automated-LCLA) at 2.5% (A) and 10% (B) levels. The difference between the average scores (i.e., Sloan-LCLA minus automated-LCLA) for the two tests is plotted against the mean of two tests. From the top to the middle and the bottom, three lines show the upper 95% LoA, the mean difference, and the lower 95% LoA, respectively.

www.optvissci.com

Optom Vis Sci 2020; Vol 97(5)

374

Automated Low-contrast Letter Acuity Test — Pang et al.

TABLE 3. Mean (±SD) scores for the two kinds of LCLA tests in all participants Agreement between two tests in the all participants 2.5% Contrast level (n = 39) Automated

Sloan

Mean difference

10% Contrast level (n = 49)

t Test and P value

95% LoA Automated

0.75 (0.20) 0.71 (0.17) −0. 04 (0.10) t = −2.195; P = .005*

±0.19

Sloan

Mean difference

t Test and P value

0.51 (0.33) 0.38 (0.30) −0.13 (0.12) t = −7.06; P < .001*

95% LoA ±0.24

The mean difference and 95% LoA are also shown. Paired t test was used to compare two tests. *Statistical significance. LCLA = low-contrast letter acuity; LoA = limit of agreement; SD = standard deviation.

in previous studies. Kollbaum et al.8 investigated 20 normal-vision and 20 low-vision participants; their limits of agreement in normalvision participants were ±0.19, ±0.19, and ±0.15 for iPad, PelliRobson, and Freiburg, respectively. They reported limits of agreement for low-vision participants as ±0.24, ±0.23, and ±0.21, respectively, for the three tests.8 Dougherty et al.15 studied 37 participants and reported that limits of agreement were ±0.20 for the Mars test and ±0.20 for the Pelli-Robson test. Balcer et al.6 measured the Sloan letter acuity at contrast levels of 100, 5, 1.25, and 0.6% on individuals with normal visual acuity as well as individuals with multiple sclerosis. They reported the interrater agreement ICC between 0.86 and 0.95 at each contrast level; however, they did not study limits of agreement.6 We have previously reported that individuals with amblyopia associated with myopic anisometropia had clinically and statistically significantly reduced contrast sensitivity at the middle and higher spatial frequencies.16 Although a contrast sensitivity function test can thoroughly measure individuals' contrast sensitivity, the length of the procedure poses an obstacle for application in routine clinical care. Meanwhile, low-contrast acuity not only detects vision loss, which could be missed by high-contrast visual acuity measurement, but also measures contrast sensitivity because the decrease in letter size incorporates testing of different spatial frequencies. In addition, the automated low-contrast letter acuity is a much easier and quicker test compared with contrast sensitivity function measurement; thus, automated low-contrast letter acuity may result in broader application in both clinical care and clinical trials.

Limitations (1) Participants with poorer high-contrast visual acuity (20/30 to 20/100) had difficulty reading 2.5% contrast level (measurement range, 20/10 to 20/200) such that fewer participants could provide the test scores. Therefore, caution must be used in generalizing our findings to individuals whose high-contrast visual acuity is worse than 20/100. (2) In some studies, low-contrast letter acuity tests were conducted binocularly (both eyes open), as this approach integrates possibly relevant binocular summation/inhibition

ARTICLE INFORMATION Submitted: February 13, 2019 Accepted: January 25, 2020 Funding/Support: National Eye Institute (EY026664; to JW). Conflict of Interest Disclosure: This study was supported by M&S Technologies (M&S Technologies provide research equipment and compensation to study subjects) and by grants from the National Eye Institute (EY026664; JW) and The Pennsylvania Lions Sight Conservation and Eye Research Foundation Grant (JW).

www.optvissci.com

effects,17 providing a measure of overall visual function closer to the “real-life” situation than does monocular testing. Our study tested low-contrast letter acuity monocularly. A future study is warranted to measure repeatability of binocular low-contrast letter acuity. (3) Age relationship with tests was not investigated in this study because of the limited number of participants in different age categories. Although this sample had no significant difference in age, the older participants often had poorer vision.

Significance The automated low-contrast letter acuity has several advantages: consistent luminance, reduced chance of individuals memorizing test letters, and convenient and accurate visual acuity reporting functions. Although the automated low-contrast letter acuity might report a higher logMAR value (worse acuity) than the Sloan chart, considering its advantages, the automated test may be a viable alternative to the Sloan low-contrast letter acuity chart in both clinical practice and research. Low-contrast letter acuity, being potentially more sensitive than high-contrast visual acuity, is not very commonly used in clinical practice. It has been recommended that clinical trials adopt a low-contrast letter acuity test as an outcome measure.1–4 Thus, an automated low-contrast letter acuity test may facilitate low-contrast letter acuity application.

CONCLUSIONS The automated low-contrast letter acuity test showed fairly good test-retest repeatability in participants at both 2.5 and 10% contrast levels. In addition, the agreement between the automated and the Sloan low-contrast letter acuity tests was comparable with test-retest agreement. Although the automated low-contrast letter acuity test reported slightly worse acuity than the Sloan lowcontrast letter acuity test, it could be an appropriate alternative to the Sloan low-contrast letter acuity test.

Author Contributions: Conceptualization: YP, JW; Data Curation: YP, LS, EN; Formal Analysis: YP, JW; Funding Acquisition: YP, JW; Investigation: YP, JW; Methodology: YP, LS, EN, JW; Project Administration: YP, EN; Resources: YP, EN; Software: YP; Supervision: YP; Validation: YP, JW; Visualization: YP; Writing – Original Draft: YP, JW; Writing – Review & Editing: YP, LS, EN, JW.

Outcome Measure for Multiple Sclerosis. Mult Scler 2017;23:734–47. 2. Elliott DB, Situ P. Visual Acuity versus Letter Contrast Sensitivity in Early Cataract. Vision Res 1998;38: 2047–52.

REFERENCES

3. Loeffler M, Wise JS, Gans M. Contrast Sensitivity Letter Charts as a Test of Visual Function in Amblyopia. J Pediatr Ophthalmol Strabismus 1990;27: 28–31.

1. Balcer LJ, Raynowska J, Nolan R, et al. Validity of Low-contrast Letter Acuity as a Visual Performance

4. Stavrou EP, Wood JM. Letter Contrast Sensitivity Changes in Early Diabetic Retinopathy. Clin Exp Optom 2003;86:152–6.

Optom Vis Sci 2020; Vol 97(5)

375

Automated Low-contrast Letter Acuity Test — Pang et al. 5. Cole SR, Beck RW, Moke PS, et al. The National Eye Institute Visual Function Questionnaire: Experience of the ONTT. Optic Neuritis Treatment Trial. Invest Ophthalmol Vis Sci 2000;41:1017–21. 6. Balcer LJ, Baier ML, Pelak VS, et al. New Low-contrast Vision Charts: Reliability and Test Characteristics in Patients with Multiple Sclerosis. Mult Scler 2000;6:163–71.

Eyes with 20/17 or Better Visual Acuity. Optom Vis Sci 2006;83:635–40.

Formulation Trials. Ophthalmic Physiol Opt 1988; 8:397–401.

10. Ravikumar A, Marsack JD, Bedell HE, et al. Change in Visual Acuity Is Well Correlated with Change in Imagequality Metrics for Both Normal and Keratoconic Wavefront Errors. J Vis 2013;13:1–16.

14. Reeves BC, Hill AR, Aspinall PA. The Clinical Significance of Change. Ophthalmic Physiol Opt 1987;7: 441–6.

7. Bach M. The Freiburg Visual Acuity Test—Automatic Measurement of Visual Acuity. Optom Vis Sci 1996;73: 49–53.

11. Ravikumar A, Sarver EJ, Applegate RA. Change in Visual Acuity Is Highly Correlated with Change in Six Image Quality Metrics Independent of Wavefront Error and/or Pupil Diameter. J Vis 2012;12:1–13.

8. Kollbaum PS, Jansen ME, Kollbaum EJ, et al. Validation of an iPad Test of Letter Contrast Sensitivity. Optom Vis Sci 2014;91:291–6.

12. Bland JM, Altman DG. Statistical Methods for Assessing Agreement between Two Methods of Clinical Measurement. Lancet 1986;1:307–10.

9. Applegate RA, Marsack JD, Thibos LN. Metrics of Retinal Image Quality Predict Visual Performance in

13. Elliott DB, Sheridan M. The Use of Accurate Visual Acuity Measurements in Clinical Anti-cataract

www.optvissci.com

15. Dougherty BE, Flom RE, Bullimore MA. An Evaluation of the Mars Letter Contrast Sensitivity Test. Optom Vis Sci 2005;82:970–5. 16. Pang Y, Allen M, Robinson J, et al. Contrast Sensitivity of Amblyopic Eyes in Children with Myopic Anisometropia. Clin Exp Optom 2019;102:57–62. 17. Pineles SL, Birch EE, Talman LS, et al. One Eye or Two: A Comparison of Binocular and Monocular Low-contrast Acuity Testing in Multiple Sclerosis. Am J Ophthalmol 2011;152:133–40.

Optom Vis Sci 2020; Vol 97(5)

376

Ophthalmic and Physiological Optics ISSN 0275-5408

TECHNICAL NOTE

Repeatability of an automated ETDRS contrast threshold measurement Yi Pang1

, Lauren Sparschu1 and Jingyun Wang2

Illinois College of Optometry, Chicago, Illinois, USA, and 2SUNY College of Optometry, New York, New York, USA

Citation information: Pang Y, Sparschu L, & Wang J. Repeatability of an automated ETDRS contrast threshold measurement. Ophthalmic Physiol Opt. 2021. https://doi.org/10.1111/opo.12829

Keywords: contrast sensitivity, repeatability

Abstract

Correspondence: Jingyun Wang E-mail address: jwang@sunyopt.edu

This technical report presents the repeatability of an automated Early Treatment Diabetic Retinopathy Study (ETDRS) contrast threshold (ETDRSCT) test in participants (N = 40) with normal vision as well as in subjects with reduced visual acuity. The automated ETDRS-CT test showed good testretest repeatability between the two administrations in both normal and reduced vision participants. Measurement at the retest yielded 0.05 log higher contrast sensitivity than at the first measurement, which might be due to a learning effect among participants.

Received: 15 September 2020; Accepted: 5 March 2021

Introduction

Participants

With recent advances in technology, computerised tests are commonly used to measure visual function. If test distance, luminance and contrast of the test screen are carefully calibrated and external glare is limited, computerised equipment can generate results similar to gold standard charts.1 Several computer-based automated contrast sensitivity tests have become available commercially, including the Freiburg Visual Acuity and Contrast Test,2 the iPad test of letter contrast sensitivity,3 a smartphone-based Peek Contrast Sensitivity test,4 and the Spaeth/Richman Contrast Sensitivity Test.5 The commercially available automated Early Treatment Diabetic Retinopathy Study contrast threshold (ETDRSCT) test is one of the newer computerised tests. Previously, we validated an EDTRS low-contrast letter acuity test.6 The purpose of this study was to determine the repeatability of an ETDRS-CT test in participants with normal vision as well as in subjects with reduced visual acuity.

Participants were enrolled from an urban eye clinic, the Illinois Eye Institute (Chicago, Illinois, USA), which provides both primary and secondary/tertiary eye care. Informed consent was obtained from all participants. Eligibility inclusion criteria: (1) Normal Group: high contrast best-corrected visual acuity of 6/7.5 or better in each eye. (2) Reduced-vision Group: high contrast bestcorrected visual acuity between 6/9 and 6/30 in at least one eye. All participants were cognitive competent adults.

Methods This research protocol and the informed consent forms were approved by the Institutional Review Board of the Illinois College of Optometry (Chicago, Illinois, USA). The Health Insurance Portability and Accountability Act (1996, USA) rules were followed during this study.

Procedures The automated ETDRS-CT test (both hardware and software, M&S Technologies, mstech-eyes.com) includes a laptop computer with a high-resolution 33cm (13 inch) display and a wireless control tablet for the examiner.6 The system is calibrated for both the distance to the subject and the number of pixels per inch. The computer screen was auto-calibrated to a luminance level of 85 candles/m2 with a photometer for all tests. The size of the test letters was 6/30 and the letters were darker than the background. The contrast started at 10% and decreased by 0.1 log units per step with the lowest tested contrast at 0.4%. Participants were encouraged to read the test letters without a time limit. If the participants could not read the letters at 10% contrast level, then the test

Automated ETDRS contrast threshold

Y Pang et al.

increased the contrast by 0.3 log units until they could read them to obtain the starting level. The same protocol was followed by decreasing contrast at 0.1 log units per step. The endpoint of five total errors has been used in previous studies.7 When the participants were unable to read any letters on a line correctly or when lower contrast was not available, the test was stopped. After completion of the measurement, the test outcomes were displayed on the computer with the following parameters: right or left eye, test distance, contrast threshold score as a percentage, contrast sensitivity (CS) in log equivalent. All automated ETDRS-CT tests were conducted by the same examiner in the same room at 3 m while participants wore their habitual refractive correction. Contrast threshold was measured in the right eye of normal vision participants; in the reduced-vision subjects, the right eye was tested unless visual acuity lay outside the enrolment criteria, in which case the left eye (N = 6) was tested. After 45 ( 15) minutes, all participants were retested with the same procedure by the same examiner. Statistical analysis The contrast score as a percentage was converted to log (1/contrast score) (log(CS)) units. A paired t-test was used to compare test and retest. The repeatability between the two automated ETDRS-CT tests was evaluated with BlandAltman analysis, using the 95% limits of agreement, which corresponds to 1.96 * standard deviation of the differences between administrations.8 The difference between the scores for each administration was calculated for each participant. The distribution of these differences was described by the mean, standard deviation and the 95% limits of agreement (namely, the coefficient of repeatability). The breadth of the 95% limits of agreement indicates the repeatability of the test. The narrower the limits of agreement, the more repeatable the test. The average difference between test and retest indicates the accuracy of the test. The closer the bias to zero, the more accurate the test.8 An independent t-test was used to compare the mean contrast score of the test and retest between the two groups. A Pearson correlation between the first contrast threshold test and habitual visual acuity through the habitual refractive correction was also calculated. Data analysis was performed using R 3.5.0 statistics software, including “blandr” and “irr” packages. (R Foundation for Statistical Computing, R-project.org). Results A total of 40 participants were tested, including 21 in the reduced-vision group and 19 in the normal group. Table 1 shows the basic characteristics of the participants. Over half 2

were African American; 80% of participants were female. Reduced vision resulted from uncorrected refractive error (N = 8), degenerative myopia (N = 3), optic atrophy (N = 3), cystoid macular oedema (N = 1), diabetic retinopathy (N = 1), oculocutaneous albinism (N = 1), wet age-related macular degeneration (N = 1), primary open angle glaucoma (N = 1), Stargardt disease (N = 1) or strabismic amblyopia (N = 1). The time to test one eye varied between 3 and 10 min in this group. The repeatability of the ETDRS-CT test is shown in Figure 1, where the test-retest difference is plotted as a function of the mean of two administrations using the BlandAltman method. The mean (SD) first and second ETDRSCT test results were 1.18 0.46 log(CS) and 1.23 0.44 log(CS), respectively. The average difference between test and retest was 0.05 log(CS), i.e., half a line or 2.5 letters better on retest, with a statistically significant difference (paired t-test: t = 3.27, df = 39, p = 0.002). The 95% limits of agreement between test and retest, or the coefficient of repeatability (CoR), was 0.18 log(CS). Figure 1 shows comparison between the normal and reduced-vision group as a Bland-Altman plot. There was a significant difference between the two groups (Normal: 1.52 0.17, Reduced vision: 0.91 0.43 log(CS), t = 5.97, df = 27, p < 0.001). Additionally, there was a significant correlation between the first ETDRS-CT test result in log units and the habitual visual acuity (0.20 0.25 logMAR) (R = 0.86, t = 10.36, df = 38, p < 0.001). Discussion To the best of our knowledge, this is the first study to investigate the repeatability of the ETDRS-CT test. We found

Table 1. Demographic characteristics of the participants (N = 40) Number of Participants (%) Visual Acuity Normal (6/7.5 or better) Reduced Vision (6/9 to 6/30) Range (logMAR) Mean (Standard Deviation, logMAR) Gender Female Male Race Black Hispanic White Asian Age (years) Range Mean (Standard Deviation)

19 (48) 21 (52) 0 to 0.7 0.2 (0.3) 32 (80) 8 (20) 22 (55) 7 (18) 8 (20) 3 (8) 22.2–75.0 47.6 (13.8)

Automated ETDRS contrast threshold

Y Pang et al.

that this test had good repeatability with 95% limits of agreement of 0.18 in both normal and reduced vision participants. In addition, contrast sensitivity tested with ETDRS-CT correlated significantly with habitual visual acuity. Kollbaum et al. reported that both the iPad and Freiburg computerised tests demonstrated better contrast sensitivity values than the Pelli-Robson chart.3 They compared the iPad, Pelli-Robson and Freiburg tests, finding 95% limits of agreement of 0.24, 0.23 and 0.21, respectively.3 Habtamu et al. reported 95% limits of agreement for the smartphone-based Peek Contrast Sensitivity test of 0.30.4 The 95% limits of agreement in the present study was 0.18, which is slightly better than these previous two investigations. Dougherty et al. reported good repeatability of the Mars Letter Contrast Sensitivity Test (95% LoA of 0.20) and excellent agreement with the Pelli-Robson test (95% LoA of 0.21).9 Their result is comparable to the findings of this investigation, which indicate that the ETDRS-CT test is a relatively precise automated test. Retesting of the ETDRS-CT yielded mean findings 0.05 log(CS) higher (2.5 letters more) than the first test. Similarly, the iPad retest results were 0.04 log(CS) higher than the first test.3 Kollbaum et al. reported that a change of 0.25 log(CS) was clinically meaningful for the iPad test of contrast sensitivity based on their 95% limits of agreement.3 Our findings indicated that a change of 0.20 log(CS) could be regarded as being clinically meaningful for the automated ETDRS-CT test based on the 95% limits of agreement. Previously, we reported that patients with amblyopia associated with myopic anisometropia had significantly reduced contrast sensitivity at the middle and higher spatial frequencies.10 Although a contrast sensitivity function test measures an individual’s contrast sensitivity thoroughly, it

is a long procedure which may present some obstacles in routine clinical application. The automated ETDRS-CT test, which requires minimum effort from the participants and less clinical testing time may have a broader application in both clinical care and experimental trials. Limitations There are at least four limitations: (1) Our study employed monocular testing only. A future study is warranted to measure repeatability of the automated ETDRS-CT test binocularly. (2) Any age relationship with testing was not investigated due to a limited number of participants in different age categories. (3) Due to the size limitation of the computer screen and EDTRS test protocol (5 letters in a row), the largest letter size possible is 6/30. For individuals with visual acuity poorer than this value, a shortened test distance must be used to determine the CS threshold. (4) We did not compare the ETDRS-CT test with the PelliRobson chart because they are not directly comparable based on test distance and letter size. The Pelli-Robson chart is designed to be used at 1 m with letter size of 1/34 (6/204), while the automated ETDRS-CT test was designed to be used at 3 m with letter size of 6/30. Significance The automated ETDRS-CT test has several advantages: consistent luminance, reduced chance of individuals memorising test letters and convenient and accurate contrast threshold reporting functions to avoid human recording errors. Although the automated ETDRS-CT test might report a slightly higher contrast sensitivity value at retest (half a line or 2.5 letters better), this change is not clinically

Figure 1. Repeatability of the automated contrast threshold test shown as a Bland-Altman plot. The mean difference between the second and the first administration (second minus first, in black) of each individual is plotted against the mean of the two tests, while 1.96 SD is plotted in blue. CoR ( 0.18) is marked. The black triangles indicate the normal subjects, while the red circle symbols indicate the reduced vision group participants. The fine dashed lines show the 95% CI for the mean and SD, respectively. One darker symbol indicates repeated data from two individuals.

Automated ETDRS contrast threshold

Y Pang et al.

significant and may due to a learning effect. It is still controversial whether the learning effect in functional tests is significant, with studies reporting a learning effect in automated perimetry11,12 while others reporting no learning effect in pulsar perimetry.13 Considering its advantages, this test may be a viable alternative to the Pelli-Robson chart in both clinical practice and research. Conclusion The automated ETDRS contrast threshold test showed good test-retest repeatability between two administrations in both normal and reduced vision participants. Measurement at the retest yielded 0.05 log(CS) higher contrast sensitivity than at the first measurement, which might be due to a learning effect amongst participants. Funding This study was supported by M&S Technologies (M&S Technologies provide research equipment and compensation to study subjects). Conflict of interest The authors report no conflicts of interest and have no proprietary interest in any of the materials mentioned in this article. Author contributions Yi Pang: Conceptualization (lead); Data curation (equal); Formal analysis (supporting); Funding acquisition (equal); Investigation (equal); Methodology (equal); Project administration (equal); Resources (equal); Software (equal); Supervision (equal); Validation (equal); Visualization (equal); Writing-original draft (supporting); Writing-review & editing (supporting). Lauren Sparschu: Data curation (lead); Investigation (equal); Methodology (equal); Project administration (equal); Validation (equal); Writing-original draft (supporting). Jingyun Wang: Conceptualization (supporting); Formal analysis (lead); Investigation (equal); Methodology (supporting); Software (equal); Visualization (lead); Writing-original draft (lead); Writing-review & editing (lead).

References 1. Pang Y, Sparschu L & Nylin E. Validation of an automatedETDRS near and intermediate visual acuity measurement. Clin Exp Optom 2019; 103: 663–667. 2. Bach M. The Freiburg visual acuity test-automatic measurement of visual acuity. Optom Vis Sci 1996; 73: 49–53. 3. Kollbaum PS, Jansen ME, Kollbaum EJ & Bullimore MA. Validation of an iPad test of letter contrast sensitivity. Optom Vis Sci 2014; 91: 291–296. 4. Habtamu E, Bastawrous A, Bolster NM et al. Development and validation of a smartphone-based contrast sensitivity test. Transl Vis Sci Technol 2019; 8: 13. Available at: https:// tvst.arvojournals.org/article.aspx?articleid=2751399. Accessed April 15, 2021 5. Faria BM, Duman F, Zheng CX et al. Evaluating contrast sensitivity in age-related macular degeneration using a novel computer-based test, the Spaeth/Richman contrast sensitivity test. Retina 2015; 35: 1465–1473. 6. Pang Y, Sparschu L, Nylin E & Wang J. Validation of an automated early treatment diabetic retinopathy study lowcontrast letter acuity test. Optom Vis Sci 2020; 97: 370–376. 7. Ravikumar A, Marsack JD, Bedell HE et al. Change in visual acuity is well correlated with change in image-quality metrics for both normal and Keratoconic Wavefront errors. J Vis 2013; 13: 28. Available at: https://jov.arvojournals.org/ article.aspx?articleid=2193847. Accessed April 15, 2021 8. Bland JM & Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986; 1: 307–310. 9. Dougherty BE, Flom RE & Bullimore MA. An evaluation of the Mars letter contrast sensitivity test. Optom Vis Sci 2005; 82: 970–975. 10. Pang Y, Allen M, Robinson J & Frantz KA. Contrast sensitivity of amblyopic eyes in children with myopic anisometropia. Clin Exp Optom 2019; 102: 57–62. 11. Aydin A, Kocak I, Aykan U et al. The influence of the learning effect on automated perimetry in a Turkish population. J Fr Ophtalmol 2015; 38: 628–632. 12. Wild JM, Dengler-Harles M, Searle AE et al. The influence of the learning effect on automated perimetry in patients with suspected glaucoma. Acta Ophthalmol (Copenh) 1989; 67: 537–545. 13. Salvetat ML, Zeppieri M, Parisi L et al. Learning effect and test-retest variability of pulsar perimetry. J Glaucoma 2013; 22: 230–237.

Report

Measuring Contrast Sensitivity Using the M&S Smart System II versus the Pelli-Robson Chart

Contrast sensitivity (CS) refers to the ability of the visual system to detect differences in luminance (i.e., brightness) between an object and its background.1 Assessment of CS provides valuable information in the early detection and monitoring of ocular diseases, as well as evaluating the impact of therapy.2 The most widely used clinical spatial CS test is the Pelli-Robson chart (Clement Clarke International, Essex, UK).3 Several factors may influence the CS threshold measured. First, although the recommended luminance is 85 candelas/m2 (range, 60e120), maintaining consistent luminance across the entire chart can be difficult. Overhead lighting in most examination rooms illuminates preferentially the top portion of the chart, and decreases nonuniformly toward the lower portion. In addition, patients tested in different examination rooms with different light fixtures may exhibit some variation in threshold measurement. Second, the Pelli-Robson chart fades over time with exposure, with a manufacturer-recommended expiry of 7 years. Variation and inaccuracy may occur when comparing measurements using charts of different ages. Third, the chart has only 2 versions with different triplets of optotypes. Patients may recall letters with frequent use, especially those letters that are found around their threshold. The M&S Smart System II (MSSS-II; M&S Technologies Inc, Niles, IL) includes a computer-generated, letter-based CS test. The luminance of the liquid crystal display (LCD) screen can be adjusted to recommended level of 85 candelas/m2 using the built-in control and can be monitored for any changes using an external light meter or luminance probe. The testable contrast ranges from 0.0 to 2.3 log units (similar to the PelliRobson chart), with each level corresponding to a change of 0.1 log units. Unlike the Pelli-Robson chart, the letters are not arranged in triplets of equal contrast. Instead, a single Sloan letter is displayed randomly in the center of the screen for any given contrast level. The keypad is used by the examiner to access the CS test, to choose randomization options, and to increase or decrease the contrast level of each letter. The system offers several advantages over the Pelli-Robson chart. The test is conducted in a dark room, thereby avoiding issues related to variation in room illumination. The system can be cali-brated for various viewing distances (1.8e6.7 m) and does not require recalibration to account for the lighting environment at each testing distance. Furthermore, presentation of random letters prevents patients from memorizing the letters (Table 1, available at http://aaojournal.org). Currently, there are no published data on the validity or reliability of CS testing using the MSSS-II. We compared the MSSS-II with the PelliRobson chart as a clinical test for measuring CS in a large population. Testing was performed on 134 adults and children (262 eyes). The mean age ( standard deviation) was 19.5 14.9 years (range, 5e69 years; 78 females). There were 66 eyes from 33 visually normal participants (mean age, 29.4 15.7 years; range, 5e56; 23 females; visual acuity, 20/15e20/25) and 196 eyes from 101patients (mean age, 16.3 years; range, 6e69; 55 females; visual acuity, 20/20e20/400). The ophthalmic diagnoses included glaucoma, diabetic retinopathy, macular drusen, retinitis pigmen-tosa, optic neuritis, idiopathic intracranial hypertension, optic glioma, and amblyopia.

Six eyes with visual acuity of 20/200 were excluded. Participants who were not able to read a standard Early Treatment of Diabetic Retinopahty Study (ETDRS) letter chart and those with a prior history of refractive surgery were also excluded. The study was approved by the Research Ethics Board at The Hospital for Sick Children and all protocols adhered to the guidelines of the Declaration of Helsinki. Informed consent was obtained from each participant. Participants were tested using the MSSS-II and the Pelli-Robson chart in random order during monocular viewing. The MSSS-II optotype size of 1.5 logarithm of the minimum angle of resolution at a testing distance of 4 m was chosen to match the visual angle subtended by the letters presented on the Pelli-Robson chart at 1 m, representing a spatial frequency of 1 cycle per degree for both distances. Participants were tested with the Pelli-Robson chart with each letter being scored individually and assigned a score of 0.05 for each correct response.4 For the MSSS-II, participants were asked to name the letter that was displayed in the center of the screen. The experimenter then increased or decreased the contrast level based on the previous response. A single Sloan letter was displayed for each contrast level starting at 100% contrast. Once the participant approached their threshold, as determined by any hesitation in response or error in letter identification, randomly selected Sloan letters were presented 2 more times at the same contrast level and the participant was required to identify 2 of the 3 letters per contrast level correctly before the contrast threshold was finalized. The outcome measure was the agreement between MSSSII and Pelli-Robson. BlandeAltman analysis5 demonstrated that the MSSS-II test and Pelli-Robson chart show comparable CS values for both visually normal participants and patients (Fig 1; available at http:// aaojournal.org). For visually normal participants (n ¼ 66 eyes), mean CS was 1.670.12 log units with MSSS-II, and 1.640.04 log units with the Pelli-Robson chart. For patients (n ¼ 196 eyes), mean CS was 1.440.29 log units with the MSSS-II, and 1.480.28 log units with the Pelli-Robson chart. The mean CS difference detected between the 2 testing methods was minimal: 0.030.12 log units (95% confidence interval of limits of agreement, 0.20 to 0.26 log units) for visually normal participants and 0.040.12 log units (95% confidence interval of limits of agreement, 0.28 to 0.19 log units) for patients. This variation may be due to the minimal differences in the log progression of the 2 charts (0.10 log units between CS levels for MSSS-II and 0.15 log units for Pelli-Robson). The testing time with the MSSS-II system was shorter because participants were screened with only 1 letter at a high contrast level, and the contrast level was decreased imme-diately if they identified the single letter with ease. This differed from using the Pelli-Robson chart, which required the participants to read all 3 letters in the triplet for every contrast level. In conclusion, the close agreement of CS thresholds suggests that the updated version of MSSS-II, when carefully calibrated, can be used as an alternative method to the Pelli-Robson chart in the measurement of CS in a wide variety of ophthalmic conditions, in both adults and children.

Ophthalmology Volume 120, Number 10, October 2013

2160

Ophthalmology Volume 120, Number 10, October 2013

B Patients

0.6 0.4 0.2 0 -0.2 -0.4 -0.6

0.4

0.6

0.8

1.2

1.4

1.6

1.8

Difference in CS measured by MSSS-II and Pelli-Robson tests (log units)

Visually normal participants 0.6

0.4

0.2

-0.2

-0.4

-0.6

0.4

0.6

0.8

1.2

1.4

1.6

1.8

Mean CS measured by MSSS-II and Pelli-Robson tests (log units)

Figure 1. Bland-Altman plots comparing the M&S Smart System II (MSSS-II) and the Pelli-Robson chart for (A) visually normal participants and (B) patients. CS ¼ contrast sensitivity.

Table 1. Features and Speciﬁcations of M&S Smart System II and Pelli-Robson Chart

Feature

M&S Smart System II

Pelli-Robson Chart

Viewing distance CS range (log units) Unit decrement (log units) Source of illumination Setup

Wall mounted at 1e6.7 m 0.0e2.3 0.1 Intrinsic (monitor conﬁguration) Calibration is not challenging, only requiring monitor conﬁguration to set appropriate luminance

Wall mounted at 1 or 3 m 0.0e2.25 (1/O2) 0.15 per triplet Extrinsic (overhead lighting) Calibration may be challenging depending on the conﬁguration of ambient lighting

Durability Expiration Other considerations

Software transferable to new computer/display No expiration Built-in visual acuity and CS tests provide convenience and conﬁguration ﬂexibility

Chart may fade over time requiring replacement 7 years More affordable and portable; requires less space

CS ¼ contrast sensitivity.

References 1

MANOKARAANANTHAN CHANDRAKUMAR, HBSC LINDA COLPA, OC(C)1 Y. ARUN REGINALD, MD, FRCS1 HERBERT C. GOLTZ, PHD1,2 AGNES M.F. WONG, MD, PHD, FRCSC1,2 1

Department of Ophthalmology and Vision Sciences, The Hospital 2 for Sick Children, Toronto, Ontario, Canada, University of Toronto, Toronto, Ontario, Canada

1. Wei H, Sawchyn AK, Myers JS, et al. A clinical method to assess the effect of visual loss on the ability to perform activities of daily living. Br J Ophthalmol 2012;96:735–41. 2. Owsley C. Contrast sensitivity. Ophthalmol Clin North Am 2003;16:171–7. 3. Pelli DG, Robson JG, Wilkins AJ. The design of a new letter chart for measuring contrast sensitivity. Clin Vis Sci 1988;2:187–99. 4. Elliott DB, Whitaker D. Clinical contrast sensitivity chart evaluation. Ophthalmic Physiol Opt 1992;12:275–80. 5. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986;1:307–10.

2161

Normal values for a clinical test of letterrecognition contrast thresholds Arshad M. Khanani, MD, Sandra M. Brown, MD, Ke Tom Xu, PhD Purpose: To investigate the contrast thresholds (CTs) in normal subjects using a high-luminance, letter-recognition task under clinically relevant testing conditions. Setting: Texas Tech University Health Sciences System, Lubbock, Texas, USA. Methods: Sixty normal subjects aged 20 to 49 years with a best corrected visual acuity of 20/20 or better in both eyes participated. M & S Technologies software was used to display black-on-white Sloan letters at contrast levels of 25%, 20%, 15%, 12%, and 10% through 1% in 1% decrements. The effects of age, sex, optotype size, eye dominance, ambient illumination level (bright 625 630 lux; dim 3 lux), and direction of approach to threshold were analyzed using a multivariate, ordinary, least-squares analysis. Results: Age and sex did not influence CTs. Ascending versus descending testing was not statistically significant (P .5). The effects of room illumination and eye dominance were significant (P .01). Significant differences were found between 20/30 and 20/50, 20/30 and 20/70, and 20/50 and 20/70 optotype sizes (P .01 for all comparisons). Conclusions: A commercially available, computer-based test of CTs was easy to administer and apparently easy for inexperienced subjects to perform. The results suggest criteria for detecting visual problems concerned with familiar but complex spatial-image shapes. This information might be used to assess the effects of treatments such as laser refractive surgery on recognition contrast. Further study is warranted. J Cataract Refract Surg 2004; 30:2377–2382  2004 ASCRS and ESCRS

ision quality loss after laser refractive surgery (LRS) has raised awareness of the need for objective measures of vision function other than high-contrast visual acuity. Contrast threshold evaluation (CTE) may prove to be a useful assessment.1–10 Although CTE using sinewave gratings has existed as a laboratory tool for decades, it has not penetrated into regular clinical care, possibly due to the practical difficulties in administration and the inability of doctors to easily correlate test results with patient complaints.7,11 Compared with sine-wave grating, a CTE based on letter recognition12–16 evaluates more functionally relevant aspects of daily vision. A Accepted for publication March 30, 2004. Reprint requests to Sandra Brown, MD, 3601 Fourth Street STOP 7217, Lubbock, Texas 79430-7217, USA. E-mail: sandra.brown@ ttuhsc.edu.  2004 ASCRS and ESCRS Published by Elsevier Inc.

letter-recognition test requires the observer to not only detect the presence of an object(s) but also perceptually separate the spatial contours of 1 object from those belonging to neighboring objects and then to actually identify the object. Contrast thresholds measured with sine-wave gratings are significantly lower (better) than those measured by letter recognition, and this difference increases as letter size decreases.17 Therefore, a patient could detect the orientation of a sine-wave grating yet not recognize a familiar shape of comparable size at the same contrast level. The present study addresses practical issues concerning a spatial-recognition CTE that is commercially available. The specific aims were to investigate the validity of the test (whether the test produces the expected results), assess whether the test was easy to administer and easy for a typical patient to perform, and derive 0886-3350/04/$–see front matter doi:10.1016/j.jcrs.2004.05.027

NORMAL VALUES OF LETTER-RECOGNITION CONTRAST THRESHOLDS

Table 1. Subject demographics. Age (Years)

Number

Male

Female

20–29

30–39

40–49

criteria for judging whether a contrast threshold (CT) value should be considered normal.

Subjects and Methods Institutional Review Board approval was obtained. Sixty normal subjects aged 20 to 49 years were enrolled. Demographic information is given in Table 1. Exclusion criteria were a best corrected visual acuity worse than 20/20 in either eye; a history of eye injury, eye disease, or eye surgery; and lack of familiarity with the Roman alphabet. Patients requiring correction were tested with correction (glasses or contact lenses). A subset of Sloan optotypes considered to have the same difficulty18,19 (ZNHVRKD) was used in a letter-recognition test. The letters were displayed as black against a white background, with 5 letters per line spaced according to Early Treatment Diabetic Retinopathy Study standards and a single-line display. Contrast threshold, the lowest luminance contrast at which the subject could identify at least 3 of 5 letters twice, was used to quantify the letter-recognition performance. Contrast thresholds were determined bidirectionally (ie, ascending from unseen to seen and descending from seen to unseen). Testing was performed in dominant and nondominant eyes in bright-light and low-light conditions. The optotype sizes were always decreased stepwise using 20/70, 20/50, and 20/30. Twenty-four tests were performed per subject. The optotypes were generated by M & S Technologies SmartSystem II 2020, using contrast levels in percentile decrements of 25, 20, 15, 10, 9, 8, 7, 6, 5, 4, 3, 2, and 1. The monitor was a Phillips Professional Series model 107P20 flat screen, aperture grill-display type. It has a 0.25 mm horizontal dot pitch, with the resolution set to 1280 1024. The monitor luminance was matched to the American National Standards Institute (ANSI) reflectance level for white board charts by measuring the emitted light as if it were reflected. The ANSI standard is 85 candelas (cd)/m2. When measured with a Tekroniix J16 digital photometer with a cd/m2 probe, the monitor “reflected” 95 cd/m2 (averaged over its entire area), a close match. This monitor has a color temperature of 5300 Kelvin (tested with a Minolta Color Meter IIIF) and is capable of producing 65 million colors and 255 shades of pure gray (in which the red, green, and blue channels are equal). The luminance values of successive shades of gray were measured 2378

with multiple light meters and with the white channel of a PM2L dark-room color analyzer (Beseler) and were found to be essentially linear. Therefore, using the gray scale, it is possible to divide the range of luminance from none (pure black) to pure white into 100/255 steps (or 0.39% per step). Contrast was defined as the gray-scale pixel density of the optotypes relative to the pixel density of the pure white background. In this system, 1% contrast represents a pixel density of 99% of the white value. The same windowless room was used for all tests. The testing distance was 17 feet, and the optotype dimensions were calibrated for this working distance by the software. Bright-light testing was performed with fluorescent ceiling panel lights on (ambient illumination near the subject of about 630 lux or 59 foot-candles [ft-c]). Low-light testing was performed in complete darkness except for a dim examiner work light placed behind the subject (ambient illumination near the subject of less than 3 lux or 0.3 ft-c). The amount of light falling on the monitor was measured for consistency with a light meter positioned beside the monitor. Typical bright-light readings were 950 – 980 lux (88 91 ft-c), and low-light readings were less than 3 lux. The difference in bright illumination near the subject and near the monitor was due to the location of the fluorescent ceiling panels and was constant throughout the study. Subjects nearing threshold were encouraged to “wait for letters to fade in.” This typically resulted in a 1% to 2% decrease in the scored CT level (ie, better contrast sensitivity) relative to the first level that the patient perceived as difficult. The percentage contrast data set was converted to log10 values. A multivariate, ordinary, least-squares analysis was used to determine which conditions would significantly influence CT given other factors and controlling for intrasubject correlation across different test conditions. The CT was then regressed over subjects’ age, sex, eye dominance (dominant eye versus nondominant eye), ambient illumination (bright light versus low light), direction of approach to threshold (decreasing versus increasing), and optotype size (20/50 and 20/70 versus 20/30). The significant conditions (P .05) were used as stratifying variables to establish categories indicating different combinations of the test settings. The mean of nonsignificant settings was used for each subject in each category. For example, if eye dominance, ambient illumination, and optotype size were significant and direction of approach to threshold was not, the CT mean obtained from ascending and descending approaches was used for a given subject in the category of dominant eye/bright light/optotype size 20/30 and the other 11 possible category combinations. For each category, a distribution was constructed for the 60 subjects using the means of the settings that were not statistically significant in the multivariate analysis. The mean and the upper value of the 95% confidence intervals (CIs) were calculated for each combination.

J CATARACT REFRACT SURG—VOL 30, NOVEMBER 2004

NORMAL VALUES OF LETTER-RECOGNITION CONTRAST THRESHOLDS

Table 2. Means of the distributions of CTs for various conditions. Bright Light

Low Light

Optotype

Dominant Eye

Nondominant Eye

Dominant Eye

Nondominant Eye

20/30

6% ( 1.26)

6% ( 1.24)

6% ( 1.22)

6% ( 1.20)

20/50

3% ( 1.55)

3% ( 1.52)

20/70

2% ( 1.69)

2% ( 1.65)

2% ( 1.66)

2% ( 1.64)

Values are percentage contrast rounded (up or down) to the nearest clinically measurable value; values in parentheses are the log10 of percentage contrast. CT contrast threshold

To validate the appropriateness of this approach, Shapiro-Wilk and Shapiro-Francia W tests for normality assumptions were performed for all distributions.

Results Results were obtained from all subjects recruited for the study. On average, subjects completed the 24measurement test in about 30 minutes. Table 1 shows the age and sex distribution of the 60 subjects. Results of the multivariate analysis showed that age (P .97) and sex (P .38) did not influence the CTs. The differences between dominant eye and nondominant eye, bright-light and low-light ambience, and optotype sizes were statistically significant (all P .01). The direction of approach to threshold was not statistically significant (P .28). Based on the multivariate analysis results, 12 categories were constructed under different combinations of eye dominance (2), ambient illumination (2), and optotype size (3). The Shapiro-Wilk and Shapiro-Francia W tests could not reject the null hypothesis that the distributions of the CTs were normal for all combinations except the combination of dominant eye/low light/ optotype 20/50 at .05. Table 2 shows the mean values under different conditions. Overall, bright-light ambience produced

lower CTs than low-light ambience. Dominant eyes had lower CTs than nondominant eyes, and larger optotypes produced lower CTs than smaller optotypes. Table 3 shows the upper limit of the 95% CIs of the means. For example, the results indicate that for a dominant eye under bright-light ambience with an optotype size of 20/30, a CT of 6% or less can be considered normal. The fact that eye dominance and ambient illumination affected the population result is evident from the log values, but the magnitude is modest. When testing a particular subject, the percentage contrast values for the mean and upper 95% CI for all conditions are identical at a given optotype size with 1 exception (20/30, low light, dominant eye).

Discussion The LRS industry has rekindled interest in practical office CTE,20 a subjective test that differs from highcontrast visual acuity in that it characterizes how well the visual system performs in a complex environment composed of a wide range of target sizes and luminance levels. Contrast threshold evaluation differs from objective tests such as wavefront analysis in that the entire visual pathway is tested, including the optical components of the eye (cornea and lens) and the sensory

Table 3. Upper 95% CI limit of the distributions of CTs for various conditions. Bright Light

Low Light

Optotype

Dominant Eye

Nondominant Eye

Dominant Eye

Nondominant Eye

20/30

6% ( 1.24)

6% ( 1.22)

6% ( 1.20)

7% ( 1.18)

20/50

3% ( 1.54)

3% ( 1.53)

3% ( 1.56)

3% ( 1.50)

20/70

2% ( 1.67)

2% ( 1.63)

2% ( 1.64)

2% ( 1.61)

Values are percentage contrast rounded (up or down) to the nearest clinically measurable value; values in parentheses are the log10 of percentage contrast. CT contrast threshold

J CATARACT REFRACT SURG—VOL 30, NOVEMBER 2004

2379

NORMAL VALUES OF LETTER-RECOGNITION CONTRAST THRESHOLDS

pathways (retina, optic nerve, optic radiations, and visual cortex). Higher-order optical aberrations induced by the corneal ablation that do not significantly degrade the patient’s optical performance would also not affect CTs. Maeda et al.21 report that letter contrast sensitivity could be predicted by corneal topography regularity indices when normal patients and patients with keratoconus who had 20/20 best corrected visual acuity were compared. Rabin and coauthor22,23 believe that smallletter CTE might be a more discriminating test of visual resolution than high-contrast visual acuity. Rabin24 demonstrated that small-letter CTE was sensitive to subtle amounts of optical defocus, making it a potentially valuable quantitative tool for assessing vision quality after refractive surgery. We determined normal letter-recognition CTs using a commercially available vision test that was straightforward to administer and easy for normal subjects to perform, as evidenced by 100% testability and the amount of time required to complete the experimental test sequence of 24 measurements. Measurable differences in the CTs varied in the expected direction for the various letter sizes (Table 2).17,25 High-luminance, low-contrast visual acuity testing using front-illuminated board charts shows no age-related decline until the sixth decade.26 Although our study did not attempt to measure low-contrast visual acuity thresholds, that we found no age-related differences in CT at small optotype sizes is consistent with this research. We believe this is the first study to investigate the difference in CT between dominant and nondominant eyes, which we found to be slight (difference of the means being 0.00 to 0.04 log units better in the dominant eye) but statistically significant across a normal population. This finding requires further investigation. With sine-wave testing, the region of maximum contrast sensitivity is found at low grating frequencies equivalent to optotype sizes of 20/200 to 20/600. However, under typical clinical working distances (distance between the patient and the chart), using this letter-size range would allow only 1 or 2 optotypes per presentation on a computer monitor or would require adjustment of the working distance to less than the typical minimum distance; this may influence low-light pupil diameter, a concern which will be addressed further. For mirrored examination rooms in which the display is at optical infinity, it would be impossible to create 2380

a 20/600 or larger optotype and maintain the standard display requirement of 1 optotype width of surrounding background on a monitor of practical size. Ghaith et al.4 conclude that CTs do not correlate with patients’ self-reported satisfaction after LRS. They used sine-wave grating and low-frequency (20/640) letters. Their results may indicate that low-frequency, letterrecognition CTs are measuring a psychovisual function that has little direct correlation with subjective vision quality after LRS. We think that using smaller optotypes is more representative of visually challenging tasks such as reading road signs at dusk. For example, a 9.0 cm letter (35⁄8 inches) on a street sign, when viewed at 200 feet (about one third of a typical city block), would be roughly equivalent to a 20/20 optotype. All our CT testing was photopic because the monitor luminance was high and remained constant throughout the study. We measured light falling on the computer monitor to ensure that the degree of monitor glare from ambient room lights was consistent. However, contrast testing can be conducted at other levels of background luminance. Using sine-wave gratings on a reflecting chart that was illuminated at an intermediate level, MontésMicó and coauthors9 found a statistically significant decrease in mesopic contrast sensitivity 6 months after laser in situ keratomileusis (LASIK) for grating frequencies of 12 and 18 cycles per degree (cpd) (equivalent to 20/50 and 20/32 optotypes) but not for 1.5, 3, and 6 cpd (equivalent to 20/400, 20/200, and 20/100). Using the SKILL Card26 (a low-luminance, low-contrast test with black letters on a dim gray background), young observers generally lose about 3 lines of visual acuity compared with high-contrast, high-luminance acuity. Intermediate- and low-luminance letter-contrast tests could be genereated by programming the effective monitor luminance to specific reduced levels and may provide additional useful information about CT changes after LRS. For LRS-related testing based on our data, the issue of ambient lighting deserves additional comment. Some commonly used terms to describe light levels have specific meanings in psychovisual testing. Illuminance refers to the amount of light falling on a surface. Luminance in this case refers to the amount of light coming toward the eye from a source; when the source is a board chart, this light is via reflectance from the chart surface. For this reason, when using board charts, illumination, back-

J CATARACT REFRACT SURG—VOL 30, NOVEMBER 2004

NORMAL VALUES OF LETTER-RECOGNITION CONTRAST THRESHOLDS

ground color, working distance, and cleanliness must be standardized so luminance is consistent. For backilluminated charts using light boxes, luminance is determined by the light box, the transmission properties of the chart, and the working distance. Clinically important variation in high-contrast visual acuity can be induced when back-illuminated charts of varying luminance are used; this problem may be exacerbated in patients who have had refractive surgery procedures (S.M.B., personal observation). When considering the entire study population, we found a statistically significant increase in mean CT under low-light illumination (Table 2, parenthetical log values). However, clinically the difference was less than 1% for all categories, which is unlikely to be functionally relevant in everyday life. Since the difference in light reflected off the monitor was probably trivial at a working distance of 17 feet, this increase is likely due to some change in the eye, possibly pupil diameter and/ or the state of retinal adaptation.25 Although we did not use a standardized dark-adaptation routine or measure pupil diameter, low light 20/30 testing was performed after about 10 minutes in the dark, which is sufficient to elicit maximum pupil dilation.27 Pupil size may have a greater effect on CTs in patients after LRS because of light scatter from the blend zone or the unablated cornea.3,7 It is recognized that light-box testing in which different charts have unequal luminance levels may confound inter-test and inter-study comparison, possibly by introducing pupil size as an unrecognized bias7 (particularly with shorter testing distances). Montés-Micó and coauthors9 measured sine-wave CT under photopic luminance and 3 levels of mesopic luminance 6 months after LASIK; a 5-minute adaptation period was performed before each measurement. The mean pupil diameters varied as expected; 3.5 mm for photopic luminance and 6.1 mm at the lowest luminance. High-frequency grating contrast was reduced under mesopic conditions, which the authors attributed to the greater degree of higher-order optical aberrations present at larger pupil diameters. We suggest that bright-light and low-light testing be performed on LRS patients who are followed longitudinally with CTE. Test distance and ambient illumination levels must be standardized, with adequate time allowed for retinal dark adaptation and pupil dilation before low-light measurements. Since the order of ap-

proach to threshold was not significant, clinical test time can be reduced by 50% compared to the experimental protocol by testing in the customary direction (seen to not seen) only; for presumably normal subjects, testing can begin at 10% contrast or lower, which will also shorten the duration of the examination. This study used only young, normal subjects and was not designed to determine the sensitivity or specificity of the test for disease or postoperative states. Anecdotally, 1 individual who had had LRS was inadvertently enrolled and tested. During the testing sequence, it was quickly evident that the results were atypical and the subject was questioned, at which point a history of LASIK was obtained. (The subject did not consider LASIK to be “eye surgery.”) In summary, we have determined normal values for a commercially available, computer-based test of high-luminance, small-optotype, letter-recognition CTs. The test was easy to understand and perform by inexperienced subjects. Compared with the detection-only task for sinusoidal grating CTE, letter-recognition CTE requires detection plus perceptual separation and then recognition. It may more comprehensively represent the function of contrast sensitivity in everyday life. Currently, no information on changes in letter-recognition CTs after LRS and their relationship to patient visual function or satisfaction exists. Such studies are indicated, and we hope they will be conducted with the introduction of this facile method of contrast-sensitivity testing.

References 1. Montés-Micó R, Charman WN. Mesopic contrast sensitivity function after excimer laser photorefractive keratectomy. J Refract Surg 2002; 18:9–13 2. Montés-Micó R, Charman WN. Choice of spatial frequency for contrast sensitivity evaluation after corneal refractive surgery. J Refract Surg 2001; 17:646–651 3. Holladay JT, Dudeja DR, Chang J. Functional vision and corneal changes after laser in situ keratomileusis determined by contrast sensitivity, glare testing, and corneal topography. J Cataract Refract Surg 1999; 25: 663–669 4. Ghaith AA, Daniel J, Stulting RD, et al. Contrast sensitivity and glare disability after radial keratotomy and photorefractive keratectomy. Arch Ophthalmol 1998; 116:12–18 5. O’Day D. Comment on article by Ghaith AA, Daniel J, Stulting RD, et al. Contrast sensitivity and glare dis-

J CATARACT REFRACT SURG—VOL 30, NOVEMBER 2004

2381

NORMAL VALUES OF LETTER-RECOGNITION CONTRAST THRESHOLDS

10.

11. 12.

13.

14.

15. 16.

17.

18.

ability after radial keratotomy and photorefractive keratectomy. J Refract Surg 1998; 14:361 Pérez-Santonja JJ, Sakla HF, Alió JL. Contrast sensitivity after laser in situ keratomileusis. J Cataract Refract Surg 1998; 24:183–189 Boxer Wachler BS, Durrie DS, Assil KK, Krueger RR. Role of clearance and treatment zones in contrast sensitivity: significance in refractive surgery. J Cataract Refract Surg 1999; 25:16–23 Chan JWW, Edwards MH, Woo GC, Woo VCP. Contrast sensitivity after laser in situ keratomileusis; one-year follow-up. J Cataract Refract Surg 2002; 28:1774–1779 Montés-Micó R, España E, Menezo JL. Mesopic contrast sensitivity function after laser in situ keratomileusis. J Refract Surg 2003; 19:353–356 Mutyala S, McDonald MB, Scheinblum KA, et al. Contrast sensitivity evaluation after laser in situ keratomileusis. Ophthalmology 2000; 107:1864–1867 Boxer Wachler BS, Krueger RR. Normalized contrast sensitivity values. J Refract Surg 1998; 14:463–466 Mäntyjärvi M, Laitinen T. Normal values for the PelliRobson contrast sensitivity test. J Cataract Refract Surg 2001; 27:261–266 Pelli DG, Robson JG, Wilkins AJ. The design of a new letter chart for measuring contrast sensitivity. Clin Vis Sci 1988; 2:187–199 Regan D, Giaschi DE, Fresco BB. Measurement of glare susceptibility using low-contrast letter charts. Optom Vis Sci 1993; 70:969–975 Regan D, Neima D. Low-contrast letter charts as a test of visual function. Ophthalmology 1983; 90:1192–1200 Zhang L, Pelli DG, Robson JG. The effects of luminance, distance, and defocus on contrast sensitivity as measured by the Pelli-Robson chart. Abstract 58. Invest Ophthalmol Vis Sci 1989; 30:406 Ginsburg AP. Visual Information Processing Based on Spatial Filters Constrained by Biological Data. WrightPatterson Air Force Base, Air Force Aerospace Medical Research Laboratory, 1978 Camparini M, Cassinari P, Ferrigno L, Macaluso C. ETDRS-Fast: implementing psychophysical adaptive methods to standardized visual acuity measurement with

2382

19.

20. 21. 22. 23. 24. 25. 26.

27.

ETDRS charts. Invest Ophthalmol Vis Sci 2001; 42: 1226–1231 Ferris FL III, Freidlin V, Kassoff A, et al. Relative letter and position difficulty on visual acuity charts from the Early Treatment Diabetic Retinopathy Study. Am J Ophthalmol 1993; 116:735–740 Hoffman RS, Packer M, Fine IH. Contrast sensitivity and laser in situ keratomileusis. Int Ophthalmol Clin 2003; 43(2):93–100 Maeda N, Sato S, Watanabe H, et al. Prediction of letter contrast sensitivity using videokeratographic indices. Am J Ophthalmol 2000; 129:759–763 Rabin J. Small letter contrast sensitivity: an alternative measure of visual resolution for aviation candidates. Aviat Space Environ Med 1995; 66:56–58 Rabin J, Wicks J. Measuring resolution in the contrast domain: the small letter contrast test. Optom Vis Sci 1996; 73:398–403 Rabin J. Optical defocus: differential effects on size and contrast letter recognition thresholds. Invest Ophthalmol Vis Sci 1994; 35:646–648 Blommaert FJJ, Timmers H. Letter recognition at low contrast levels: effects of letter size. Perception 1987; 16:421–432 Enoch JM, Werner JS, Haegerstrom-Portnoy G, et al. Forever young: visual functions not affected or minimally affected by aging: a review. J Gerontol A Biol Sci Med Sci 1999; 54:B336–B351 Brown SM, Khanani AM, Xu KT. Day to day variability of the dark-adapted pupil diameter. J Cataract Refract Surg 2004; 30:639–644

From the Department of Ophthalmology and Visual Sciences (Khanani, Brown) and the Department of Health Services Research and Management (Xu), Texas Tech University Health Sciences Center, Lubbock, Texas, USA. Supported by a grant from M & S Technologies, Chicago, Illinois, USA. Software designed by Steven Nordstrom. Mr. Nordstrom is a partner in M & S Technologies, USA. Mr. Khanani received a research stipend at the conclusion of the study. No author has a commercial or proprietary interest in any product mentioned. Rockefeller Young, PhD, assisted with the study design, interim statistical analysis, and manuscript review.

J CATARACT REFRACT SURG—VOL 30, NOVEMBER 2004

ORIGINAL STUDY

M&S Smart System Contrast Sensitivity Measurements Compared With Standard Visual Function Measurements in Primary Open-Angle Glaucoma Patients Jessica L. Liu, BA,*w J. Jason McAnany, PhD,* Jacob T. Wilensky, MD,* Ahmad A. Aref, MD,* and Thasarat S. Vajaranant, MD*

Purpose: To evaluate the nature and extent of letter contrast sensitivity (CS) deficits in glaucoma patients using a commercially available computer-based system (M&S Smart System II) and to compare the letter CS measurements to standard clinical measures of visual function. Methods: Ninety-four subjects with primary open-angle glaucoma participated. Each subject underwent visual acuity, letter CS, and standard automated perimetry testing (Humphrey SITA 24-2). All subjects had a best-corrected visual acuity (BCVA) of 0.3 log MAR (20/40 Snellen equivalent) or better and reliable standard automated perimetry (fixation losses, false positives, and false negatives <33%). CS functions were estimated from the letter CS and BCVA measurements. The area under the CS function (AUCSF), which is a combined index of CS and BCVA, was derived and analyzed. Results: The mean ( ± SD) BCVA was 0.08 ± 0.10 log MAR (B20/25 Snellen equivalent), the mean CS was 1.38 ± 0.17, and the mean Humphrey Visual Field mean deviation (HVF MD) was 7.22 ± 8.10 dB. Letter CS and HVF MD correlated significantly (r = 0.51, P < 0.001). BCVA correlated significantly with letter CS (r = 0.22, P = 0.03), but not with HVF MD (r = 0.12, P = 0.26). A subset of the subject sample (B20%) had moderate to no field loss (r 6 dB MD) and minimal to no BCVA loss (r0.3 log MAR), but had poor letter CS. AUCSF was correlated significantly with HVF MD (r = 0.46, P < 0.001). Conclusions: The present study is the first to evaluate letter CS in glaucoma using the digital M&S Smart System II display. Letter CS correlated significantly with standard HVF MD measurements, suggesting that letter CS may provide a useful adjunct test of visual function for glaucoma patients. In addition, the significant correlation between HVF MD and the combined index of CS and BCVA (AUCSF) suggests that this measure may also be useful for quantifying visual dysfunction in glaucoma patients. Key Words: contrast sensitivity (CS), primary open-angle glaucoma (POAG), area under contrast sensitivity function (AUCSF), M&S Smart System II (MSSS-II)

(J Glaucoma 2017;26:528–533) Received for publication September 2, 2016; accepted February 22, 2017. From the *Department of Ophthalmology and Visual Sciences, University of Illinois at Chicago, Chicago, IL; and wSaint Louis University, St. Louis, MO. Supported by NIH grants K23EY022949 (T.S.V.) and EY001792 (UIC Department of Ophthalmology). Komarek-Hyde-McQueen Glaucoma Research Fund. A Dolly Green Special Scholar Award (J.J.M.) and an unrestricted departmental award from Research to Prevent Blindness. Disclosure: The authors declare no conﬂict of interest. Reprints: J. Jason McAnany, PhD, Department of Ophthalmology and Visual Sciences, University of Illinois at Chicago, 1855W Taylor St, Chicago, IL 60612 (e-mail: jmcana1@uic.edu). Copyright r 2017 Wolters Kluwer Health, Inc. All rights reserved. DOI: 10.1097/IJG.0000000000000659

laucoma is a leading cause of irreversible blindness. It is an insidious disease that damages retinal ganglion cells, which results in characteristic optic nerve and visual field changes. It is estimated that B80 million individuals will be affected by glaucoma by 2020. Of those, B11 million are expected to be bilaterally blind from glaucoma by 2020.1 Given the significant limitations that advanced disease places on individuals and their quality of life,2–6 early detection of glaucoma is critical so that timely interventions can be made. Although standard clinical techniques are routinely used to assess visual dysfunction in patients who have glaucoma, there is a need for more sensitive methods to quantify functional abnormalities. Currently, standard automated perimetry (SAP) is most commonly used in the clinic to assess visual function. There are, however, a number of limitations to SAP. For example, the test requires considerable cooperation from the subject, as constant attention and maintained fixation are needed throughout the relatively lengthy test. Lapses of attention and fixation instability contribute to high test-retest variability. Contrast sensitivity (CS), which is the ability to detect small differences in luminance, is an essential component of visual function and it is correlated with overall quality of life,7–12 target identification in natural images,13 driving, walking, and the ability to recognize faces.14 It has been suggested that functional complaints in some glaucoma patients who have relatively good visual acuity (VA) and minimal visual field deficits may be related to central visual field CS loss.15 Measurement of CS using letter targets is a promising, but underutilized, approach to assessing visual function in glaucoma. Previous reports have documented CS losses in glaucoma patients, even in individuals who have relatively good VA.16 Furthermore, our previous work has shown that letter CS and SAP visual field sensitivity are correlated,15 which may be expected since both tests assess CS. Thus, CS can be a useful measure for quantifying visual function in patients who have glaucoma. Measurements of letter CS are most commonly performed with the Pelli-Robson letter CS chart.17 Although the Pelli-Robson chart has received considerable use by researchers, it has not been widely adopted for routine clinical use. There are a number of practical reasons for its limited use, such as the relatively large size of the chart, which makes it difficult to illuminate evenly by standard overhead lighting. Furthermore, the chart fades over time and it can be difficult to keep clean and free of defects. Recently, M&S Technologies Inc. (Niles, IL) has introduced the M&S Smart System II (MSSS-II) that includes a computer-based letter CS test. This system has the potential to overcome some of the limitations of the Pelli-Robson chart, making it more

J Glaucoma

�

Volume 26, Number 6, June 2017

attractive for regular clinical use. For example, the test is conducted in a dark room using a wall-mounted video display, which largely obviates room lighting limitations. As the same display is used for VA and CS measurements, there is no need to change devices or relocate the subject during testing. Importantly, CS measurements with this system have shown good agreement with the Pelli-Robson chart.4 Thus, the purpose of the present study was to determine the relationships among letter CS, assessed by the MSSS-II, and routine measures of visual function including VA and SAP in patients who have glaucoma. If letter CS and SAP provide similar information regarding functional losses in glaucoma, then CS may be useful as a surrogate measure of visual function for glaucoma patients who cannot reliably perform SAP. In addition, the CS function (CSF), which relates CS to letter size, was approximated from the letter CS and VA measurements and the area under the CSF (AUCSF) was derived and compared with SAP measurements. The AUCSF has been shown to be a useful 1-number metric18 that is jointly dependent on VA and large letter CS; however, to our knowledge, AUCSF has not been reported in patients who have glaucoma.

METHODS Participants This prospective study included subjects with a diagnosis of primary open-angle glaucoma (POAG) who presented to the Illinois Eye and Ear Infirmary Glaucoma Service. The eye with the highest Humphrey Visual Field mean deviation (HVF MD) score was selected for testing (the “better-seeing” eye). POAG was defined by the presence of optic nerve damage and/or visual field loss without secondary causes. All subjects were noted to have open angles on gonioscopy. Each subject voluntarily provided informed written consent and the study was approved by an Institutional Review Board at the University of Illinois at Chicago. All subjects were Z18 years of age and had the ability to understand the procedures and a willingness to comply with the study. Inclusion criteria included: (1) best-corrected VA of 0.3 log MAR or better (equivalent to 20/40 or better Snellen); (2) pupils >4 mm; (3) a wellcontrolled IOP; (4) no recent ophthalmic surgery within 6 months; and (5) a reliable SAP test (Humphrey 24-2) within the past 6 months (fixation losses, false positives, and false negatives <33%). The lens of each subject was graded by slit lamp examination using a subjective clinical scale that ranged from clear to 4 + . Subjects with nuclear sclerotic, posterior subcapsular, or cortical lens opacities >1 + , which corresponds to minimal cataract, were excluded. In addition, no subject had diabetic retinopathy, age-related macular degeneration, optic nerve disease other than glaucoma, evidence of infection, inflammation, or other ocular or systemic conditions known to affect visual function.

Testing Procedures and Analysis Each patient underwent best-corrected VA (BCVA) and CS testing on the same day using the MSSS-II in accordance with the instructions provided by the manufacturer. VA was measured by presenting a single letter from the Sloan set and decreasing the letter size according to the log MAR scale (0.1 log unit steps). For CS measurements, the letter size was set to 20/600, which is Copyright

M&S Smart System Contrast Sensitivity Measurement

equivalent to the size of the letters on the Pelli-Robson chart. A single letter from the Sloan set was selected at random, presented, and the subject was asked to verbally identify the letter. The first letter presented was at the highest contrast produced by the display (100%) and the contrast was decreased following each correct response (B0.1 log unit steps) until the subject could no longer correctly identify the letter presented. If a subject answered incorrectly, the contrast was increased by 0.3 log units to confirm a correct answer and then the subject was tested again by reducing the contrast in B0.1 log unit steps until the subject answered incorrectly a second time. The contrast value for the last correct response was recorded as contrast threshold. The luminance of the MSSS-II system was calibrated using a Minolta LS-110 luminance meter (Konica Minolta). The display luminance was set to 85 cd/m2 and the luminance of the letter targets was measured. Contrast was defined as Weber contrast: ðLletter �Lbackground Þ/Lbackground :

ð1Þ

Lletter is the luminance of the letter and Lbackground is the luminance of the background. CS was defined as the log of 1/letter contrast threshold. For the particular display used in the present study, the g value was set to 1.4 through the software interface, which provided a good correspondence between the nominal contrast reported by the MSSS-II and the contrast derived from the luminance measurements. Visual field perimetry was performed using the HVF analyzer SITA 24-2 standard algorithm (Humphrey Instruments Inc., CA). The present analysis utilized the mean deviation (MD) value to group the subjects according to the severity of the field loss: mild (MDZ �6 dB) and advanced (MD < � 6 dB).19 The eye with the better HVF MD score was selected for further CS testing. Subjects were also subdivided into smaller groups for analysis based on their HVF MD scores: 1 to �3, � 3 to �6, �6 to �12, and �12 to �32 dB (nominally: no, mild, moderate, and severe field loss, respectively). The CSF was approximated based on the large letter CS value and the BCVA value obtained with the MSSS-II. To accomplish this, a model of the CSF obtained from previous research was used.20 This model assumes that the CSF measured with letter stimuli has the same low-pass shape for all subjects. Specifically, the model predicts that CS is related to letter size by the following function: CS ¼ Af n e�pf : ð2Þ CS is the contrast sensitivity at letter size f (1/MAR), n governs the CS attenuation low spatial frequencies, and A and p are vertical and horizontal scaling parameters. In the present study, A and p were free parameters and n was set to 0.1, based on previous findings.20 This CSF model was shifted vertically and horizontally on log-log coordinates (adjusting parameters A and p) to minimize the meansquared error between the model CSF and the measured data points (large letter CS and BCVA). The area under this curve (AUCSF) was then derived, as described elsewhere.18 The AUCSF is a useful single-number metric that is jointly dependent on the subject’s large letter CS and BCVA.

RESULTS The analysis included 94 eyes of 94 subjects with a diagnosis of POAG. The mean age of the subjects was

www.glaucomajournal.com |

529

Liu et al

65 years (range, 38 to 88 y). Fifty-one subjects (54.3%) were African American, 18 (19.1%) were white, 12 (12.8%) were Hispanic and 13 (13.8%) were classified as “other” (the majority of whom were Asian). Fifty-nine subjects (63%) were classified as having mild glaucoma (HVF MD loss <6 dB) and 35 (37%) had advanced glaucoma (HVF MD loss >6 dB). Given the inclusion criteria, all subjects in our sample had BCVA of 0.3 log MAR (20/40 Snellen equivalent) or better. The mean ( ± SD) BCVA for the subject sample was 0.08 ± 0.10 log MAR (approximate Snellen equivalent of 20/ 25), the mean CS was 1.38 ± 0.17, and the mean HVF MD was 7.22 ± 8.10 dB. BCVA and large letter CS were correlated significantly (r = 0.22, P = 0.03), but BCVA was not correlated significantly with HVF MD (r = 0.12, P = 0.26). Figure 1A shows the HVF MD value as a function of log letter CS for each subject. The black triangles represent subjects with MD values better than 6 dB and the red squares represent subjects with MD values worse than 6 dB; solid lines are linear regression fits to the data. The gray region represents the expected range of normal, based

FIGURE 1. HVF MD as a function of log letter CS. Data are shown for subjects who have a HVF MD loss <6 dB (triangles) and for subjects who have a HVF MD loss >6 dB (squares) in panel A. The lines represent linear regression fits to the data as described in the text. Panel B replots the data in A for subjects separated into 4 groups. The lines in B represent linear regression fits to the data with slopes constrained to 0, as described in the text. The gray region in both panels represents the normal range based on previous data. CS indicates contrast sensitivity; HVF MD, Humphrey Visual Field mean deviation. Figure 1 can be viewed in color online at www.glaucomajournal.com.

J Glaucoma

Volume 26, Number 6, June 2017

on previous findings.4 Figure 1A shows that the range of letter CS spanned a factor of B10 (log CS values of 0.8 to 1.8), whereas the range of HVF MD was much larger, spanning a factor of >1000 (1 to 32 dB). These 2 measures were correlated significantly when compared for all subjects (r = 0.51, P < 0.001) and also when subdivided into subjects with HVF MDZ 6 dB (r = 0.32, P = 0.01) and HVF MD < 6 dB (r = 0.41, P = 0.01). As an additional approach to examine the relationship between HVF MD and large letter CS, the subjects were further subdivided based on MD values into 4 groups that had progressively greater field loss. Figure 1B shows the relationship between HVF MD and log letter CS for these 4 groups. Linear regression analysis was performed for each group, but the slopes were not significantly different from 0. Consequently, regression lines with a slope constrained to 0 were fit to each data set (horizontal solid lines). The results indicate that there were no significant correlations between HVF MD and large letter CS within these subgroups. Green circles within the white area (to the left of the gray normal range) represent subjects who had relatively good VA (0.3 log MAR or better), minimal to no field loss (< 3 dB), but had low letter CS. CSFs estimated from the large letter CS and VA measurements are shown in Figure 2. In this figure, log CS is plotted as a function of log letter size, with letter size decreasing from left to right. The black triangles (Fig. 2A) represent the mean CSF for subjects with MD loss <6 dB, whereas the red squares represent the mean CSF for subjects with MD loss >6 dB. The functions fit to the data represent the model template for the letter CSF (Equation 2).20 These mean CSFs show that the 2 groups had different large letter CS values (left data points), despite similar VA values (right data points). Figure 2B shows the CSFs for the subjects subdivided into 4 groups, based on HVF MD, as discussed above. Consistent with the pattern shown in Figure 2A, the large letter CS decreased systematically as the HVF MD value decreased (greater field loss), whereas the VA values showed greater similarity among the 4 groups. On the basis of each subject’s CSF, the AUCSF was derived. Figure 3A shows the HVF MD value as a function of the AUCSF. Overall, the AUCSF was correlated significantly with HVF MD (r = 0.46, P < 0.001). The subjects were grouped into those who had MD loss <6 dB (black triangles) and those who had MD loss >6 dB (red squares). The AUCSF was found to be correlated significantly with HVF MD for subjects who had a <6 dB MD loss (r = 0.38, P < 0.01), but not for subjects who had a >6 dB HVF MD loss (r = 0.23, P = 0.18). As in the previous plots, the subjects were further subdivided into 4 groups based on their HVF MD values, which were plotted as a function of AUCSF in Figure 3B. Linear regression analysis was performed for each group and the slopes were not significantly different from 0. Consequently, regression lines with a slope constrained to 0 were fit to each data set (horizontal solid lines). The results indicate that there were no significant correlations between HVF MD and AUCSF within these subgroups.

DISCUSSION This study quantiﬁed the relationships among letter CS, assessed by the MSSS-II, and routine measures of visual function including VA and SAP in subjects with glaucoma. Although previous work has examined CS in glaucoma patients using a variety of tests,15,21–23 the present

J Glaucoma

Volume 26, Number 6, June 2017

M&S Smart System Contrast Sensitivity Measurement

FIGURE 3. HVF MD as a function of AUCSF. Conventions are as in Figure 1. AUCSF indicates area under contrast sensitivity function; HVF MD, Humphrey Visual Field mean deviation. Figure 3 can be viewed in color online at www.glaucomajournal.com.

FIGURE 2. Estimated CS functions (CSFs) are shown for subjects who have a HVF MD loss <6 dB (triangles) and for subjects who have a HVF MD loss >6 dB (squares) in panel A. The CSFs were derived from a previous model, as described in the text. Panel B shows the CSFs for subjects divided into 4 groups based on their HVF MD values. CS indicates contrast sensitivity; HVF MD, Humphrey Visual Field mean deviation. Figure 2 can be viewed in color online at www.glaucomajournal.com.

report is the ﬁrst to evaluate CS using the digital MSSS-II display that has recently become commercially available. We show that the large letter CS measurements assessed with the MSSS-II are correlated signiﬁcantly with standard HVF MD measurements, suggesting that large letter CS may be a useful adjunct test of visual function in glaucoma patients. In addition, a combined index of the VA and letter CS measurements (AUCSF) was derived and shown to correlate with HVF MD. The MSSS-II has advantages over standard chartbased CS tests. For example, the display is self-luminous, which obviates the dependence on room illumination. The MSSS-II also permits VA and CS to be assessed with the same device at a constant test distance, which reduces test Copyright

time and increases the ease of use. We note, however, that calibration is an important consideration in the use of the MSSS-II. Although the display is calibrated by the manufacturer for test distance to ensure accurate VA measurements, luminance calibration to ensure accurate CS measurements is not as easily achieved. Specifically, adjustment of the display’s g function and measurement of letter and screen luminance with a photometer was required to ensure that the nominal contrast values reported by the display were accurate. Furthermore, informal testing with different MSSS-II displays indicated that the gamma adjustment required to best match the nominal and measured contrast values may vary among displays. Nevertheless, when properly calibrated, we found the MSSS-II to be a useful, easily implemented test of letter CS. Routine CS measurement may be of value in assessing visual function in glaucoma patients, given the finding of low CS in patients who otherwise had minimal or no field loss (< 3 dB) and relatively good VA (0.3 log MAR or better). Although glaucoma has traditionally been thought to affect peripheral visual function in its early stages, there is evidence of altered foveal/parafoveal function as well.15,23,24 Reduced central field CS may, at least in part, account for some of the subjective complaints of glaucoma patients who have minimal field loss and relatively good VA. This speculation is consistent with the finding that CS is better related to “real-world” function than VA in patients with early stages of glaucoma.25 Large letter CS

www.glaucomajournal.com |

531

Liu et al

measurements may also be of value in cases where reliable HVF measures cannot be obtained. HVF measurement is heavily dependent on patient alertness and cooperation. Large letter CS testing such as visual field perimetry, is a subjective test of visual function, but letter CS tests can be completed quickly and may be more patient-friendly than visual field perimetry. Despite the relative ease of letter CS testing and its significant correlation with HVF MD, letter CS testing may not be an ideal approach for early detection of glaucoma. That is, patients who had good letter CS had HVF MD values that ranged from 0 to 30 dB. Conversely, patients who had low letter CS could have normal (or nearly normal) HVF MD values. This suggests that the sensitivity and specificity of letter CS for detecting early glaucoma would be relatively poor, consistent with the findings of Wood and Lovie-Kitchin.26 Nevertheless, letter CS measurements may be useful for: (1) patients who have subjective complaints of poor vision and minimal to no VA or HVF MD loss; (2) patients who cannot reliably perform SAP, as there is a significant correlation between HVF MD and letter CS, despite the variation. One important limitation of standard letter CS testing is an inability to provide information about sensitivity losses localized to the peripheral visual field, as letter CS is typically a functional test of the central macula. Similarly, the use of HVF MD may not be an ideal metric for quantifying glaucomatous field loss, as this measure can also ignore localized defects. The good correspondence between CS and “real-world” function,25 noted above, suggests that there is value in CS measurement in glaucoma patients. A second limitation to consider is that both letter CS and SAP can be affected by factors such as media opacity, pupil size, and age. To avoid possible effects of media opacity, subjects who had significant cataracts were excluded from the present study. Furthermore, a subanalysis showed no significant differences in letter CS (F = 2.46, P = 0.07) or HVF MD (F = 1.47, P = 0.23) among patients who had different levels of lens opacity (pseudophakic, clear, trace, 1 +). The effects of pupil size on letter CS are also likely to be negligible in our sample, as there was no significant correlation between log letter CS and pupil diameter (r = 0.09, P = 0.40) or HVF MD and pupil diameter (r = 0.20, P = 0.06). Similarly, age was not significantly correlated with letter CS (r = 0.18, P = 0.07) or HVF MD (r = 0.03, P = 0.75) in this sample of subjects. The MSSS-II provides measures of VA in addition large letter CS, which permits the CSF, and the AUCSF, to be estimated. In the present study, the AUCSF was found to be correlated with HVF MD for subjects who had moderate to no field loss (< 6 dB loss). VA and large letter CS were also significantly correlated with HVF MD for these subjects, so the finding that AUCSF is correlated with HVF MD is expected, as AUCSF is jointly dependent on VA and large letter CS. Conversely, for subjects with marked visual field loss (HVF MD reduction of >6 dB), AUCSF was not correlated with HVF MD. VA was also not correlated with HVF MD for these subjects, which likely attributed to the nonsignificant relationship between HVF MD and AUCSF. The glaucoma subjects in our sample all had VA better than 0.3 log MAR, which limited the possible amount of variation in VA among the subjects. It would be of interest to evaluate the relationship between AUCSF and HVF MD in glaucoma patients who have larger VA losses. It would also be of interest to derive the complete CSF using the MSSS-II in patients who have glaucoma and varying disease stages. The AUCSF derived

J Glaucoma

Volume 26, Number 6, June 2017

in the present study is dependent on the assumption that the shape of the CSF is similar for all patients. Although this likely to be a reasonable assumption, the method to derive the AUCSF in the present study would not be sensitive to selective deficits (“notches”) at specific letter sizes. Future work is needed to completely define the shape of the letter CSF in patients who have different stages of glaucoma.

CONCLUSIONS CS testing using the MSSS-II can be performed quickly, is less demanding than visual field perimetry, and can be easily incorporated into a busy clinical practice. Furthermore, for glaucomatous eyes with VA of 20/40 or better, large letter CS assessed with the MSSS-II correlated significantly with visual field loss. As such, letter CS testing may provide a useful supplementary measure of visual function, particularly for patients from whom reliable visual field measurements cannot be obtained. Future work is needed to determine the extent to which large letter CS and the AUCSF are useful for early detection of glaucoma and for quantifying the progression of functional losses. REFERENCES 1. Quigley HA, Broman AT. The number of people with glaucoma worldwide in 2010 and 2020. Br J Ophthalmol. 2006; 90:262–267. 2. Gutierrez P, Wilson MR, Johnson C, et al. Influence of glaucomatous visual field loss on health-related quality of life. Arch Ophthalmol. 1997;115:777–784. 3. Parrish RK II, Gedde SJ, Scott IU, et al. Visual function and quality of life among patients with glaucoma. Arch Ophthalmol. 1997;115:1447–1455. 4. Chandrakumar M, Colpa L, Reginald Y, et al. Measuring Contrast Sensitivity using the M&S Smart System II versus the Pelli-Robson Chart. Ophthalmology. 2013;10:2160–2161. 5. Viswanathan AC, McNaught AI, Poinoosawmy D, et al. Severity and stability of glaucoma: patient perception compared with objective measurement. Arch Ophthalmol. 1999;117: 450–454. 6. Odberg T, Jakobsen JE, Hultgren SJ, et al. The impact of glaucoma on the quality of life of patients in Norway, II: patient response correlated to objective data. Acta Ophthalmol Scand. 2001;79:121–124. 7. Owsley C. Contrast sensitivity. Ophthalmol Clin North Am. 2003;16:171–177. 8. Mills RP, Janz NK, Wren PA, et al. Correlation of visual field with quality-of-life measures at diagnosis in the Collaborative Initial Glaucoma Treatment Study (CIGTS). J Glaucoma. 2001; 10:192–198. 9. Nelson P, Aspinall P, Papasouliotis O, et al. Quality of life in glaucoma and its relationship with visual function. J Glaucoma. 2003;12:139–150. 10. Ross JE, Bron AJ, Clarke DD. Contrast sensitivity and visual disability in chronic simple glaucoma. Br J Ophthalmol. 1984; 68:821–827. 11. Jampel HD, Schwartz A, Pollack I, et al. Glaucoma patients’ assessment of their visual function and quality of life. J Glaucoma. 2002;11:154–163. 12. Richman J, Lorenzana LL, Lankaranian D, et al. Relationships in glaucoma patients between standard vision tests, quality of life, and ability to perform daily activities. Ophthalmic Epidemiol. 2010;17:144–151. 13. Owsley C, Sloane ME. Contrast sensitivity, acuity, and the perception of “real-world” targets. Br J Ophthalmol. 1987;71: 791–796. 14. Rubin GS, Ng ESW, Bandeen-Roche K, et al. A prospective, population-based study of the role of visual impairment in

J Glaucoma

15. 16. 17. 18.

19. 20.

Volume 26, Number 6, June 2017

motor vehicle crashes among older drivers: the SEE study. Invest Ophthalmol Vis Sci. 2007;48:1483–1491. Wilensky J, Hawkins A. Comparison of contrast sensitivity, visual acuity, and Humphrey Visual Field testing patients with glaucoma. J Glaucoma. 2003;12:134–138. Richman J, Lorenzana LL, Lankaranian D, et al. Importance of visual acuity and contrast sensitivity in patients with glaucoma. Arch Ophthalmol. 2010;128:1576–1582. Pelli DG, Robson JG, Wilkins AJ. The design of a new letter chart for measuring contrast sensitivity. Clin Vis Sci. 1988;2: 187–199. Applegate RA, Hilmantel G, Howland HC. Area under log contrast sensitivity function: A concise method of following changes in visual performance. Vision Science and Its Applications. Technical Digest Series. 1997;1:98–101. Hodapp E, Richard K, Parrish II, et al. Clinical Decisions in Glaucoma. St. Louis, MO: Mosby; 1993. McAnany JJ, Alexander KR. Contrast sensitivity for letter optotypes vs. gratings under conditions biased toward parvocellular and magnocellular pathways. Vision Res. 2006;46:1574–1584.

M&S Smart System Contrast Sensitivity Measurement

21. Arden GB, Jacobson JJ. A simple grating test for contrast sensitivity: preliminary results indicate value in screening for glaucoma. Invest Ophthalmol Visual Sci. 1978;17:23–32. 22. Pomerance GN, Evans DW. Test-retest reliability of the CSV1000 contrast test and its relationship to glaucoma therapy. Invest Ophthalmol Vis Sci. 1994;35:3357–3361. 23. Lahav K, Levkovitch-Verbin H, Belkin M, et al. Reduced mesopic and photopic foveal contrast sensitivity in glaucoma. Arch Ophthalmol. 2011;129:16–22. 24. Marx MS, Bodis-Wollner I, Lustgarten JS, et al. Electrophysiological evidence that early glaucoma affects foveal vision. Doc Ophthalmol. 1987;67:281–301. 25. Szlyk JP, Taglia DP, Paliga J, et al. Driving performance in patients with mild to moderate glaucomatous clinical vision changes. J Rehabil Res Dev. 2002;39:467–482. 26. Wood JM, Lovie-Kitchin JE. Contrast sensitivity measurement in the detection of primary open angle glaucoma. In: Mills RP, Heijl A, eds. Perimetry Update 1990/1991 Proceedings of the IXth International Perimetric Society Meeting, June 1990. Malmo, Sweden; 1991.

www.glaucomajournal.com |

533

Ophthalmic and Physiological Optics ISSN 0275-5408

TECHNICAL NOTE

Repeatability of an automated ETDRS contrast threshold measurement Yi Pang1

, Lauren Sparschu1 and Jingyun Wang2

Illinois College of Optometry, Chicago, Illinois, USA, and 2SUNY College of Optometry, New York, New York, USA

Citation information: Pang Y, Sparschu L, & Wang J. Repeatability of an automated ETDRS contrast threshold measurement. Ophthalmic Physiol Opt. 2021. https://doi.org/10.1111/opo.12829

Keywords: contrast sensitivity, repeatability

Abstract

Correspondence: Jingyun Wang E-mail address: jwang@sunyopt.edu

Received: 15 September 2020; Accepted: 5 March 2021

Introduction

Participants

Automated ETDRS contrast threshold

Y Pang et al.

19 (48) 21 (52) 0 to 0.7 0.2 (0.3) 32 (80) 8 (20) 22 (55) 7 (18) 8 (20) 3 (8) 22.2–75.0 47.6 (13.8)

Automated ETDRS contrast threshold

Y Pang et al.

Automated ETDRS contrast threshold

Y Pang et al.

THANK YOU For more information please call 1-847-763-0500

The First Choice in Vision Testing Systems