Page 25


Development of Evaluation Methods

In the original proposal in February 2008, we proposed to compare the experience in villages served by therapists with those of villages that currently do not have a therapist, whose residents rely on other means to obtain dental services, typically by itinerant dentists or through travel to a regional clinic. 6 Such a design seemed like a straightforward approach to assessing the performance and impact of dental services provided by therapists compared to the status quo. Upon further investigation, however, several key factors influenced our redirection of the design. First, our site visits, discussions with the Alaska Tribal Coordinating Committee (ATCC, which was a statewide project advisory committee), and additional research led us to realize that the burden of oral disease is quite substantial in Alaska and that it may take 10 years and many more therapists before a visible change in overall community oral health could reasonably be expected. In essence, there is a huge backload of unmet need. Second, there are other strong prevailing behavioral factors—such as diet and use of tobacco and alcohol—that influence oral health, and the therapist was likely to have little immediate control over these confounding factors. Third, the DHAT program is a young program; the longest-serving therapist at the time this program evaluation was initiated had been working for 3 years, and others had been on the job for only 1 year. Therefore, it would be premature to attempt to evaluate the DHAT experience on the basis of oral health outcomes at this time. Rather, we felt it was appropriate to focus on evaluating program implementation, with particular attention to fidelity to program intent and any creation of unintended consequences, while concentrating on the two overriding topics of interest to the sponsoring foundations: provision of care and perceived changes in access to care. Moreover, the National Advisory Committee, in recognition that any long-term evaluation of the DHAT program will need a carefully designed and executed baseline assessment, recommended that this evaluation provide such a rigorous and foundational perspective for future use. We learned from an April 2008 site visit to Alaska that each tribal health organization has some latitude in how a particular therapist is integrated into the existing dental health services system, depending upon local needs, resources, practices, and program philosophy. We further appreciated the tremendous diversity—in geography, number of villages, modes of transportation, and weather—that comprises the various regions of Alaska and that influences how a therapist is utilized. In essence, although each therapist has had similar training in a New 6

Regional clinics are typically located in larger communities (e.g., Bethel, Kotzebue) where the tribal health organization’s hospital is located and where the DHAT’s supervisory dentist works.