Education Corner: Mind the Gap: Appraisal of FOAM

Mind the Gap: Appraisal of FOAM

By Carmen J. Martinez Martinez, MD, MSMEd, FACEP, FAAEM By Caroline M. Molins, MD, MSMEd, FACEP, FAAEM

Free Open Access Medical Education (FOAM) has exponentially grown over the last decade. Each year, more residents turn away from textbooks and move towards using FOAM as a source of education and information. As an innovative form of education, FOAM is disruptive, challenging traditional modes of knowledge translation and dissemination.1 Although some educators are skeptical of its formal use, FOAM is appealing to learners for several reasons. It has a low cost, is easily accessible, and quickly disseminates new evidence and techniques when compared to traditional formats. In addition, the information is delivered in a variety of formats, which makes it suitable to the different types of learning styles. As educators, how do we know if they are using high-quality resources? Are we teaching our learners how to navigate and appraise these resources? There is a gap in the education on how to critically appraise different forms of FOAM resources.

Given the skepticism of many educators, many have turned to developing quality markers that help assess the quality of a given FOAM, to critically appraise and help guide both educators and learners alike. This article will discuss existing tools to help our learners assess FOAM and some best practices to help teach how best to use FOAM.

FOAM can describe many components, including medical education websites and individualized online educational resources. These include a variety of applications such as blogs, podcasts, social media components, opinion, commentary and archives. Some examples of FOAM within emergency medicine include, but are not limited to: Life in the Fast Lane; EMCrit, Academic Life in Emergency Medicine, CanadiEM, Dr Smith’s ECG Blog and EMS 12 Lead (Fig. 1).

Some studies have suggested that the individual gestalt of assessment of these resources was unreliable. As a result, several investigators have worked to develop structured, critical appraisal tools to identify higher quality resources. For our field of emergency medicine, available critical appraisal tools are the Social Media Index (SMi), Academic Life in Emergency Medicine Approved Instructional Resource (ALiEM Air score), the revised Medical Education Translational Resources: Impact and Quality (rMETRIQ score), and Colmers' Quality checklists (Table 1). From these, SMi, rMETRIQ and ALiEM AIR score are the most used.

The SMi is the most used tool to judge the overall impact of FOAM. At the early stages in which SMi was used, many critiqued its parameters for online readership, followership and popularity, but these have diminished as more research has been applied.2 SMi is based on a mathematical model, which allows for transparency and objectivity. Drawbacks to SMi include that they can be artificially overestimated, and since there are likely many contributors to one website, the quality can differ from each page or post. It is important to remember that SMi rates the website in its entirety; not an individual blog or podcast.1 The latest ranking can be viewed online ataliem.com/socialmedia-index.

The ALiEM Air score is a scoring instrument used by medical educators for rating online educational resources. It is composed of five domains: Best Evidence in Emergency Medicine

Top 5 FOAM Websites

(ranked by SMi as of 2021)

#1 #2 #3 #4 #5

LIFL EMCRIT ALiEM REBELEM EMCASES

Courtesy of aliem.com/social-media-index

(BEEM) Rater Scale, Content Accuracy, Educational Utility, Evidence-Based Medicine, and Referencing. When used by medical educators for rating online resources, this score is moderately to highly reliable.3 A major disadvantage to mention is that a minimum of nine assessors was required to achieve that reliability.

The rMETRIQ score is intended to be used by a variety of healthcare providers, from students to content experts, derived from the previous METRIQ score (METRIQ - 8 and METRIQ 5). It looks at 3 main components: content, credibility and peer review. It is designed for pointof-care use in the assessment of blog articles.

Table 1: Quality Assurance Tools for FOAM1,2,5

Appraisal Tool Intended Audience Intended Use Domains Assessed Tips

Social Media Index (SMi)

FOAM users FOAM Websites Relies on mathematical model (Alexa traffic rank score, Twitter followers, Facebook likes); Initially used for most impactful FOAM Websites; Generally utilized for EM and critical care websites

rMETRIQ

FOAM Users – any level of healthcare providers (trainee to expert level) Individualized FOAM Resources Assess Concise content; content construction; editorial process; references, publisher, writing quality, postpublication commentary Rigorous derivation; based on prior METRIQ/ MERSQI scorings Pending validation

ALiEM AIR Score Expert evaluators Individualized FOAM Resources

Colmers' Quality checklists

Producers, Curators; Editors; Educators; Researchers; FOAM Users Specific to Blogs and Podcasts Credibility; Content; Design Simple; userfriendly; pending validation

BEEM Rater scale; Assess content accuracy; educational utility; EBM; referencing Impact on EM Clinical Practice

approach to medical education, but it should be considered a strong adjunct. We know that many young medical professionals are using it on a regular basis and some may solely use these resources as their primary source of education. Furthermore, it is an enticing resource that promotes engagement, knowledge dissemination, networking and collaboration. But we cannot forget that it has its limitations.4 The unguided use of these resources by learners can pose a danger to their education and potentially patient care.

Next time you are looking at a FOAM resource, “mind the gap” and consider applying these tools! ■

REFERENCES:

1. Ting, Daniel K et al. “Quality Appraisal and Assurance Techniques for Free

Open Access Medical Education (FOAM)

Resources: A Rapid Review.” Seminars in nephrology vol. 40,3 (2020): 309-319. doi:10.1016/j.semnephrol.2020.04.011 2. Chan, Teresa M et al. “Thinking

Critically About Appraising FOAM.” AEM education and training vol. 3,4 398-402. 23 May. 2019, doi:10.1002/aet2.10352 3. Chan, Teresa Man-Yee et al. “Examining

Reliability and Validity of an Online

Score (ALiEM AIR) for Rating Free Open

Access Medical Education Resources.”

Annals of emergency medicine vol. 68,6 (2016): 729-735. doi:10.1016/j. annemergmed.2016.02.018 4. Nickson, Christopher. "Free Open-

Access Medical education (FOAM) and critical care." ICU Management &

Practice vol. 17,4 (2017): 222-225 5. Colmers, Isabelle et al. "The Quality

Checklists for Health Professions

Blogs and Podcasts." The Winnower 9:e144720.08769 (2015). doi:10.15200/ winn.144720.08769 6. Chin, A., Taher, A., Thomas, A., Bigham,

B., Thoma, B., & Woods, R. (2020, June 17). Using quality appraisal techniques to find trustworthy content in the foam universe. CanadiEM. canadiem.org/ using-quality-appraisal-techniques-infoam/ 7. Carley, S. (2017, April 29). The social media index (SMI): CAN & should we measure #foamed? St.Emlyn's. www. stemlynsblog.org/the-social-mediaindex-smi-is-it-flawed/