Vaccination Programs: Client Reminder and Recall Systems

Effectiveness Review

Analytic Framework - Effectiveness Review

• Logic Model [PDF (Print Only) - 279 kB]
• Text Description [PDF - 171 kB]

When starting an effectiveness review, the systematic review team develops an analytic framework. The analytic framework illustrates how the intervention approach is thought to affect public health. It guides the search for evidence and may be used to summarize the evidence collected. The analytic framework often includes intermediate outcomes, potential effect modifiers, potential harms, and potential additional benefits.

Effectiveness Review

• Summary Evidence Table - Search Period: 1997-2007 [PDF - 388 kB]
• Summary Evidence Table - Search Period: 2007-2012 [PDF - 172 kB]

The number of studies and publications do not always correspond (e.g., a publication may include several studies or one study may be explained n several publications).

Baker AM, McCarthy B, Gurley VF, Yood MU. Influenza immunization in a managed care organization. Journal of General Internal Medicine 1998;13:469-75.

Britto MT, Pandzik GM, Meeks CS, Kotagal UR. Combining evidence and diffusion of innovation theory to enhance influenza immunization. Joint Commission Journal on Quality and Patient Safety 2006;32(8):426-32.

Daley MF, Barrow J, Pearson K, et al. Identification and recall of children with chronic medical conditions for influenza vaccination. Pediatrics 2004;113:e26-e33.

Daley MF, Steiner JF, Kempe A, et al. Quality improvement in immunization delivery following an unsuccessful immunization recall. Ambulatory Pediatrics 2004;4:217-23.

Dini EF, Linkins RW, Sigafoos J. The impact of computer-generated messages on childhood immunization coverage. American Journal of Preventive Medicine 2000;18:132-9.

Findley SE, Sanchez M, Mejia M, Ferreira R, Pena O, Matos S et al. Effective strategies for integrating immunization promotion into community programs. Health Promotion Practice 2009;10(2 Suppl):128S-37S.

Gill JM, Saldarriaga AM. The impact of a computerized physician reminder and a mailed patient reminder on influenza immunizations for older patients. Delaware Medical Journal 2000;72(10):425-30.

Hambidge SJ, Davidson AJ, Phibbs SL, Chandramouli V, Zerbe G, LeBaron CW et al. Strategies to improve immunization rates and well-child care in a disadvantaged population: a cluster randomized controlled trial. Archives of Pediatrics & Adolescent Medicine 2004; 158:162-9.

Hambidge SJ, Phibbs SL, Chandramouli V, Fairclough D, Steiner JF. A stepped intervention increases well-child care and immunization rates in a disadvantaged population. Pediatrics 2009;124(2):455-64.

Hogg W, Bass MJ, Colange N, Crouch H, Satenstein G. Randomized controlled study of customized preventive medicine reminder letters in a community practice. Canadian Family Physician 1998;44:81-8.

Hull S, Hagdrup N, Hart B, Griffiths C, Hennessy E. Boosting uptake of influenza immunisation: a randomised controlled trial of telephone appointing in general practice. British Journal of General Practice 2002;52:712-6.

Humiston SG, Bennett NM, Long C, et al. Increasing inner-city adult influenza vaccination rates: a randomized controlled trial. Public Health Reports 2011;126:39-47.

Irigoyen MM, Findley S, Wang D, et al. Challenges and successes of immunization registry reminders at inner-city practices. Ambulatory Pediatrics 2006;6:100-4.

Kempe A, Daley MF, Barrow J, et al. Implementation of universal influenza immunization recommendations for healthy young children: results of a randomized, controlled trial with registry-based recall. Pediatrics 2005;115:146-54.

Kharbanda EO1, Stockwell MS, Fox HW, Andres R, Lara M, Rickert VI. Text message reminders to promote human papillomavirus vaccination. Vaccine 2011;29(14):2537-41. doi: 10.1016/j.vaccine.2011.01.065.

LeBaron CW, Starnes D, Dini EF, Chambliss JW, Chaney M. The impact of interventions by a community-based organization on inner-city vaccination coverage: Fulton County, Georgia, 1992-1993. Arch Pediatr Adolesc Med 1998;152(4):327-32.

LeBaron CW, Starnes DM, Rask KJ. The impact of reminder-recall interventions on low vaccination coverage in an inner-city population. Archives of Pediatrics & Adolescent Medicine 2004;158:255-61.

Lemstra M, Rajakumar D, Thompson A, Moraros J. The effectiveness of telephone reminders and home visits to improve measles, mumps and rubella immunization coverage rates in children. Paediatrics and Child Health 2011;16(1):e1-5.

Lieu TA, Capra AM, Makol J, Black SB, Shinefield HR. Effectiveness and cost-effectiveness of letters, automated telephone messages, or both for underimmunized children in a health maintenance organization. Pediatrics 1998;101:E3.

Muehleisen B, Baer G, Schaad UB, Heininger U. Assessment of immunization status in hospitalized children followed by counseling of parents and primary care physicians improves vaccination coverage: an interventional study. Journal of Pediatrics 2007; 151(6):704-6.

Nowalk MP, Zimmerman RK, Lin CJ, et al. Raising adult vaccination rates over 4 years among racially diverse patients at inner-city health centers. Journal of the American Geriatrics Society 2008;56(7):1177-82

Puech M, Ward J, Lajoie V. Postcard reminders from GPs for influenza vaccine: are they more effective than an ad hoc approach? Australian and New Zealand Journal of Public Health 1998;22:254-6.

Rodewald LE, Szilagyi PG, Humiston SG, Barth R, Kraus R, Raubertas RF. A randomized study of tracking with outreach and provider prompting to improve immunization coverage and primary care. Pediatrics 1999;103:31-8.

Sansom S, Rudy E, Strine T, Douglas W. Hepatitis A and B vaccination in a sexually transmitted disease clinic for men who have sex with men. Sexually Transmitted Diseases 2003;30:685-8.

Stockwell MS1, Kharbanda EO, Martinez RA, Vargas CY, Vawdrey DK, Camargo S. Effect of a text messaging intervention on influenza vaccination in an urban, low-income pediatric and adolescent population: a randomized controlled trial. JAMA 2012;307(16):1702-8. doi: 10.1001/jama.2012.502.

Szilagyi PG, Schaffer S, Barth R, et al. Effect of telephone reminder/recall on adolescent immunization and preventive visits: results from a randomized clinical trial. Archives of Pediatrics & Adolescent Medicine 2006;160:157-63.

Szilagyi PG, Humiston SG, Gallivan S, Albertin C, Sandler M, Blumkin A. Effectiveness of a citywide patient immunization navigator program on improving adolescent immunizations and preventive care visit rates. Archives of Pediatrics & Adolescent Medicine 2011;165(6):547-53.

Vora S, Verber L, Potts S, Dozier T, Daum RS. Effect of a novel birth intervention and reminder-recall on on-time immunization compliance in high-risk children. Human Vaccines 2009; 5(6):395-402.

Winston CA, Mims AD, Leatherwood KA. Increasing pneumococcal vaccination in managed care through telephone outreach. American Journal of Managed Care 2007;13(10):581-8.

Wood D, Halfon N, Donald SC, et al. Increasing immunization rates among inner-city, African American children. A randomized trial of case management. JAMA 1998;279:29-34.

Included Studies - Economic Review

Baker AM, McCarthy B, Gurley VF, Yood MU. Influenza immunization in a managed care organization. J Gen Intern Med. 1998;13(7):469-475. http://dx.doi.org/10.1046/j.1525-1497.1998.00136.x.

Berg GD, Thomas E, Silverstein S, Neel CL, Mireles M. Reducing medical service utilization by encouraging vaccines Randomized controlled trial. Am J Prev Med 2004;27(4):284-8.

Buchner DM, Larson EB, White RF. Influenza vaccination in community elderly. A controlled trial of postcard reminders. J Am Geriatr Soc 1987;35(8):755. http://dx.doi.org/10.1111/j.1532-5415.1987.tb06354.x.

Chiu TT. Community mobilization for preschool immunizations: the Shots by Two Project. Am J Public Health 1997;87(3):462-3.

Dini EF, Linkins RW, Sigafoos J. The impact of computer-generated messages on childhood immunization coverage. Am J Prev Med 2000;18(2):132-9. http://dx.doi.org/10.1016/S0749-3797(99)00086-0.

Frame PS, Zimmer JG, Werth PL, Hall WJ, Eberly SW. Computer-based vs manual health maintenance tracking: a controlled trial. Arch Fam Med 1994;3(7):581. http://dx.doi.org/10.1001/archfami.3.7.581.

Frank JW, McMurray L, Henderson M. Influenza vaccination in the elderly: 2. The economics of sending reminder letters. CMAJ 1985;132(5):516.

Franzini L, Rosenthal J, Spears W, et al. Cost-effectiveness of childhood immunization reminder/recall systems in urban private practices. Pediatrics 2000;106(Suppl 1):177-83.

Grabenstein JD, Hartzema AG, Guess HA, Johnston WP, Rittenhouse BE. Community pharmacists as immunization advocates: cost-effectiveness of a cue to influenza vaccination. Med Care 1992:503-513. http://dx.doi.org/10.1097/00005650-199206000-00004.

Hull S, Hagdrup N, Hart B, Griffiths C, Hennessy E. Boosting uptake of influenza immunisation: a randomised controlled trial of telephone appointing in general practice. Br J Gen Pract 2002;52(482):712.

Irigoyen MM, Findley S, Earle B, Stambaugh K, Vaughan R. Impact of appointment reminders on vaccination coverage at an urban clinic. Pediatrics 2000;106(4):919.

Kreuter MW, Caburnay CA, Chen JJ, Donlin MJ. Effectiveness of individually tailored calendars in promoting childhood immunization in urban public health centers. Am J Public Health 2004;94(1):122. http://dx.doi.org/10.2105/AJPH.94.1.122.

LeBaron CW, Starnes DM, Rask KJ. The impact of reminder-recall interventions on low vaccination coverage in an inner-city population. Arch Pediatr Adolesc Med 2004;158(3):255-61. http://dx.doi.org/10.1001/archpedi.158.3.255.

Lemstra M, Rajakumar D, Thompson A, Moraros J. The effectiveness of telephone reminders and home visits to improve measles, mumps and rubella immunization coverage rates in children. Pediatr Child Health 2011;16(1):e1.

Lieu TA, Black SB, Ray P, et al. Computer-generated recall letters for underimmunized children: how cost-effective? Pediatr Infect Dis J 1997;16(1):28-33. http://dx.doi.org/10.1097/00006454-199701000-00007.

Lieu TA, Capra AM, Makol J, Black SB, Shinefield HR. Effectiveness and cost-effectiveness of letters, automated telephone messages, or both for underimmunized children in a health maintenance organization. Pediatrics 1998;101(4):e3. http://dx.doi.org/10.1542/peds.101.4.e3.

McLeod D, Bowie RD, Kljakovic M. The cost of childhood immunisation in general practice. N Z Med J 1998;111(1061):73.

Moran WP, Nelson K, Wofford JL, Velez R, Case LD. Increasing influenza immunization among high-risk patients: education or financial incentive? Am J Med. 1996;101(6):612-20. http://dx.doi.org/10.1016/S0002-9343(96)00327-0.

Nexoe JR, Kragstrup J, Ronne T. Impact of postal invitations and user fee on influenza vaccination rates among the elderly: a randomized controlled trial in general practice. Scand J Prim Health Care 1997;15(2):109-12. http://dx.doi.org/10.3109/02813439709018497.

Rask KJ, LeBaron CW, Starnes DM. The costs of registry-based immunization interventions. Am J Prev Med 2001;21(4):267-71. http://dx.doi.org/10.1016/S0749-3797(01)00370-1.

Rosser WW, Hutchison BG, McDowell I, Newell C. Use of reminders to increase compliance with tetanus booster vaccination. CMAJ 1992;146(6):911.

Smith DM, Zhou XH, Weinberger M, Smith F, McDonald RC. Mailed reminders for area-wide nfluenza immunization: a randomized controlled trial. J Am Geriatr Soc. 1999;47(1):1. http://dx.doi.org/10.1111/j.1532-5415.1999.tb01893.x.

Winston CA, Mims AD, Leatherwood KA. Increasing pneumococcal vaccination in managed care through telephone outreach. Am J Manag Care 2007;13(10):581.

Yokley JM, Glenwick DS. Increasing the immunization of preschool children; an evaluation of applied community interventions. J Appl Behav Anal 1984;17(3):313. http://dx.doi.org/10.1901/jaba.1984.17-313.

Hong K, Leidner AJ, Tsai Y, Tang Z, Cho B, Stokley S. Costs of interventions to increase vaccination coverage among children in the United States: a systematic review . Academic Pediatrics 2021;21(4):S67-77.

Search Strategy - Economic Review

The present review included studies that reported economic outcomes from the 2000 review (search period 1980-1997) combined with studies identified from updated searches (search period 1997- February 2012) within the standard medical and health-related research databases, Google Scholar, and databases specialized to economics and social sciences. The details of the two sets of searches are provided below.

Details of the Updated Search (1997- February 2012)

The team conducted a broad literature search to identify studies assessing interventions to improve vaccination rates. The following nine databases were searched during the period of 1997 up to February 2012: CABI, CINAHL, The Cochrane Library, EMBASE, ERIC, MEDLINE, PSYCHINFO, Soci Abs and WOS. In addition, Google Scholar and specialized databases (CRD-University of York: NHS EED, EconLit, and JSTOR), were also searched. Reference lists of articles reviewed as well as lists in review articles were also considered, and subject matter experts consulted for additional references.

Search Terms

• Immunization
• Vaccination
• Immunization Programs

To be included in the updated review, a study had to do the following:

• Have a publication date of 1997- February 2012
• Evaluate vaccinations with universal recommendations
• Meet the evidence review and Community Guide review team's definition of the interventions
• Be a primary research study with one or more outcomes related to the analytic framework(s)
• Take place in a high income country or countries
• Be written in English; and
• Compare a group of persons who had been exposed to the intervention with a group who had not been exposed or who had been less exposed.

Details of the Original Search (1980-1997)

The following five electronic databases were searched during the original review period of 1980 up to1997: MEDLINE, Embase, Psychlit, CAB Health, and Sociological Abstracts. The team also reviewed reference lists in articles and consulted with immunization experts. To be included, a study had to do the following:

• Have a publication date of 1980–1997
• Address universally recommended adult, adolescent, or childhood vaccinations
• Be a primary study rather than, for example, a guideline or review
• Take place in an industrialized country or countries
• Be written in English
• Meet the definition of the interventions
• Provide information on one or more outcomes related to the analytic frameworks; and
• Compare a group of persons who had been exposed to the intervention with a group who had not been exposed or who had been less exposed. In addition, we excluded studies with least suitable designs for two interventions (provider reminder/recall and client reminder/recall) where the literature was most extensive.

There is no information for this section.