Discussion
In a New York City DOHMH pilot project, of 468 recent or potentially recent online foodborne illness complaints posted on Yelp and reviewed by foodborne epidemiologists, three previously unreported restaurant outbreaks were identified. Because foodborne cases have a common exposure, a restaurant patron review-based system can identify small, point-source outbreaks that are not easily found by systems reviewing large sources of data, such as syndromic surveillance of emergency department visits,[4] Google Flu Trends,[5] and analysis of Twitter data for influenza and other public health trends.[6–8] Most importantly, foodborne epidemiologists can confirm reports because Yelp offers a way to follow-up with reviewers for interview.
In this project, only 15 (3%) of the 468 recent or potentially recent illnesses identified on Yelp were also reported directly to New York City's nonemergency 311 service, suggesting that knowledge about 311 reporting is limited. Of further note, after messages regarding the availability of 311 were sent to 290 reviewers who did not meet the project criteria, 32 responded, of whom 25 (78%) said they were unaware of the 311 system or would keep 311 in mind for the future. The 311 service receives approximately 3,000 food poisoning complaints each year, and from that number, about 1% are identified as outbreak-related (DOHMH, unpublished data, 2014).
As social media usage continues to grow among U.S. adults,[9] health departments might consider additional surveillance methods to capture illness reports from those more likely to post a restaurant review online than to contact a health department. By incorporating website review data into public health surveillance programs, health departments might find additional illnesses and improve detection of foodborne disease outbreaks in the community. Similar programs could be developed to identify other public health hazards that reviewers might describe, such as vermin in food establishments.
The findings in this report are subject to at least four limitations. First, to increase the likelihood of identifying true foodborne illness, a narrow focus was chosen for the individual criteria used to score reviews. Therefore, it is possible that some foodborne illnesses were not picked up by the screening software because of low average review scores (e.g., because of illnesses resulting from toxins with short incubation periods). Second, personal contact information for reviewers was unavailable, requiring reviewers to check their Yelp accounts and provide a telephone number to participate, which extended the time from review to interview and might have affected the response rate. Third, investigators were not able to identify any of the infectious agents in the outbreaks. Finally, the system required substantial resources; in addition to programming expertise, staff members were needed to read reviews, send e-mails, interview reviewers, and perform follow-up inspections.
Additional work using social media might improve health department abilities to use the Internet for disease detection. Working with the Chicago Department of Public Health, the Smart Chicago Collaborative recently developed a system to contact those who post foodborne illness complaints either on its website or on Twitter.* For health departments looking for an alternative to analyzing review data weekly, creating an illness-reporting vehicle such as the Utah Department of Health's "I Got Sick" website[1] could be a more practical solution, although it might be less widely used than a review website such as Yelp. Review websites could assist by offering a link to the reviewer's local health department's reporting system at the time of review posting.
DOHMH plans to continue to refine this project. To shorten the time from review to investigation, Yelp will provide daily instead of weekly review feeds, and, to increase sensitivity, the project will be expanded to include additional review websites. To improve response rates, DOHMH will offer a link to an electronic survey. Finally, DOHMH is exploring the possibility of linking multiple complaints pertaining to the same restaurant, using data from different review websites and DOHMH databases.
* Available at https://foodborne.smartchicagoapps.org.
Morbidity and Mortality Weekly Report. 2014;63(20):441-445. © 2014 Centers for Disease Control and Prevention (CDC)
