Skip to main content
  1. Life
  2. Health & Fitness
  3. Nutrition

How Facebook, Twitter, and Yelp reviews aid foodborne illness source tracking

See also

Social media along with DNA typing help to identify a source of foodborne strep outbreak, according to the CIDRAP article, "Food, Facebook figured in strep throat outbreak." It's important to track social media reviews and complaints to find out where the source of a foodborne illness is coming from. Foodborne sickness also is commonly labeled food poisoning. You also may wish to check out the JAMA news article, "Social Media Helps Sound Alert for Foodborne Strep Throat." Use of social media helped alert public health authorities about a foodborne outbreak of strep throat.

That article explains how one Minnesota parent used a child’s Facebook posts to alert public health officials about a large foodborne outbreak of strep throat. Currently, the CDC is tracking foodborne illnesses through Yelp restaurant reviews by checking out the social media reports/complaints about food poisoning after eating at certain restaurants. But certain criteria have to be met to see whether a person's illness is related to the food or the eatery.

When someone eats at a restaurant or other type of eatery and becomes sick, how do healthcare professionals and other researchers know the source of the food-related contamination that lead to the illness? Social media is one place to look. Have you checked out the website recently of the Infectious Diseases Society of America? Social media is fast becoming a source of information where foodborne illness reporting can be tracked to a possible source.

For example, Facebook posts helped alert public health officials to a strep throat outbreak among a high school dance team in 2012, and DNA fingerprinting led investigators to pasta prepared by a previously ill parent as the likely source. Although strep throat, or Group A Streptococcus (GAS) pharyngitis, usually spreads from person to person by droplets, foodborne transmission is possible, as a report, "Foodborne Outbreak of Group A Streptococcus Pharyngitis Associated With a High School Dance Team Banquet—Minnesota, 2012," published online July 17, 2013, in the journal Clinical Infectious Diseases, found. The most common form of GAS illness is strep throat, but some cases can have more severe consequences, according to the July 18, 2013 news release, "Social media, DNA typing help identify source of foodborne strep outbreak."

Among 63 people who consumed food at a Minnesota high school dance team banquet, 18 came down with strep throat less than three days later

Was the strep throat bacteria in the cooked pasta? When multiple posts soon appeared on the team's Facebook page about ill dance team members and relatives, a parent contacted the state health department. After interviewing approximately 100 people by telephone—those who attended the banquet, household contacts of attendees, and those who did not attend but ate banquet leftovers—and conducting DNA typing of bacterial strains isolated from those who became ill, lead report author Sarah Kemble, MD, and her team of investigators at the Minnesota Department of Health narrowed the possible source of the outbreak to cooked pasta served at the banquet.

The DNA fingerprints of the strep bacteria isolated from the throats of those who became ill matched those of the bacteria identified in the pasta. In addition, one person who became ill and did not attend the banquet, but who ate some of the leftover pasta brought home by family members who did attend, helped confirm how the bacteria was transmitted. This person had a laboratory-confirmed GAS infection that matched the same DNA fingerprint pattern. No one else in the household had symptoms of strep throat, and throat swabs on all the other household members were negative for the bacteria.

DNA fingerprints of bacteria causing foodborne sickness

"We suspect cooked food was contaminated by respiratory droplets from a person who carried the strep bacteria in the throat when the food was cooling or reheating," Dr. Kemble said, according to the July 18, 2013 news release, Social media, DNA typing help identify source of foodborne strep outbreak. "The food probably was not kept hot or cold enough to stop bacterial growth." Both the parent who prepared the pasta and a child in the same household reported having strep throat three weeks before the banquet."Foodborne illness is not limited to diseases that cause vomiting and diarrhea," Dr. Kemble explained, according to the news release. Noteworthy is that when someone talks, whistles, or breathes over food while preparing the food, droplets of saliva invisible to the eye cover the food. The same applies to blowing out candles on a birthday cake. If the bacteria is in the household and in someone's body fluids, just by talking or breathing over the food, especially after it's already cooked, can contaminate the food. Pay attention to that when someone talks to you while preparing food to hand to you such as a sandwich or deli food item, for example.

The rapid communication possible within a large group using online social media played an important role in bringing this outbreak to the attention of a parent, who then contacted the health department, Dr. Kemble said. A more formalized use of social media for disease surveillance and outbreak investigations may have the potential to benefit public health in some circumstances, the authors noted, according to the news release.

Tips for Reducing the Spread of Foodborne Illness

  • Do not prepare food for others if you are ill, especially if you are experiencing diarrhea, vomiting, or have a respiratory infection and are coughing or sneezing. If you are receiving treatment for an illness, ask your doctor how long you should wait after treatment before preparing food for others.
  • When preparing food in large batches (e.g., for large groups of people), ensure the food is kept hot or cold. Disease-causing bacteria grow best in the "temperature danger zone" of 41° F to 140° F.
  • Use a thermometer to ensure that food items are meeting proper temperature requirements.
  • Educational materials for those cooking for large groups are available from the U.S. Department of Agriculture. For further information on educational materials, you may wish to check out the FSIS website.

How the CDC makes use of Yelp (reviews) and other social media so epidemiologists can confirm reports of foodborne illness reporting and track the sources of possible contamination

Many people who review restaurants on social media are familiar with Yelp reviews. Currently, foodborne epidemiologists can confirm reports because Yelp offers a way to follow-up with reviewers for interview. The restaurant review site, Yelp is now being used by the CDC to track food poisoning cases through restaurant reviews that mention foodborne illness, says the May 23, 2014 CDC report, "Using Online Reviews by Restaurant Patrons to Identify Unreported Cases of Foodborne Illness - NYC, 2012-2013." Or you can check out the Fox News article, "Officials use Yelp in food poisoning probes."

Another excellent source of information on how food poisoning is tracked is the site, "Foodborne Chicago: Report Food Poisoning." The website explains, "If you think you have food poisoning in Chicago, please complete this form. The info will be sent to the Chicago Department of Public Health so they can take any necessary action. Did we @reply you on Twitter? Here’s why. We use computers and code to search Twitter for tweets related to food poisoning in Chicago. We do as much as we can to automatically zero-in on the tweets we think are really about a possible food poisoning case and really coming from Chicago. Then real humans review the tweets and reply back to people with a link back to this page. More detail here.

Restaurant reviews used to track cases of foodborne illnesses

How the idea began to look at restaurant reviews in order to track cases of foodborne illness, commonly known as food poisoning, the NY City Department of Health and Mental Hygiene (DOHMH) found that outbreaks of stomach flu associated with eating at certain restaurants, and outbreaks of various types of gastrointestinal sicknesses specifically associated with eating at a particular restaurant are being mentioned in restaurant reviews on Yelp, the social media site for online restaurant reviews.

While investigating an outbreak of gastrointestinal disease associated with a restaurant, the New York City Department of Health and Mental Hygiene (DOHMH) noted that patrons had reported illnesses on the business review website Yelp that had not been reported to DOHMH. To explore the potential of using Yelp to identify unreported outbreaks, the Department of Health and Mental Hygiene (DOHMH) worked with Columbia University and Yelp on a pilot project to prospectively identify restaurant reviews on Yelp that referred to foodborne illness.

During July 1, 2012–March 31, 2013, approximately 294,000 Yelp restaurant reviews were analyzed by a software program developed for the project. The program identified 893 reviews that required further evaluation by a foodborne disease epidemiologist. Of the 893 reviews, 499 (56%) described an event consistent with foodborne illness (e.g., patrons reported diarrhea or vomiting after their meal), and 468 of those described an illness within 4 weeks of the review or did not provide a period.

Health departments might think of using social media to track illness reports from those who post restaurant reviews online

Another site to check out is "The City of New York. NYC 311. New York, NY: The City of New York; 2014." A lot of people don't know how to reach their Department of Health to report foodborne illness after eating in a restaurant. Also, people may be too weak and nauseated from the illness to try to call someone to report the illness until after they feel better, days later. Only 3% of the illnesses referred to in the 468 reviews had also been reported directly to DOHMH via telephone and online systems during the same period.

What the CDC also found upon closer examination determined that 129 of the 468 reviews required further investigation, resulting in telephone interviews with 27 reviewers. From those 27 interviews, three previously unreported restaurant-related outbreaks linked to 16 illnesses met DOHMH outbreak investigation criteria; environmental investigation of the three restaurants identified multiple food-handling violations. The results suggest that online restaurant reviews might help to identify unreported outbreaks of foodborne illness and restaurants with deficiencies in food handling. However, investigating reports of illness in this manner might require considerable time and resources, says the CDD's report, "Using Online Reviews by Restaurant Patrons to Identify Unreported Cases of Foodborne Illness - NYC, 2012-2013."

Project Protocol

Beginning in April 2012, Yelp provided the New York City Department of Health and Mental Hygiene (DOHMH) with a private data feed of New York City restaurant reviews. The feed provided data publicly available on the website but in an XML format, and text classification programs were trained to automatically analyze reviews. For this pilot project, a narrow set of criteria were chosen to identify those reviews with a high likelihood of describing foodborne illness.

Reviews were assessed retrospectively, using the following criteria: 1) presence of the keywords "sick," "vomit," "diarrhea," or "food poisoning" in contexts denoting foodborne illness; 2) two or more persons reported ill; and 3) an incubation period, around 10 hours. Ten hours was chosen because most foodborne illnesses are not caused by toxins but rather by organisms with an incubation period of about 10 hours.

Researchers used data mining software to train the text classification programs. A foodborne disease epidemiologist manually examined output results to determine whether reviews selected by text classification met the criteria for inclusion, and programs with the highest accuracy rate were incorporated into the final software used for the pilot project to analyze reviews prospectively. If you check out the CDC's report, "Using Online Reviews by Restaurant Patrons to Identify Unreported Cases of Foodborne Illness - NYC, 2012-2013," it contains numbered footnote references to studies on these methods of data mining and text classification.

The software program downloaded weekly data and provided the date of the restaurant review, a link to the review, the full review text, establishment name, establishment address, and scores for each of three outbreak criteria (i.e., keywords, number of persons ill, and incubation period), plus an average of the three criteria.

Scores for individual criteria ranged from 0 to 1, with a score closer to 1 indicating the review likely met the score criteria. Reviews submitted to Yelp during July 1, 2012–March 31, 2013 were analyzed. Reviews that met all three of the CDC's criteria were then investigated further by DOHMH. In addition, reviews were investigated further if manual checking identified multiple reviews within 1 week that described recent foodborne illness at the same restaurant.

To identify previously reported complaints of foodborne illness, reviews were compared with complaints reported to DOHMH by telephone or online at 311, New York City's nonemergency information service that can be used by the public to report suspected foodborne illness

Yelp reviews categorized as indicating recent or potentially recent illness were compared with complaints from the previous 4 weeks in the 311 database. To follow up with reviewers, DOHMH created a Yelp account to send private messages to reviewers' Yelp accounts. Reviewers needed to log in at Yelp to view their messages.

For reviews not requiring further investigation and not found in the 311 database, DOHMH sent messages advising reviewers of the availability of 311 reporting. For reviews requiring further investigation, DOHMH sent messages requesting telephone interviews. Reviewers consenting to interviews were asked to provide details about the restaurant visit, meal date, foods consumed during the meal, party size, illness symptoms, and a history of foods consumed in the 3 days before symptom onset.

Review-Based Findings

During July 1, 2012–March 31, 2013, the software system screened approximately 294,000 reviews and identified 893 with an average score of ≥0.5, indicating possible foodborne illness (Figure). Of these reviews, 499 (56%) described an event consistent with foodborne illness, as determined by the manual checking of a foodborne epidemiologist. This equated to an average of 23 reviews evaluated by a foodborne epidemiologist each week, with an average of 13 reviews categorized as consistent with foodborne illness. The remaining 394 (44%) reviews contained keywords but did not suggest foodborne illness. For example, some keywords were: "I didn't get sick at all after my meal."

Of the 499 reviews describing an event consistent with foodborne illness, 468 (94%) indicated recent or potentially recent illness. Of these 468 reviews, only 15 (3%) were also reported to 311 during the same period. A total of 339 reviews that indicated only one person became ill and had no scombroid poisoning or severe neurologic symptoms were excluded, leaving 129 reviews that required further investigation (Figure).

Of the 129, a total of 27 (21%) reviewers completed a telephone interview inquiring about meals and illnesses. The median time from review date to DOHMH contact to schedule a telephone interview was 8 days. The interviews provided information on 27 restaurants, and 24 restaurants were identified as potential locations of recent exposure because the meal dates were within 4 weeks of the interview.

From the 27 interviews, DOHMH determined whether the complaints warranted an outbreak investigation by considering the following criteria: 1) more than one person became ill, 2) no other common meals were suspected, 3) ill persons lived in different households, and 4) the cases had similar onset periods (indicating a likely foodborne cause rather than person-to-person transmission). For scombroid poisoning or neurologic symptoms, DOHMH considered whether symptoms and onset were consistent with scombrotoxin, ciguatera toxin, or botulism poisoning.

Three outbreaks meeting DOHMH outbreak investigation criteria were identified, accounting for 16 illnesses not previously reported to DOHMH.

Interviews with reviewers identified likely food items associated with illness at each of the three restaurants: house salad, shrimp and lobster cannelloni, and macaroni and cheese spring rolls (Table)

The reviews of the three restaurants had been posted on Yelp 2–5 days after the meals. Environmental investigations were conducted at two of the three restaurants during the week after the interviews; a routine DOHMH inspection had already been conducted at the other restaurant 2 days after the meal. The two investigations and the routine inspection identified multiple violations at each of the outbreak restaurants (Table). Investigators were unable to obtain laboratory data that might have identified the infectious agents.

In a New York City DOHMH pilot project, of 468 recent or potentially recent online foodborne illness complaints posted on Yelp and reviewed by foodborne epidemiologists, three previously unreported restaurant outbreaks were identified. Because foodborne cases have a common exposure, a restaurant patron review-based system can identify small, point-source outbreaks that are not easily found by systems reviewing large sources of data, such as syndromic surveillance of emergency department visits. For more sources of information on this type of finding, you may wish to check out the article, "CDC. Three years of emergency department gastrointestinal syndromic surveillance in New York City: what have we found? In: Syndromic surveillance: reports from a national conference, 2004. MMWR 2005;54(Suppl):175–80."

People also looked at the website for Google Flu Trends.

You can read more about this type of searching for trends in the study, "Google trends: a web-based tool for real-time surveillance of disease outbreaks." The research is published in the journal Clin Infect Disease, November 15, 2009. Authors are Carneiro H, and Mylonakis E.

Google Flu Trends can detect regional outbreaks of influenza 7-10 days before conventional Centers for Disease Control and Prevention surveillance systems, says the study's abstract. People also search Twitter for public health trends. You may wish to check out the paper, "Which restaurants should you avoid today?" In: Proceedings of the First AAAI Conference on Human Computation and Crowdsourcing, November 6–9, 2013; Palm Springs, California.

In this project, only 15 (3%) of the 468 recent or potentially recent illnesses identified on Yelp were also reported directly to New York City's nonemergency 311 service, suggesting that knowledge about 311 reporting is limited. Of further note, after messages regarding the availability of 311 were sent to 290 reviewers who did not meet the project criteria, 32 responded, of whom 25 (78%) said they were unaware of the 311 system or would keep 311 in mind for the future. The 311 service receives approximately 3,000 food poisoning complaints each year, and from that number, about 1% are identified as outbreak-related (DOHMH, unpublished data, 2014). For further information about this, you may wish to check out the report about the NYC 311 project: "The City of New York. NYC 311. New York, NY: The City of New York; 2014."

As social media usage continues to grow among U.S. adults, health departments might consider additional surveillance methods to capture illness reports from those more likely to post a restaurant review online than to contact a health department. By incorporating website review data into public health surveillance programs, health departments might find additional illnesses and improve detection of foodborne disease outbreaks in the community.

Similar programs could be developed to identify other public health hazards that reviewers might describe, such as vermin in food establishments. For more information on the use of social media to track foodborne illnesses, check out the report from the Pew Internet research project: The Web at 25 in the U.S.; 2014. Authors are Fox S, and Rainie L, Pew Research Center.

The findings in this report are subject to several limitations, says the CDC's report, "Using Online Reviews by Restaurant Patrons to Identify Unreported Cases of Foodborne Illness - NYC, 2012-2013"

To increase the likelihood of identifying true foodborne illness, researchers chose a narrow focus for the individual criteria used to score reviews. Therefore, it is possible that some foodborne illnesses were not picked up by the screening software because of low average review scores, for example, because of illnesses resulting from toxins with short incubation periods.

Personal contact information for reviewers was unavailable, requiring reviewers to check their Yelp accounts and provide a telephone number to participate, which extended the time from review to interview and might have affected the response rate. Investigators then were not able to identify any of the infectious agents in the outbreaks.

The system required substantial resources. In addition to programming expertise, staff members were needed to read reviews, send e-mails, interview reviewers, and perform follow-up inspections, according to the CDC's report, Using Online Reviews by Restaurant Patrons to Identify Unreported Cases of Foodborne Illness - NYC, 2012-2013."

Additional work using social media might improve health department abilities to use the Internet for disease detection

Working with the Chicago Department of Public Health, the Smart Chicago Collaborative recently developed a system to contact those who post foodborne illness complaints either on its website or on Twitter. For health departments looking for an alternative to analyzing review data weekly, creating an illness-reporting vehicle such as the Utah Department of Health's "I Got Sick" website could be a more practical solution, although it might be less widely used than a review website such as Yelp. Another site where people can report a foodborne sickness that you may wish to see is, "Report a Foodborne Illness - Utah Department of Health."

Review websites could assist by offering a link to the reviewer's local health department's reporting system at the time of review posting. For more information, see, the Utah Department of Healthsite, "Report a foodborne illness." Salt Lake City, UT: State of Utah, Utah Department of Health; 2012.

DOHMH plans to continue to refine this project. To shorten the time from review to investigation, Yelp will provide daily instead of weekly review feeds, and, to increase sensitivity, the project will be expanded to include additional review websites.

To improve response rates, DOHMH will offer a link to an electronic survey. Presently, DOHMH is exploring the possibility of linking multiple complaints pertaining to the same restaurant, using data from different review websites and DOHMH databases.

You also may wish to see related sites such as: "New York City Department of Health and Mental Hygiene; 2CDC/CSTE," check out, "Using Online Reviews by Restaurant Patrons to Identify Unreported Cases of Foodborne Illness." Or see, "You are what you tweet: analyzing Twitter for public health. In: Proceedings of the Fifth International AAAI Conference on Weblogs and Social Media, July 17–21, 2011; Barcelona, Spain. Palo Alto, CA: AAAI Press; 2011:265–72. Paul M, Dredze M. Or you also may wish to view, "nEmesis: which restaurants should you avoid today?" In: Proceedings of the First AAAI Conference on Human Computation and Crowdsourcing, November 6–9, 2013; Palm Springs, California.

Advertisement

Life

  • ESPYS party
    Athletes and celebrities hit the red carpet at Body at the ESPYS party
    Today's Buzz
  • Healthy balance
    Keeping a healthy balance is so important for your own well-being
    Camera
    15 Photos
  • Apollo
    NASA observes 45th anniversary of Apollo 11: One giant leap for mankind
    Camera
    14 Photos
  • Unique jewelry
    Nothing catches the eye or completes and outfit like a great piece of jewelry
    Camera
    12 Photos
  • Student Farmworker Alliance
    Student-Farmworker Alliance launches 'Boot the Braids' web site
    Camera
    15 Photos
  • Exercise
    Find out why you should exercise to compliment your diet
    Camera
    10 Photos

Related Videos:

  • Don't tell children that veggies are good for them says new study.
    <div class="video-info" data-id="517095486" data-param-name="playList" data-provider="5min" data-url="http://pshared.5min.com/Scripts/PlayerSeed.js?sid=1304&width=480&height=401&playList=517095486&autoStart=true"></div>
  • Expired meat repackaged and sold with new dates: New China food scandal.
    <div class="video-info" data-id="518330256" data-param-name="playList" data-provider="5min" data-url="http://pshared.5min.com/Scripts/PlayerSeed.js?sid=1304&width=480&height=401&playList=518330256&autoStart=true"></div>
  • Future farmers and retiring farmers.
    <div class="video-info" data-id="518019324" data-param-name="playList" data-provider="5min" data-url="http://pshared.5min.com/Scripts/PlayerSeed.js?sid=1304&width=480&height=401&playList=518019324&autoStart=true"></div>