The online review platform Yelp is an excellent resource to see how restaurants or other local businesses are rated, but new research shows how it can also provide insights on essential public health services.
Researchers from the University of Pennsylvania pulled a decade’s worth of Yelp reviews from 2014 to 2023 on essential healthcare organizations in the US. These are organizations such as urgent care centers, doctor’s offices, diagnostic facilities, and hospitals that provide services health insurance plans must cover under the Affordable Care Act.
The total tally was 1,445,706 online reviews from 151,307 facilities nationwide, which were then sorted by zip code. The researchers’ findings were published on November 22 in the journal JAMA.
Dropping customer satisfaction. In general, the researchers observed a sharp decrease in positive online reviews (four or five stars)—from 54.3% to 47.9%—correlated with the start of the Covid-19 pandemic.
The data found institutions in rural areas and areas with a larger Black population were disproportionately affected. In Q1 2020, urban areas had around 51.62% positive ratings and rural areas had around 46.52%. Ratings for both areas dropped to a record low in Q4 2021: approximately 43.32% and 32.71%, respectively.
During this same period, facilities located in areas with the lowest proportion of Black residents saw positive ratings fall from 56.23% to 48.97%. Facilities in areas with the highest proportion of Black residents saw ratings fall from from 43.14% to 33.29%, according to the study.
“Before March 2020…over 50% of reviews were four or five stars,” Neil Sehgal, a doctoral student in the department of computer and information science at the University of Pennsylvania School of Engineering and Applied Science and the lead author of the study, told Healthcare Brew. “It does look like [the positive review rate] going back up, but it’s still not to where it was before Covid.”
Navigate the healthcare industry
Healthcare Brew covers pharmaceutical developments, health startups, the latest tech, and how it impacts hospitals and providers to keep administrators and providers informed.
Seghal and his colleagues also examined the content of the reviews in hopes of pinpointing problems that might be associated with the drop in patient satisfaction.
The findings suggested that insurance and billing issues seem to have increased a lot in predominantly Black neighborhoods, Sehgal said. When the researchers analyzed how many times keywords were mentioned in negative reviews, “insurance and billing issues” had the largest regression coefficient, or increase in mentions over time, at around 0.13. Issues like “customer service and staff behavior,” for comparison, had a regression coefficient of 0.05.
Yelp reviews can capture data and insights around the qualitative aspects of patient experience that could be missed through traditional reporting metrics run by healthcare systems or government agencies, according to Sehgal. Researchers from the Veterans Affairs Medical Center and the University of Pennsylvania, for example, previously used Yelp data to understand experiences of racism at hospitals, highlighting where instances of racism can occur in a healthcare setting and who it can affect.
“What’s unique about online reviews is that they capture this raw, unfiltered feedback that traditional surveys might miss, like personal stories or frustrations. And so they hopefully give us a broader picture,” Sehgal said. “They definitely don’t replace traditional surveys—they just complement them.”