*Sehjin Han, MD
Co-founder, Human Practice
Oral Presentation – Research Track
Saturday, Sept 29, 2012: 2:35 PM – 2:55 PM – LK120
*Presenting Speaker
(Research in Progress)
Background
Most discussion (in the literature and otherwise) of online physician reviews focuses on whether they are positive or negative. Few researchers mention the important fact that physicians receive a relatively small number of reviews (Lagu et al., J Gen Intern Med, 2010, or Lieber, “The Web Is Awash in Reviews, but Not for Doctors. Here’s Why.,” New York Times, March 9, 2012).
If this trend continues, then in a fundamental way online reviews have failed in their mission to provide greater transparency for patients and should be replaced with something else. It is surprising that no researchers to our knowledge have suggested this potential consequence of the available data.
Understanding in more detail the nature of and underlying reasons for this scarcity is important, both to confirm the trend more thoroughly, and because a deeper understanding could aid the design of better tools that provide more value to patients and physicians.
Methods
In this study we extend the work of Lagu et al. in several ways.In each of four locations (Boston, Chicago, Houston, San Francisco), 500 physicians that meet study selection criteria will be selected at random using data from the NPI database and state medical licensing boards. The five most popular reviews sites will be selected using Alexa.com daily visitor data and a survey of review prevalence from a subsample of the physicians.
The number of numerical ratings and narrative reviews on each of these sites will be tallied for each physician. We will compare review prevalence across cities (normalized by population size), allowing us to examine whether doctor review scarcity occurs everywhere or only in some locations.
Second, we will use available data on the date reviews were made to study time variation in the number of reviews per year. This will allow us to determine whether reviewing is becoming more popular, and whether the scarcity gap is likely to close in the future.
Third, we will use Fisher’s exact test to determine whether certain categories of medical specialties (e.g. primary care vs specialty care) get more reviews than others. Again our goal is to determine whether scarcity is uniformly a problem, or only a problem for certain specialties. Lagu et al. found no statistically significant difference, so we will include a greater number of reviews for more statistical power.
Finally, preliminary data suggests that some physicians have far more ratings than other physicians, i.e. there are “spikes” in the distribution. Using a mixture of graphical comparison and statistical modeling (comparing the data to a naïve Poisson process), we will examine this phenomenon and determine whether modeling can help pick apart underlying effects.
Results
We have completed a preliminary study in one city (Chicago) using 100 physicians. These data suggest that review scarcity continues to be a problem, though less pronounced than in the Lagu data taken in Boston in 2009. More details await further study.
