Washington, March 2, 2021—A first-of-its-kind analysis of parents’ reviews of U.S. public K–12 schools, posted primarily from 2009 to 2019 on the popular school information site GreatSchools.org, found that most reviews were written by parents at schools in affluent neighborhoods and provided information that correlated strongly with test scores, a measure that closely tracks race and family income. Language associated with school effectiveness, which measures how much students improve in their test scores over time and is less correlated with demographics, was much less used. The research was published today in AERA Open, a peer-reviewed, open access journal of the American Educational Research Association.
“Our results reveal the large weight that parents in this timeframe placed on test scores as a measure of quality,” said study coauthor Nabeel Gillani, a doctoral student at the Massachusetts Institute of Technology. “Parents seemed to value schools based on current students’ performance, not growth over time, which perhaps reflects the longstanding focus of education policies on test scores as a primary marker of school quality.”
“School rating websites have come under scrutiny for ratings systems that overemphasize test scores,” said Gillani. “Now we’ve found that subjective online parent reviews can do the same.”
“Further research is needed to understand whether reviews reflecting school demographics have the potential to exacerbate neighborhood segregation or other conditions that impact access to high-quality education,” Gillani said.
For their study, Gillani and his coauthors, Eric Chu, Doug Beeferman, and Deb Roy, also at MIT, and Rebecca Eynon, at the University of Oxford, applied recent advances in natural language processing to analyze about 830,000 reviews of more than 110,000 schools that were posted by self-identified parents primarily from 2009 to 2019 on GreatSchools.org. Their analyses identified and measured the prevalence of words and phrases that correlated with different measures of school quality—namely, test scores and student learning progress measures—along with school demographics like race and family income, drawn from the Stanford Education Data Archive.
Schools in urban areas and those serving affluent families were found to be more likely to receive parent reviews, and there were clear differences in the review language used by parents of children at majority-White vs. minority-White, and high-income vs. low-income schools.
The authors found that many of the words and phrases that were statistically associated with test scores also conveyed information about the racial and income makeup of schools. For example, words like “PTA” and “emails” were more often used in descriptions of schools with smaller percentages of students receiving free or reduced-price lunch and those serving a smaller percentage of non-White students, as were reviews using the terms “small school,” “special needs,” and “IEP” (Individualized Education Plan).
“Overall, parents’ reviews tended to focus on topics that are associated with race and income in school systems,” said Roy, who directs MIT’s Center for Constructive Communication. “Wording such as ‘the PTA,’ ‘emails,’ ‘private school,’ ‘we,’ and ‘us’ are predictive of test scores, reflecting the tendency of more affluent, non-minority parents to have dual-parent households, digital connectivity, more schooling options, and more time to be involved and communicate regularly with teachers.”
“These results reveal the subtle and sometimes hidden patterns in the words we use, sending signals and encoding biases that pervade our social realities,” Roy said. “New techniques from machine learning applied to large data sets describing human behavior can help make those patterns visible.”
The findings offer evidence that parents from lower-income, minority schools may have fewer voices to learn from.
“Unfortunately, many of these parents are not always tapped into social networks where they can readily receive guidance that helps them identify and select the best schools for their children,” said Gillani.
According to the authors, rating sites should encourage parents and stakeholders to value growth as a measure of school quality. They should also adopt strategies to capture a more inclusive cross-section of parents’ perspectives about schools and prompt insights that reflect how well schools help children learn and grow.
The analysis does not explore how parent reviews may have changed as a result of GreatSchools’ methodological changes in recent years. GreatSchools updated its rating system in 2017 to include new measures of school quality and again in September 2020, after the study was conducted, to emphasize educational equity and give more weight to student growth. GreatSchools and MIT are pursuing a data and research partnership to continue examining how its site is being used and to explore modifications to mitigate potential inequity issues.
“It is critical to uplift the voices of all parents, especially those in traditionally underserved communities, to highlight their experiences within a school community,” said GreatSchools CEO Jon Deane. “As we continue to evolve measures of school quality, including prioritizing academic growth over test scores, we value working alongside researchers to find new ways to more equitably serve all parents.”
The authors suggest that parents and the public be mindful of how much weight to place on subjective assessments offered by other parents about any given school—and to be mindful of how ratings sites may factor these subjective assessments into their overall scores for schools. (GreatSchools does not factor subjective assessments into its overall ratings.) The authors also recommend that school administrators foster a culture where all parents’ voices are valued and parents are encouraged to share their holistic views about the quality of education their children receive.
“One of the goals of school rating sites is to use available data to democratize access to information about school quality,” said Gillani. “However, the school choice market is only as good as the information available to consumers. We need more representative voices talking about a more holistic set of topics if we want to maximize opportunities and outcomes for all students, especially those from less privileged backgrounds.”
Study citation: Gillani, N., Chu, E., Beeferman, D., Eynon, R., Roy, D. (2021). Parents’ online school reviews reflect several racial and socioeconomic disparities in K–12 education. AERA Open. Published March 2, 2021. https://doi.org/10.1177/2332858421992344
###
About AERA
The American Educational Research Association (AERA) is the largest national interdisciplinary research association devoted to the scientific study of education and learning. Founded in 1916, AERA advances knowledge about education, encourages scholarly inquiry related to education, and promotes the use of research to improve education and serve the public good. Find AERA on Facebook, Twitter, and Instagram.
