Computer Vision Tool Improves the Ability to Distinguish Rheumatoid Arthritis from Osteoarthritis in Damaged Joint Tissue

A new study led by Hospital for Special Surgery (HSS) investigators in New York City has found that their computer vision tool effectively distinguishes rheumatoid arthritis (RA) from osteoarthritis (OA) in joint tissue taken from patients who underwent total knee replacement (TKR). The results suggest the machine learning model will help improve research processes in the short term and optimize patient care in the future. The findings were presented today at the European Alliance of Associations for Rheumatology (EULAR) Congress 2022.

TKR is often the only management option for patients with severe knee joint damage. Identifying which disease caused the joint damage is essential for guiding treatment plans, given that RA is a systemic, inflammatory disease that may also affect the eyes or lining around the heart, while OA affects just the joints. “We know there are many more immune cells present in the synovium, or joint tissue, of patients with RA compared to those with OA,” said Bella Mehta, MBBS, MS, rheumatologist at HSS and lead author of the study. “But precisely how many more has not been clear.”

“Pathologists typically assess images of synovium to determine the extent of inflammation using a combination of approaches, including assigning the level of immune cell infiltration on a scale from 0 to 4,” said Dana Orange, MD, MS, rheumatologist at HSS, assistant professor at Rockefeller University and senior author of the study. “However, these methods are imperfect.” For example, a recent study by HSS investigators found that assessments from two highly experienced pathologists evaluating the infiltration of one type of immune cells known as lymphocytes on the same slides agreed only 67 percent of the time.¹

Drs. Orange, Mehta and colleagues at HSS and collaborating institutions developed and validated a computer vision tool that rapidly counts tens of thousands of cell nuclei in whole-slide images of synovium.² For their present study, they measured 14 different pathologist-scored features in synovium from 60 patients with RA and 147 patients with OA who underwent TKR, and used the computer vision tool to determine cell density.

The investigators identified significant differences between RA and OA features in synovium. The RA samples showed increased cell density; low numbers of mast cells, a type of white blood cell; and lower evidence of fibrosis or scarring compared to the OA samples. The probability of correctly distinguishing between RA and OA in synovium was 85 percent when using the 14 pathologist-scored features alone, 88 percent when using the computer’s score for cell density alone and 91 percent when the researchers combined the pathologists’ scores and the computer’s cell density calculation. The researchers determined a cutoff point for distinguishing RA from OA, determining that synovium containing more than 3,400 cells per mm² should be classified as RA.

“While our innovation is not ready for clinical use yet, it holds promise for assisting pathologists in the future,” Dr. Orange said. “Right now, we see it as a valuable tool for research purposes because it provides an accurate and 100% reproducible score of inflammation and look forward to developing it further.”

Dr. Orange added that in the future computer vision could be trained to glean other types of information from tissue samples, including which types of cells are present and whether they are close enough together that they are likely to be communicating with each other. This more granular assessment might enable clinicians to know more precisely which cells are causing tissue damage and tailor treatments accordingly.

Authors: Bella Mehta, MBBS, MS, Susan M. Goodman, MD, Edward F. DiCarlo, MD, Deanna Jannat-Khah, J. Alex Gibbons, Miguel Otero, PhD, Laura Donlin, PhD (HSS), Tania Pannellini, MD, PhD (Weill Cornell Medicine), William Robinson, MD, PhD (Stanford University), Peter K. Sculco, MD, Mark P. Figgie, MD, Jose A. Rodriguez, MD (HSS), Jessica Kirschmann (Stanford University), James Thompson, David Slater, Damon Frezza (The MITRE Corporation), Zhenxing Xu, Fei Wang, PhD (Weill Cornell Medicine), Dana Orange, MD, MS (HSS and Rockefeller University).

References

1. Orange DE, Agius P, DiCarlo EF, et al. Identification of Three Rheumatoid Arthritis Disease Subtypes by Machine Learning Integration of Synovial Histologic Features and RNA Sequencing Data. Arthritis Rheumatol. 2018;70(5):690-701. doi:10.1002/art.40428

2. Guan S, Mehta B, Slater D, et al. Rheumatoid Arthritis Synovial Inflammation Quantification Using Computer Vision. ACR Open Rheumatol. 2022;4(4):322-331. doi:10.1002/acr2.11381

About HSS

HSS is the world’s leading academic medical center focused on musculoskeletal health. At its core is Hospital for Special Surgery, nationally ranked No. 1 in orthopedics (for the 12th consecutive year), No. 4 in rheumatology by U.S. News & World Report (2021-2022), and the best pediatric orthopedic hospital in NY, NJ and CT by U.S. News & World Report “Best Children’s Hospitals” list (2021-2022). In a survey of medical professionals in more than 20 countries by Newsweek, HSS is ranked world #1 in orthopedics for a second consecutive year (2022). Founded in 1863, the Hospital has the lowest complication and readmission rates in the nation for orthopedics, and among the lowest infection rates. HSS was the first in New York State to receive Magnet Recognition for Excellence in Nursing Service from the American Nurses Credentialing Center five consecutive times. An affiliate of Weill Cornell Medical College, HSS has a main campus in New York City and facilities in New Jersey, Connecticut and in the Long Island and Westchester County regions of New York State, as well as in Florida. In addition to patient care, HSS leads the field in research, innovation and education. The HSS Research Institute comprises 20 laboratories and 300 staff members focused on leading the advancement of musculoskeletal health through prevention of degeneration, tissue repair and tissue regeneration. The HSS Innovation Institute works to realize the potential of new drugs, therapeutics and devices. The HSS Education Institute is a trusted leader in advancing musculoskeletal knowledge and research for physicians, nurses, allied health professionals, academic trainees, and consumers in more than 145 countries. The institution is collaborating with medical centers and other organizations to advance the quality and value of musculoskeletal care and to make world-class HSS care more widely accessible nationally and internationally. www.hss.edu.