Vanderbilt researchers are developing an artificial intelligence algorithm that can precisely track and count monkeypox lesions. They are also spearheading the effort to develop lesion classification guidelines, which will be used when evaluating the potential use of tecovirimat, an antiviral that is FDA-approved against smallpox, as a therapy for monkeypox.
The work is being led by Dr. Eric Tkaczyk, director of the Vanderbilt Dermatology Translational Research Clinic, assistant professor of dermatology and biomedical engineering and VA staff physician, and Benoit Dawant, Cornelius Vanderbilt Professor of Engineering and director of the Vanderbilt Institute for Surgery and Engineering.
Monkeypox is a disease caused by infection with the monkeypox virus, which has caused significant morbidity in several central and western African countries. There are currently no proven, safe and effective treatments for monkeypox. Currently, tracking the progress of the disease in someone who is infected means manually counting and measuring changes in their lesions—as many as 3,000 of them—until they resolve. This approach is labor-intensive and prone to human error. Tkaczyk and Dawant’s objective is to develop photography and AI tools that enable a reliable evaluation of the severity and progression of monkeypox.
This work is being conducted in collaboration with the Division of Clinical Research at the National Institute of Allergy and Infectious Diseases at NIH and the Institut National de Recherche Biomédicale in the Democratic Republic of the Congo.
A collaboration trajectory
Vanderbilt’s participation in this project is a result of existing collaborations between Tkaczyk and Dawant. Earlier collaborations have focused on developing AI tools to monitor skin rashes that appear after bone marrow and stem cell transplantation. They are developing these tools with Dr. Edward W. Cowen, who leads the NIH Dermatology Consultation Service at the National Institute of Arthritis and Musculoskeletal and Skin Diseases.
“In 2021, Lori Dodd, a colleague of Dr. Cowen, discussed with him the then-rare disease called monkeypox. Considering the challenges they encountered during clinical trials in Central Africa—mainly that manually counting thousands of lesions every day was proving an ineffective strategy—Dr. Cowen mentioned our work on outlining skin rashes,” said Tkaczyk, also a Department of Veterans Affairs Career Development Program awardee. “When Dr. Cowen examined the monkeypox lesion photos and recognized the difficulties the group faced, he got in touch with us to inquire if synergy could be achieved between the technology powering our existing research and what the team handling monkeypox wanted to achieve.”
The existing method is laborious. “The physicians … ended up having to use pens to draw lines and dots on their patients, marking areas and lesions that they were counting, just to keep track,” Tkaczyk said. “It’s a time of need due to the more severe and lethal monkeypox variant in the DRC. Over the past year outbreaks have become much more extensive. The hope is that the global attention to the disease will result in improvements, including trials of new therapies for patients, especially in Central Africa where patients have been suffering the most severe complications.”
Transfer learning to measure monkeypox
As soon as the monkeypox research collaboration was underway, existing monkeypox data was shared with Tkaczyk, Dawant and postdoctoral research fellow Andrew McNeil. In work just published in the Journal of Investigative Dermatology, it turned out that their existing artificial intelligence, developed to quantify rashes in cancer patients, did reasonably well at identifying monkeypox lesions with only modest refinements.
“Leveraging our previous work and training of neural networks we had developed for skin assessment, we did what is called transfer learning,” Dawant said. “We took a deep learning network that was trained for one task, modified and fine-tuned it for this new task. The monkeypox-specific neural network is trained to find and count each lesion on the human body from a photograph, and to categorize those lesions by their maturity. Our collaborators at the Institut National de Recherche Biomédicale in Kinshasa are going to provide the data and images necessary to help the technology reach the stage where it can be used clinically.”
Engineers like to solve problems
In reference to Dawant’s and Tkaczyk’s affiliation with Vanderbilt University and Vanderbilt University Medical Center, respectively, both credit open channels of communication between engineers and physicians for making this and other VISE collaborations possible. “Engineers like to solve problems, and there is always the possibility that some of the work that we have done in the past can be used, applied or modified to solve a new problem,” Dawant said.
Inga Saknite, adjoint assistant professor of dermatology at Vanderbilt Dermatology Translational Research Clinic, has recently traveled to the DRC to help the collaborators set up rigorous photography protocols to capture images. Meticulous photo documentation is essential for AI uses in the future. On the ground, Saknite is also helping the team develop manuals for training Vanderbilt’s AI to ensure accuracy in identifying different stages of monkeypox lesions.
“It is exciting to serve critical roles for the current therapeutic trial,” Tkaczyk said. “In the future, we hope the algorithm utilized here can be broadly applicable even after this trial is complete.”
The initial collaboration was supported by a Discovery Grant from the Office of the Provost at Vanderbilt University to develop a neural network to analyze skin photos in cancer patients and a training grant at VUMC to support Tkaczyk’s career development in clinical oncology applications (K12 CA090625). The promising early findings led to external funding from the NIH grant R21 AR074589 and Tkaczyk’s Career Development Award from the VA (IK2 CX001785).
These funding resources have enabled an expanded research team mentored jointly by Tkaczyk and Dawant, including Saknite, McNeil and recent graduates Bohan Jiang, Walker House, Ziche Chen, Jianing Wang, Fuyao (Joanna) Chen, Austin Cronin and Kelsey Parks (now a Vanderbilt dermatology resident), among many other mentees.
Key external collaborators from the DRC and NIH include Placide Mbala-Kingebeni, Olivier Tshiani Mbaya, Jean-Luc Biampata, Antoine Oloma Tshomba, Edward Cowen, Lori Dodd, Véronique Nussenblatt and Ian Crozier (a former chief resident at Vanderbilt).