Before his son Noah was born, Bryan Shaw and his wife Liz went to Best Buy and picked out an inexpensive digital camera to document memories of their infant son. Months later, they realized their photos had captured signs of a rare cancer in their child’s eyes.
The camera, now scuffed and held together with duct tape, eventually led Shaw, a chemist at Baylor University, to work with his colleague, computer scientist Greg Hamerly, to find a way to use photos to detect signs of eye disease in children.
They eventually created a free app, CRADLE, that scans photos in the user’s smartphone gallery and looks for signs of leukocoria, or “white eye,” a reflection that can indicate eye disease. The app is receiving national attention again, after they published a study on the app’s abilities in the journal “Science Advances.”
People are also reading…
Shaw, an associate professor of chemistry and biochemistry, said parents are already told to report leukocoria to their pediatricians if it shows up in photos, but not everyone knows about the condition.
“We didn’t invent this idea of reporting leukocoria, and you don’t need our app to do it,” Shaw said. “Our app helps them do it. Sometimes, leukocoria is almost impossible to see.”
Noah, who is now 11, was diagnosed with retinoblastoma at four months old in 2007. He visited the U.S. House of Representatives in 2016 with his father when he presented the app during a hearing on apps that serve medical purposes.
“I told him that what we’ve made has helped a lot of kids get diagnosed early, and they didn’t lose their eye the way he did,” Shaw said.
Like many young cancer survivors, Noah gets yearly MRIs and will have a high risk of developing cancer again for the rest of his life. Shaw said his family travels to Boston, Houston and Dallas for visits with endocrinologists, radiation oncologists, regular oncologists and ophthalmologists every year.
Shaw said while his family stays vigilant, Noah has thrown himself into music and school with abandon. He plays cello, piano and bass, loves English and science in particular, and can craft just about anything out of pieces of paper.
“He’s a precocious kids in many ways,” Shaw said. “He’s a really studious guy. He’s a huge reader.”
Bryan’s wife Liz was the first to notice the telltale white reflection in photos the two had taken of Noah. Shaw said his son lost his right eye and required proton beam radiation therapy, which increased his lifetime risk of osteosarcoma.
“Looking back, we saw the white eye started showing up at 12 days old,” Shaw said. “I asked his ophthalmologist for a prognosis had I brought him in at 12 days old. His ophthalmologist said that the tumors would have been so small, we probably would have just treated them with laser photoablation therapy.”
Shaw did not have any experience with computers or ophthalmology, but at the time he was working in a lab focused on diagnostics and approached Hamerly with the idea of automatically scanning photos for leukocoria.
“Initially our idea was to put it in consumer-grade cameras,” said Hamerly, an associate professor of computer science. “We weren’t really getting traction with that.”
They realized building the hardware into the camera would be difficult, and as smartphones took over the world, the path ahead seemed clear by 2011.
“We wanted this available to people who didn’t know it existed,” Hamerly said. “If it was preinstalled, you wouldn’t have had to know about it in advance.”
Students worked on the project as well. Baylor undergrad Mike Munson curated 50,000 photos of children with leukocoria, cataloging them by the subject’s age, the angle the photo was taken, the metadata, and the lighting conditions to determine which photos showed the condition the clearest.
“Collecting the pictures is the most time-consuming part,” Shaw said. “No pictures, no app. The first thing I need to say to the parents is ‘Thank you for the images.’”
Shaw said for the app to work, the photo needs to be taken with flash and indoors, and taking a photo in the dark yields even better results.
“It needs to detect a full face,” Shaw said. “Ninja mask photos don’t work. Don’t just photograph the eyes.”
In 2013, one of Hamerly’s students, Ryan Henning, built the apps as his master’s project. The Android version runs in the background and only alerts a user once leukocoria is detected. The Apple version is not allowed to run in the background, and requires the user to scan their galleries manually. Shaw said ideally, parents would scan the photos once a month or so. The Apple version has been downloaded 91,500 times.
Hamerly said the app uses convolutional neural networks, also called deep neural networks, to “learn” how to detect leukocoria through an endless round of testing and retesting.
“We feed that into the neural network, and what pops out is the probability of being leukocoric or not,” Hamerly said. “The way it learns is we feed into it lots and lots of images of healthy eyes and leukocoric eyes. It processes them, makes a determination at the end, and we tell it how close it was to a correct answer.”
They repeat the process, teaching and testing the network over and over again to improve accuracy. The app currently has a 1% failure rate, partially caused by the false positives that come from optic disc reflection, something they cannot fully escape.
“There is an optic nerve at the back of your eye that can cause that,” Hamerly said. “At just the right angle and lighting condition, they can create the same reflection. We need to bring that down.”
The next version will address it by way of a rate-detector, which will keep the app from alerting parents to instances of leukocoria until multiple instances are recorded. Photos of Shaw’s son showed the condition in 25% of the photos taken within one month. If a child has the condition, about 7% or 8% of pictures will typically show leukocoria.
“If it shows up 10 times in a month, maybe then the app alerts the parents,” Shaw said. “That right there cuts the false positives down from 1% down to 0.1%.”
Now, Shaw and Hamerly are asking for more photos of children with and without the condition to improve the app. Shaw said it is important to include photos of children of every ethnicity possible, to ensure the app learns to detect the disease in everyone, not just white children. Variables including the shape of the eye in the photo and the person’s skin tone can lead to a flawed, biased system over time.
Shaw said the app can also be useful in detecting Coats’ disease, cataracts, retinal issues, glaucoma and even nearsightedness and farsightedness. He said the app’s real impact may be its ability to capture vision issues before a child ever enters pre-K.
“They usually don’t get that detected until they enter school and they learn to read a Snellen chart (the familiar eye-test chart with descending font sizes), or until the teacher realizes they can’t see the board,” Shaw said. “When you’re 1, 2, 3, 4, you might have a severe refractive error and nobody knows. That impacts development.”
Shaw said above all, developers need more photos of children with and without the condition, and he is very thankful to parents who have given a total of about 100,000 photos so far.
“If you’re reading this story and you’re in Waco at the Silos, and you’re from Chicago, or Tokyo, and CRADLE has helped you, please let us know,” Shaw said. “We’d love to know.”