Bench Sci marries a database of 6.9 million commercially available antibodies with a machine learning algorithm that searches scientific literature for relevant, high-affinity antibodies.Credit: MF3d/Getty Images
Getting through graduate school is hard enough. But Tom Leung encountered a particularly frustrating obstacle while studying epigenetic changes in cultured cells during his PhD at the University of Toronto. “I had to stay in the lab for 12 hours at a time to collect all the samples, and in the end, when I ran the western blot to analyze my results, it didn’t work,” he says. “It wasn’t because I did anything wrong in the experiment. It was because the antibody quality was poor.”
Leung’s experience is common. A 2008 high-profile, proteome-scale study of antibody performance from 2008 found that nearly half of the thousands of reagents tested by the authors did not deliver the expected affinity or specificity. Leung wanted a solution, and along with University of Toronto colleagues, Elvis Wianda, David Chen and Liran Belenzon, founded BenchSci, an artificial intelligence (AI) platform that allows researchers to search for optimal reagents based on figures from published experiments and make data-driven antibody choices.
The burden of choice
There are millions of antibodies available, sold by hundreds of vendors. One recent article reports that more than 5,000 antibodies exist for the human epidermal growth factor receptor protein (EGFR) alone. “Scientists know that every antibody is not going to work in every experimental context,” says Casandra Mangroo, Head of Science at BenchSci. “Even if the vendor has done some sort of testing, they don't have the capacity to test every antibody in every single experimental context.”
David Rimm, a pathologist at Yale University, has been a vocal advocate for antibody quality testing and validation. In 2016, he was part of a push to get makers of antibodies to agree to a set of best practices for quality control.
“At first, they all agreed to a sort of ‘scoring system’, but after thinking about it for a bit, they changed their mind,” he says. Without a clear-cut way to confidently rank and compare antibodies, scientists can only rely on the data in the literature, but extracting this information is labor intensive.
“PubMed and Google Scholar let you quickly look for results, and you can find a lot of papers,” Leung says. “But if you have to go down to the supplementary data, or one of the figures, to find a reagent, it’s not that easy.”
Matchmaking for antibodies
BenchSci’s AI-Assisted Antibody Selection platform automates this work, scouring text and figures in the literature to identify antibodies that might support a particular experiment. Leung notes that the company initially benefited from the wealth of publications in repositories such as PubMed Central.
“We were able to get a lot of high-tier journals and open-access journals and then use them to train a machine learning model,” he says. “Then later on, we forged a lot of partnerships with different closed-access publishers.” Springer Nature, which publishes Nature, Wiley, and Wolters Kluwer, are among the publishers that have since agreed to share article data with BenchSci.
Leung says that BenchSci’s timing was also fortuitous. “If this idea was hatched two years earlier, it probably wasn't going to work because deep learning and machine learning were not as mature yet,” he says. It helped that Chan and Wianda had extensive familiarity with these tools from their own graduate research.
BenchSci’s database contains 6.9 million antibodies from over 220 vendors, with coupled data drawn from 10 million research papers. That dataset is continually growing, both in the number of reagents and journals, as are the demands on BenchSci’s computational capacity. “We do monthly runs to update data on the platform and train our algorithms. We basically had to create a whole wall of computers to do that,” Mangroo says.
When scientists search BenchSci’s database for antibodies against a particular protein, the AI assembles a simple set of figures depicting the use of various products in different experimental contexts. One can just look for antibodies that have been used in immunohistochemistry, ELISA or flow cytometry experiments, and then assess the performance of different reagents in different studies. Today more than 15 of the top 20 pharmaceutical companies are using the BenchSci platform, as are more than 31,000 scientists at 3,600 institutions.
Rimm has found the platform quite valuable. “Because it gives you access to the figures in which an antibody was used, you can have your own criteria for what you accept as validation,” he says. “It just saves a lot of time spent digging in the literature.”
In addition to being helpful for product selection, Rimm says the platform can also streamline the experimental process. If, for example, BenchSci uncovers a published track record for a given antibody in a particular validation assay, researchers can cite prior work rather than wasting effort repeating the experiment. Conversely, Mangroo says that image search makes it easy to recognize reagents that are low quality or poorly matched for a given assay, should a figure show results that are inconsistent with other experimental data.
Network effects
Many antibody manufacturers have now lined up behind BenchSci’s efforts, sharing their catalogues and associated validation data for incorporation into the company’s database. Rimm says that even though these reagent manufacturers were hesitant about universal validation standards, many still recognized the need for better quality control. “The system sort of policed itself, and made the competitive marketplace the scoring system, where many vendors competed with each other in who could show the most validated antibody,” he says.
BenchSci builds on that sentiment. Users benefit from access to validation data while antibody companies benefit from having their products directly linked with successful published experiments. Importantly, BenchSci remains a neutral platform in terms of recommendations, allowing the data to speak for itself. “I like that BenchSci doesn’t have skin in the game,” Rimm says. “They don’t advertise or rate antibodies.”
The ability to make informed decisions about antibodies can reduce waste and lost time, but antibodies are only one part of the reproducibility problem. Researchers also rely on a host of other reagents, including molecular probes, protein-specific inhibitors and activators, and primers for sequencing and PCR amplification. Leung ultimately hopes to turn BenchSci’s AI platform toward this broader constellation of products as well.
“By linking these other reagents together, we can provide a much more complete picture of what has transpired in different publications,” Leung says. “Then we'll be able to help scientists in a much fuller regard to planning their experiments.”