Building better products: how we learn at Meta
Prototyping in collaboration with researchers
By Burcin Ikiz, Ph.D. & Michael Czerwinski, Ph.D.
Our goal of making Meta more powerful and useful to researchers requires a deep understanding of their knowledge discovery needs. Over the past year, we learned that these needs extend beyond the scientific literature, i.e., journal articles and preprints, to a broad set of resources that better reflect the breadth of scientific output created and disseminated by the community. To this end, we’ve been working to expand and enhance Meta both in the content we cover, as well as ways in which researchers access and use the data.
Late last year, we collaborated with a fantastic and generous group of researchers around the globe to demonstrate and validate the value of these new resources, which included protocols, datasets, and software/code, through a particular use case common across biomedicine. Together we co-designed and prototyped how best to identify, aggregate, organize, and link the most complete set of resources and output types in support of a specific field.
Why prototype? We run these experiments to test or validate ideas, design assumptions, and other aspects of the conceptualization so our team can make refinements or even reorient based on feedback from the research community.
To address this challenge quickly, we focused the prototype on an emerging and narrow field of biomedical research with a limited but growing publication history: brain organoids. This allowed us to tap into the expertise of our CZI Science colleagues and scientists at the cross-section of single-cell biology and neurodegeneration; it also made it possible to define and connect with select data sources and play with new machine learning techniques to create a high-definition knowledge graph in a rapid manner.
Specifically, we set out to address one of the biggest challenges researchers face: entering a new field. This is a common need, whether one is a graduate student starting a thesis, an assistant professor building a lab, or a principal investigator writing a grant. It can be overwhelming and time-consuming to dive deep into the literature and find the right research papers, reviews, and protocols to get up to speed with an unfamiliar topic.
This effort brought together brain organoid-related data from more than 8,000 journal articles and 100 preprints, dozens of datasets, protocols, codes, and compounds into a single resource.
Prototyping a researcher’s guide to brain organoids
When Meta’s internal scientists and user experience specialists worked closely with a set of brain organoid researchers, we found three major challenges in their knowledge discovery needs:
- Feeling confident about identifying the right papers to get up to speed with brain organoid research;
- Efficiently comparing and finding protocols, locating collaborators, and figuring out what tools or compounds to use;
- Easily accessing relevant data sets and repositories.
To address these challenges, our team organized a co-design workshop with brain organoid researchers from four continents and seven countries. Working directly with researchers helped us to better understand their needs and identify potential solutions. The workshop helped these researchers, too: Many commented on how wonderful it was to have the brain organoids community come together to work on solutions to their common struggles.
Based on learnings from the workshop, Meta’s design and engineering teams worked together to identify and ingest the most relevant content to develop the prototype. This effort brought together brain organoid-related data from more than 8,000 journal articles and 100 preprints, dozens of datasets, protocols, codes, and compounds into a single resource.
The prototype went a step further: It highlighted useful information from the knowledge graph, including the most-cited papers, methods, and protocols, most-published researchers, and most-mentioned clinical terms, to help researchers new to the field get oriented. It also provided a global snapshot of brain organoid research by mapping publications from research centers developing and applying brain organoid technology around the world.
For researchers who wanted to dive deeper, the prototype included a search engine that allowed users to look for specific papers, concepts, and resources.
In addition, Meta’s data scientists created an algorithm that could identify mentions of methods, genes, proteins, and compounds, as well as associated protocols and software, within a paper and display those associated resources. At a glance, researchers could quickly assess the relevance of a paper and decide whether it was worth their time to read it in full.
Next Steps
The development of the prototype was a valuable learning experience. It helped us understand how best to ingest and process data across a wide variety of repositories and content types, employ a mix of programmatic and expert curation to review the data, and understand usability challenges in making the content available to researchers. We look forward to applying what we learned in this process to further diversify Meta’s knowledge graph and to expand the search and discovery capabilities on Meta.org.
We welcome you to play with the results of our experiment here. (As with all prototypes, please pardon the dust in the data and in the display — this is still very much in an exploratory state.)
We would like to once again extend our thanks to all the brain organoid researchers who gave their time and expertise to help Meta with this research project.
Originally published at https://medium.com on January 30, 2021.
