Rules of the Collaboratory Game

If you’re a researcher studying schizophrenia, you can tap marvelous new tools such as functional magnetic resonance imaging and positron emission tomography. You can combine data from these devices for astonishingly powerful new views of how the brain works. What you can’t do is easily integrate data gathered by researchers outside your group.

Enter the Biomedical Informatics Research Network, or BIRN. Funded by the National Institutes of Health, BIRN is a virtual collaboration project for biomedical big science. It aims to let a given research community share its instrumentation, data, software tools, and other resources over very high speed networks. One of the first BIRN testbeds is for schizophrenia researchers, who will pool their images to create a national treasure trove.

BIRN is a prime example of a collaboratory: “an organizational entity that spans distance, supports rich and recurring human interaction oriented to a common research area, and provides access to data sources, artifacts and tools required to accomplish research tasks.” That’s the definition of Gary Olson, professor of human-computer interactions at the University of Michigan and his colleagues in the Science of Collaboratories project, which is backed by the National Science Foundation.

The term first cropped up in the late 1980s. Olson’s project now indentifies more than 200 collaboratories, reflecting the ambitious challenges of today’s science, the extremely expensive instrumentation that it often requires, and the availability of very-high-capacity networks and computing resources. They support research that just couldn’t be done otherwise, emphasizes Mark Ellisman, director of the BIRN coordinating center in San Diego . This trend, he says, “can’t be stopped.”

Collaboratories can be on a grand scale, as in the Human Genome Project, or the ATLAS Project at the European Organization for Nuclear Research (better known as CERN). ATLAS coordinates 1,800 particle physicists in 34 countries.

They also can marry formerly separate threads of research. For instance, the Space Physics and Aeronomy Research Collaboratory, based at the University of Michigan, gives researchers simultaneous access to both observations and predictive models, so they can predict “space weather” (such as the geomagnetic storms that produce aurora borealis events) and then see what actually happens.

But collaboratories often fail. In one early attempt to assemble genetic information, for example, an initiative crafted sophisticated software that didn’t run on researchers’ most common software platform. In another debacle, funders “put up a lot of money to study HIV/AIDS, but in the end people couldn’t figure out how to work together,” Olson says. “There’s always a delicate balance between cooperation and competition.”

The NSF-funded Science of Collaboratories project is creating a software wizard that will let collaboratory planners assess the risks by working through a few dozen questions. The project expects to post the wizard publicly next spring. In the meantime, here’s a working recipe for a successful collaboratory.

1. Make sure your research community is ready: Is it accustomed to operating this way? Particle physicists have been working in teams for decades, a necessity given the huge cost of their instrumentation. (ATLAS will exploit the Large Hadron Collider, an underground particle accelerator ring 27 kilometers around and costing at least $2 billion.) Earthquake engineers, on the other hand, traditionally work within their own labs. As pricetags soar for state-of-the-art lab equipment, research funders are pushing the collaboratory concept, but “the community is having a lot of trouble embracing this model,” Olson says.

2. Tackle big questions: Scientists may realize they need to band together to attack truly tough problems such as genome sequencing or HIV/AIDS. But many lead researchers “still have almost a Depression mentality: ‘You’ve got to hoard everything,’” says BIRN’s Ellisman. That attitude “doesn’t let us get science done as quickly as it might,” he adds. “After you’ve published whatever you’ve learned about your hypothesis, you ought to publish all your data so that other people can hypothesize about it in different ways.”

3. Get each individual participant on board: Individual researchers must be assured that their careers won’t suffer in such broad-scale efforts. The Alliance for Cellular Signaling, a large scale project studying the extraordinarily complex biochemical pathways in which cells interact, tackles this by treating data contributions as publications. There are similar concerns for the talent you need to get onboard to build the technical infrastructure. “In a computer science department, if what you’re doing has practical applications, you’ve fallen from grace,” says Ellisman.

4. Gear up for major technical challenges: Megaprojects such as BIRN’s may juggle dozens of institutions and petabytes of data over a decade or more. They also face unique challenges. For instance, the scanners gathering that schizophrenia data may each come with their own characteristic idiosyncrasies, so researchers must track which scanner produced a given image, and try to find ways to correlate images taken by all those scanners. Even in less ambitious collaboratories, researchers also must be comfortable with collaboration tools that are highly customized or simply new to them. “Not everyone has the same experience with these technologies, which can be pretty daunting,” Olson says. “A lot of the tools are a little clumsy and need a little local support. High-paid scientists just don’t have the patience to deal with something that isn’t working.”

5. Put enough resources into project management: Researchers tend to resist spending money that doesn’t go directly into science. But these complicated projects can benefit from dedicated managers with suitable training and experience.

6. Talk the same talk: The InterMed project, which has standardized clinical guidelines across medical disciplines and settings, required a huge amount of work to establish a common vocabulary Olson says. If participants didn’t agree, say, on what “patient distress” might mean for a heart attack victim, their procedures for dealing with such cases could not be fully spelled out and aligned with each other.

7. Hold your course: You need plenty of patience among the players, especially the funders. A project might take four years to hammer out data access issues, and then run a decade or more. You need visionary planning and stable management to stick it out.

“Collaboration is hard in general, whether you’re doing it online or not,” Olson emphasizes. And it needs the social glue of good relations among participants. No matter how fancy your software, he adds, “the best way to start building a personal relationship with your colleagues is face-to-face.”