The problem with computational biology papers

OK, my title is too general - it should be, "The problem with some computational biology papers that deal with certain research questions." There is a type of trendy science that frequently crops up in many journals (including good ones like Nature and Science. It basically goes like this (for a prime example, look here):

1. A computational biology lab sees one or more genomic-scale datasets that they can do some calculations on (usually microarray data).

2. The computational biologists come up with some algorithm that's supposedly better than what's out there, and they crunch the numbers on the genomic data. This results in some predictions of novel regulatory interactions - for example, they predict that certain transcription factors regulate certain genes involved in cell division. At this point we have no idea whether their predictions are right, or even persusive enough to be worth testing. But it's a start.

3. The computationl biologists "validate" their results by using (notoriously incomplete) database annotations about the genes in their predictions, or by a shallow, cursory scan of the experimental literature (which the authors are usually not that familiar with). They then state something like "75% of our predicted transcription factor-gene interactions have some basis in the literature." Up to this point things are fine (they have made predictions, and given us some reason to believe that the predictions have a chance of being right), but then they usually go on and say something like this: "Therefore, we have demonstrated that our algorithm has the ability to find new regulatory interactions..." They have demonstrated no such thing. They have made predictions, but haven't bothered to test them; instead, they do a crappy literature survey (usually with significant omissions).

The result is that you get different groups coming up with all sorts of new analyses of the same genomic data (in my field, cell cycle gene expression and genome-wide transcription factor binding data are big ones), but never really making any serious progress towards improving our understanding of the biological process in question. The worst part is that, over time, the researchers doing this kind of work start talking as if we are making progress in our understanding, even though we haven't really tested that understanding. You start getting an echo chamber resonating with these guys who are citing each other for validation more than they are citing the people actually study the relevant genes in the lab.

This means that the experimentalists ignore the echo chamber, and then computational biology becomes irrelevant to experimental biology - which is a sad thing. There are so many 'validated' predictions out there, that the experimentalists don't really know where to start, where the good predictions are. And the computational researchers don't care enough to really work with someone who will actually go test things in the lab, in spite of the fact that if these computational biologists did care enough, they would get more notice from the experimentalists.

The problem is bad enough that one journal, Nucleic Acids Research changed their policy on computational papers:

"Computational biology
Manuscripts will be considered only if they describe new algorithms that are a substantial improvement over current applications and have direct biological relevance. The performance of such algorithms must be compared with current methods and, unless special circumstances prevail, predictions must be experimentally verified. The sensitivity and selectivity of predictions must be indicated. Small improvements or modifications of existing algorithms will not be considered. Manuscripts must be written so as to be understandable to biologists. The extensive use of equations should be avoided in the main text and any heavy mathematics should be presented as supplementary material. All source code must be freely available upon request."

This is a move in the right direction. But until more journals adopt this stance, beware of researchers who claim to have calculated the gene regulatory network for this or that process, or have identified 'modules' of interacting proteins that perform a function in the cell. If these claims, usually based on noisy, less than ideal genomic data, haven't been tested with serious experiments, they remain unproven hypotheses.

A Republican War on Science? Nature's editors cop out

The Oct 19th issue of Nature contains a feature section on science and the upcoming US Conrgessional elections. In one of the editorials, (subscription required) Nature's editors criticize the phrase "Republican war on science":

"Slogans such as the 'Republican war on science', meant to sum up a host of perceived abuses, do not do justice to the complex relationship between science and each of the two major political parties."

In an effort to not be percieved as partisan, some people, Nature's editors included, can't bring themselves to truly call things as they are. In recent years, there are very good reasons to single out the Republican party for its serious corrosion of the US government's relationship with science. The phrase "Republican War on Science" (the title of Chris Mooney's recent book) is a correct, legitimate characterization for the following reasons:

1. It's true that no political party or presidential administration is monolithic. There are many Republicans who are not part of an assault on the integrity of science, and not every single decision made by the Bush adminstration has been bad for science. However, the Republican leadership in Congress and the Executive Branch, as well as the active members of the Republican base, have seriously abused science and scientists to push their ideological agenda. Whether it's pushing intelligent design (from school boards to the 'Santorum Amendment' of the No Child Left Behind Act), having an ex-physician novelist testify to Congress on climate change, diagnosing Terri Schiavo by video from Congress, or the more low-profile but perasive agency decisions that weaken protection of the environment and endangered species, promote ineffective and sometimes inaccurate 'abstinence only' sex ed programs, and restrict drugs because of anti-abortion ideology and not safety and efficacy concerns, the Republican leadership and base have attacked mainstream, scientific consensus when it stands in the way of their ideological postion. In recent years, Republicans have been much, much more guilty of this than Democrats.

2. Yes, the Republican-led Congress voted to double the NIH budget in the late 90's, and recently voted to double the NSF budget. But this is an easy vote - it doesn't offend anyone's ideology, it's a fairly small fraction of overall governtment spending on R&D, and there is large bipartisan support for these increases. These votes don't negate the Congressional meddling in research whenever that research is politically controversial for conservatives.

3. This 'war on science' fits the description given by Rep. Rush Holt of the current political climate (in Nature, subscription required):

"In official Washington, scientific subjects have become really politicized. There should be debate about the policy that is derived from science. But, historically, if science puts limits on the choices that are possible, the politicians would accept that. Now, by treating science as just another topic to be dealt with ideologically, or to be part of political trades, they will even ignore the laws of science."

Nature's editors should have shown some spine on this issue. At the very least, they shouldn't have taken a blatant swipe at Chris Mooney, who has been a serious champion of scientific integrity in both journalism and government.

Classic papers in quantitative biology

The hot trend in biology education right now is to train undergrads, grad students, and postdocs in not only biology, but also in enough math, computer science, and physics so that they can function successfully in the highly interdisciplinary research fields that are starting play a more important role in current research. Most major research universities are spending serious time and money thinking about how to help physicists and computer scientists become competent in biology, and biologists become competent in the relevant physics and computer science. We have a program for this purpose here at Washington University.

Biologists at Princeton have been experimenting with a variety of educational approaches, and in this month's issue of Nature Reviews Molecular Cell Biology has an essay about one course at Princeton that focuses on what I'd call 'classic papers in quantitative biolgy.' They put their grad students in a room - physicists, biologists, etc., all interested in biological problems, and have them discuss papers that contain not only great biology, but great examples of quantitative work as well.

Anyway, in this essay, there is a list of 12 of the best papers used in the course. If you're interested at all in genomics or what's being called systems biology, I highly recommend checking out this essay and the list of papers. Unfortunately you need some sort of university library access for most of them, but they're worth the effort to get them.

Comparing Genomes on a Chip

Last week at our lab meeting I had a chance to hear a presentation by Doug Berg, a microbiologist here at Washington University. Berg's work is a great combination of new technology, genomics and evolution, and it happens to also have potential medical relevance. He's studying the evolution of drug resistance in Helicobacter pylori, a ususally benign bacterium that is responsible for stomach ulcers. (Recall that the Nobel Prize in medicine was awarded last year to Barry Marshall and Robin Warren for their discovery of the link between H. pylori and ulcers.)

Berg is basically evolving highly drug resistant bacteria in the lab and using a new genome comparison technology to identify the genes that are changed as the bacterial strains become more resistant. The drug used in these experiments is metronidazole - a substance that itself is not harmful, but inside a bacterial cell it is metabolized into a toxin that causes significant DNA damage.

To create these drug-resistant strains, bacteria are spread on an agar plate - a peteri dish filled with a Jello-like substance that contains nutrients and metronidazole. Most bacteria are killed by the drug, but some individual bacteria survive, if they harbor random mutations that just happen to confer drug resistance. These surviving bacteria continue to divide, and after a few days a colony - a blob of bacteria all descended from the original surviving bacterium - has formed on the plate. You can pick off this colony with a toothpick and save it; you now have a drug resistant strain of bacteria. This process can be repeated - grow up the drug resistant strain, smear it on an agar plate with a higher concentration of the drug, and watch for the next round of colonies to grow from the survivors.

After doing many experiments like this, the Berg lab gets a series of H. pylori strains, each with varying drug resistance. The obvious question then is, how do the genomes of these strains differ? What specific (but randomly occuring) mutations had to take place in order for this level of drug resistance to evolve? Sequencing the genomes of each of these various drug-resistant strains would be the obvious way to find these differences, but doing that kind of sequencing is still prohibitively expensive and computationally intentive. But the Berg lab is using a new technique that rapidly compares genomes on a chip. (The link leads to an abstract of their paper; the full text is subscription only but you can check out more about the technique in a "webinar" found here, at the bottom of the page.). They use microarrays (DNA chips) that cover the entire genome (called a 'tiling array'). By hybridizing DNA from both the nomral and drug resistant strains to the microarray (check out my earlier summary of how a microarray works), they can identify those spots on the array (and thus in the genome) where the two strains differ.

Using this technique, the Berg lab has identified key genes involved in H. pylori resistance to metronidozole. There seem to be some genes that are always changed in these drug resistant strains, and some that can be different. In other words, there seem to be multiple evolutionary paths to drug resistance, but those paths often cross.

This techique of comparing genomes on a chip has a lot of potential, and within a few years, something like this may start showing up in common medical care and hospital labs, replacing the more primitive techniques that are currently used for classifying tumors or drug-resistant pathogens.

NY Times feature on science fraud

The NY Times has a feature article about the first case of scientific fraud with NIH money that resulted in a jail sentence for the scientist involved. A young technician in the lab of Eric Poehlman risked his reputation and future career to expose years of fraudulent research in the Poehlman lab. The grad students and postdocs in the lab were willing to look the other way for years, even though they knew the data that was ending up in the lab's published paper's sometimes came straight out of Poehlman's imagination.

I'm glad Poehlman went to prison. Someone who commits massive fraud like that wrecks trust within the scientific community and with the greater public. Without trust, between colleagues, and between the public and the scientists paid by the public, the whole merit-based funding system is threatened.