Peer review: not as dark as some might propose

With the provocative title, “Publish-or-perish: Peer review and the corruption of science“, David Colquhoun lays out, what he believes to be some problems with the current status of peer review, highlighting a specific example involving a manuscript about acupuncture.

It seems to me that Colquhoun lament the emphasis on the quantity of publications, with respect to career promotions and grant applications, which drives two undesirable practices:
1. The parsing of papers to the LPU (least publishable unit), resulting in more publications, but without a comprehensive story, or with overstated results. Or, in extreme cases, outright fraud; and,
2. An over-burdening of the current review system, where there are not enough qualified reviewers to systematically, and carefully, review each manuscript.

This may lead to initiation up of more and more journals, which accommodate the publication of LPU papers, and may often suffer from a lack of qualified reviewers.

He pulls out a study from a peer-reviewed journal on acupuncture, which, in hind-sight, is not the best example. This paper doesn’t appear to have any glaring examples of fraud, it was not retracted, and responses from one of the paper’s authors and one of the journal’s editors, seem to contradict the claims made in his summary. Alternatively, Colquhoun could have cited one of the more highly-publicized examples of the scientific community questioning the peer-review process. The first one that comes to mind is the study of “arsenic-based life“, and the many researchers who voiced their concerns about it. Or, a recent paper on the genetics of longevity that was retracted due to technical differences in how the control and sample datasets were analyzed.

I do agree with Colquhoun that one beneficial alternative would be something similar to publishing your paper and leave it open to comments from the community. I don’t think that reviews need be anonymous, as he supposes, after citing a failure of such an endeavor at Nature. There is an example of a successful journal, Biology Direct, in which the peer review process is open, and the reviews (as well as author responses) are published alongside the manuscript. After a quick search, I found one really wonderful example, where I learned as much from the back-and-forth between the reviewers and the authors, as I did from the article. The conversation between the authors, and the reviewers, people who have been studying the evolution of life on earth, for decades was respectful, but with disagreements.

Maybe I’m optimistic, but I think there is a lot of benefit of public reviews. It is especially useful to read questions raised by experts that I, as someone unfamiliar with the field, would not know to ask.

Alternatively, I am not sure how to address the problem of quantity-over-quality in academic promotions and funding success. Colquhoun says it very well, I think, that, “It arises from official pressure to publish when you have nothing to say”. He also does not have many suggestions, and seems pessimistic about alternative measurements of scientific success.

Perhaps, rather than simply list publications, scientists could write a summary of the findings from their publications over the review period. This might equal the playing field between a manuscript that presents a comprehensive analysis, and a list of several manuscripts, each of which are short-stories. In addition, it would encourage the dissemination of such knowledge to the public (and encourage public support of science) because Institutions could share these short summaries online every time a new review is completed. A double win!

But, it seems like the current system is in place for the foreseeable future.

I wonder if blog posts can contribute to my publication list?

Postdoctoral fellowships

Last year I had a heck of a time finding a summary of postdoctoral fellowships to apply for, so thought my compilation might assist others. This list is by no means comprehensive, but it’s a start. I’m happy to update this post with other suggestions (and links and descriptions). I especially tried to include whether the fellowships are available to international students or not. I’ve ordered them by approximate submission date, but as submission dates change every year, please don’t rely on these. Similarly, the links were active at the time of this posting. If they aren’t working, you can probably find the fellowship with a simple google search.

– Due ~October
– LSRF awards fellowships across the spectrum of the life sciences: biochemistry; cell, developmental, molecular, plant, structural, organismic population and evolutionary biology; endocrinology; immunology; microbiology; neurobiology; physiology; virology.
– Similar structure as NSF postdoc
– Only one LSRF fellow allowed in a lab at any given time
– U.S. citizens are eligible to work in any geographic location while holding an LSRF fellowship. Non-U.S. citizens must work in a U.S. laboratory to be eligible for an LSRF fellowship.

– Due ~October
– The focus this year is, “Intersections of Biology and Mathematical and Physical Sciences“.
– applicants must be U.S. citizens (or nationals) or permanent residents of the United States (i.e., have a “green card”) at deadline.
– Due ~December
– Applicants must be citizens or non-citizen nationals of the United States, or have been lawfully admitted to the United States for permanent residence.
– Due ~December
– Has both US and International options
– Due ~January
– All women scientists that are conducting research in the natural sciences regardless of nationality are eligible for application for any of the SDE/GWIS Fellowships.
– Due ~March

– Due ~July
– Fellowships may be awarded to US citizens planning to work in laboratories either in the US, Canada, or abroad and also to foreign citizens for research in laboratories in the US only.




Which exome platform is the best?

Well, it seems like, as with most technologies, it depends on the question you are asking.

The paper listed below, published in Genome Biology (,

Comprehensive comparison of three commercial human whole-exome captureplatforms

Asan2,3,*, Yu Xu1,*, Hui Jiang1,*, Chris Tyler-Smith4,*, Yali Xue4, Tao Jiang1, JiaweiWang1, Mingzhi Wu1, Xiao Liu1, Geng Tian1, Jun Wang1, Jian Wang1, HuangmingYang1,# and Xiuqing Zhang1,#.

compared similar measurements across 1) NibleGen Sequence Capture Array, 2) NimbleGen SeqCap EZ and 3) Agilent SureSelect technologies.

The authors used similar preparation procedures for the three technologies – same number of PCR cycles and minimizing the DNA input, reducing the likelihood that external variables would affect the results.

It is comforting that all three technologies performed similarly with respect to calling SNPs, assigning genotypes (>77% sensitivity), and reproducing results. All three also exhibited similar GC and reference allele biases (which I suppose is good, although it might have been nice if one had much lower biases). Both technologies were biased against low (<20%) or high (>75%) GC content, but the array technology actually performed a little better coverage of low GC-content regions. However, all technologies performed very poorly, missing most high GC (>72-80%) content regions.

As far as differences:

-Agilent covered approximately 1000 more genes and 100 more microRNA genes than NimbleGen(4.1Mb of sequence specific to NimbleGen and 7.6Mb specific to Agilent). Still, 15,883 genes overlapped between the two technologies. I looked through the supplemental materials and couldn’t find out whether the differences were due to differential captures of sex-linked genes (X or Y).

– NimbleGen provided better flanking-sequence capture (although this technology is designed to capture coding sequences, often 200bp flanking each side may also be captured)

– NimbleGen also had more uniform coverage and greater genotype sensitivity at high sequencing depth (30-100X)

A drawback of exome sequences is that they represent less than two percent of the entire genome, so functionally relevant changes in regulatory elements, or other non-coding elements will be missed.

Alternatively, some advantages of exome sequencing are that it focuses the search to the parts of the genome that we know are functionally relevant, and these regions are more likely to be involved in human diseases than a randomly selected non-coding region.

In summary, all three technologies perform similarly well, but one should be aware that Agilent has the best gene coverage, SeqCap EZ captures more flanking sequences and the one array capture (NimbleGen Sequence Capture) performs better with respect to covering low GC (<20%) regions.

Do the genes belong to the tribal council?

There has been lot of debate about tribal consent since the publication of the first aboriginal genome.   Nature ran a news story debating issues regarding tribal and individual consent for genetic studies and Razib Khan wrote a blog about it at Discover Magazine. Razib Khan’s perspective is perhaps the most interesting and informed.  He is commenting on the claim by Eske Willerslev that we would not have published the study without tribal consent.  Khan’s arguably very American perspective on this, is that only individual consent can matter.  The opinion of tribal councils or any other institutions regarding release of personal genetic information is irrelevant.  There is no reason to consult such institutions – only the individual matters.  An entirely defensible position. It is certainly the position of the legal community in America and perhaps most of the scientific community. But there are a couple of reasons why the story in this case is a bit more complex.  First, as noted in one of the online comments on Khan’s blog, the genome sequenced was from a deceased individual with no identifiable descendants.  We could not obtain informed consent from the donor or from his family.  In our opinion, the correct thing to do  was then to consult with the local representatives of the group to which the donor belonged.  After considering different options we determined that the Goldfields Land and Sea Council was our best option.  Secondly, and most importantly, there is a several hundred year old tradition for exploitation of indigenous people by anthropologists, who until recently systematically have failed to recognize the rights of indigenous people.  This dark historical relationship between indigenous people and anthropologists makes it particularly important to involve the local indigenous communities in scientific studies.  Our approach was to reach out to the local tribal council to work with them in the interpretation and presentation of our results.  I strongly believe that is the right thing to do and I hope other geneticists will do the same thing in the future when working on questions relating to indigenous people.  I am not advocating granting particular institutions the power to veto studies that individual members of the community consent to, but I would encourage researchers to involve the local community as much as possible. You do that out of respect for the local community and to promote and enable a good relationship between indigenous people and the scientists who wish to learn from them.

Khan also used our paper to revisit an old discussion regarding ownership of genetic information in the context of familial relationships.  He writes:

“I have addressed the issue of whether identical twins have “rights” to each others’ genomes. For example, if one identical twin put their genotype into the public domain, would the other be within their rights to object? For that matter, people who put their genotypes in the public domain are partially exposing their whole families. Do they have to go ask for permission? Obviously I don’t think so. I didn’t ask my siblings or my parents”.

The individualistic view again dictates that my personal information is for me to disclose, even if doing so could affect other people.  He further writes:

“But in the area where genetics and ethnology intersect too often people overestimate the power of genetics to totally reshape how we view ourselves, and how we view other human beings. The reality is that we are what we are, before and after we find out what we are in a more scientific and abstruse fashion”.

I strongly agree with the second point.  In fact, the predictive power of genetics is hugely exaggerated in the public perception.  Revealing your genomic information reveals very little about you – at least here in 2011.  Our abysmal ability to predict phenotypes in humans from DNA data has been one of the big disappointments of human genetics in the past 10 years, although some researchers such as Dick Lewontin predicted this outcome before even the first human genome was sequenced.

Even though I agree with Khan’s second point, I cannot help but worry about the issues regarding disclosure of genetic information.  As long as the public has faith in the geneticists ability to predict phenotypes, the implications of disclosing genetic information are enormous.  When my brother in a few years is being considered for a position as CEO for a major company, I am sure he wouldn’t appreciate if I disclose that I carry a mutation that disposes me to early onset of Alzheimer’s disease.  He might after all then also carry the mutation with 50% chance.  Even if we wouldn’t want to ban me from disclosing information about myself, the nice thing for me to do would nonetheless be to talk with my brother before making the disclosure.

The same thing goes for genetic studies of indigenous people.  While perhaps we should not allow particular tribal organizations to yield veto power over scientific studies, it would behoove us as scientists to consult with them as much as possible.  Perhaps the genes do not belong to the tribal council – but they certainly do not belong to the scientists.  We have hundreds of years of abuse to make up for – and starting by consulting the local community when conducting research is but one small step.







Visualization of journal profiles

Gaston Sanchez has made a remarkable web site that compares the profiles of different genetic journals.  For example, this is the word cloud for Genetics:

Perhaps we should rename the journal to the Journal of Drosophila Research?

Check out the word clouds and other graphical representations here.

Aboriginal genome sequenced

We participated in the analyses of the first sequenced aboriginal genome.  It was another fun project coordinated by Eske Willerslev from University of Copenhagen.  We are particularly proud of the way Eske succeeded in involving the Aboriginal Australian community in the research.

You can read some of the press stories about this project here:

SF Chronicle

Yahoo News

The Atlantic


Up and running…

We are up and running with the new web site and Blog for the Nielsen group.  We will use this space to spread our opinions about the state of affairs in evolutionary genomics.  If you disagree with us – leave a comment.