Blissful Ignorance: How Environmental Activists Shut Down Molecular Biology Labs in High Schools

Hearing about the HannoverGEN project made me feel envious and excited. Envious, because I wish my high school had offered the kind of hands-on molecular biology training provided to high school students in Hannover, the capital of the German state of Niedersachsen. Excited, because it reminded me of the joy I felt when I first isolated DNA and ran gels after restriction enzyme digests during my first year of university in Munich. I knew that many of the students at the HannoverGEN high schools would be similarly thrilled by their laboratory experience and perhaps even pursue careers as biologists or biochemists.

What did HannoverGEN entail? It was an optional pilot program initiated and funded by the state government of Niedersachsen at four high schools in the Hannover area. Students enrolled in the HannoverGEN classes would learn to use molecular biology tools typically reserved for college-level or graduate school courses in order to study plant genetics. Some of the basic experiments involved isolating DNA from cabbage or how learning how bacteria transfer genes to plants, more advanced experiments enabled the students to analyze whether or not the genome of a provided maize sample had been genetically modified. Each experimental unit was accompanied by relevant theoretical instruction on the molecular mechanisms of gene expression and biotechnology as well as ethical discussions regarding the benefits and risks of generating genetically modified organisms (“GMOs”). The details of the HannoverGEN program are only accessible through the the Wayback Machine Internet archive because the award-winning educational program and the associated website were shut down in 2013 at the behest of German anti-GMO activist groups, environmental activists, Greenpeace, the Niedersachsen Green Party and the German organic food industry.

Why did these activists and organic food industry lobbyists oppose a government-funded educational program which improved the molecular biology knowledge and expertise of high school students? A press release entitled “Keine Akzeptanzbeschaffung für Agro-Gentechnik an Schulen!” (“No Acceptance for Agricultural Gene Technology at Schools“) in 2012 by an alliance representing “organic” or “natural food” farmers accompanied by the publication of a critical “study” with the same title (PDF), which was funded by this alliance as well as its anti-GMO partners, gives us some clues. They feared that the high school students might become too accepting of biotechnology in agriculture and that the curriculum did not sufficiently highlight all the potential dangers of GMOs. By allowing the ethical discussions to not only discuss the risks but also mention the benefits of genetically modifying crops, students might walk away with the idea that GMOs could be beneficial for humankind. The group believed that taxpayer money should not be used to foster special interests such as those of the agricultural industry which may want to use GMOs.

A response by the University of Hannover (PDF), which had helped develop the curriculum and coordinated the classes for the high school students, carefully analyzed the complaints of the anti-GMO activists. The author of the anti-HannoverGEN “study” had not visited the HannoverGEN laboratories, nor had he had interviewed the biology teachers or students enrolled in the classes. In fact, his critique was based on weblinks that were not even used in the curriculum by the HannoverGEN teachers or students. His analysis ignored the balanced presentation of biotechnology that formed the basis of the HannoverGEN curriculum and that discussing potential risks of genetic modification was a core topic in all the classes.

Unfortunately, this shoddily prepared “study” had a significant impact, in part because it was widely promoted by partner organizations. Its release in the autumn of 2012 came at an opportune time for political activists because Niedersachsen was about to have an election. Campaigning against GMOs seemed like a perfect cause for the Green Party and a high school program which taught the use of biotechnology to high school students became a convenient lightning rod. When the Social Democrats and the Green Party formed a coalition after winning the election in early 2013, nixing the HannoverGEN high school program was formally included in the so-called coalition contract. This is a document in which coalition partners outline the key goals for the upcoming four year period. When one considers how many major issues and problems the government of a large German state has to face, such as healthcare, education, unemployment or immigration, it is mind-boggling that de-funding a program involving only four high schools received so much attention that it needed to be anchored in the coalition contract. In fact, it is a testimony to the influence and zeal of the anti-GMO lobby.

Once the cancellation of HannoverGEN was announced, the Hannover branch of Greenpeace also took credit for campaigning against this high school program and celebrated its victory. The Greenpeace anti-GMO activist David Petersen said that the program was too cost intensive because equipping high school laboratories with state-of-the-art molecular biology equipment had already cost more than 1 million Euros. The previous center-right government which had initiated the HannoverGEN project was planning on expanding the program to even more high schools because of the program’s success and national recognition for innovative teaching. According to Petersen, this would have wasted even more taxpayer money without adequately conveying the dangers of using GMOs in agriculture.

The scientific community was shaken up by the decision of the new Social Democrat-Green Party coalition government in Niedersachsen. This was an attack on the academic freedom of schools under the guise of accusing them of promoting special interests while ignoring that the anti-GMO activists were representing their own special interests. The “study” attacking HannoverGEN was funded by the lucrative “organic” or “natural food” food industry! Scientists and science writers such as Martin Ballaschk or Lars Fischer wrote excellent critical articles stating that squashing high-quality, hand-on science programs could not lead to better decision-making. How could ignorant students have a better grasp of GMO risks and benefits than those who receive relevant formal science education and thus make truly informed decisions? Sadly, this outcry by scientists and science writers did not make much of a difference. It did not seem that the media felt this was much of a cause to fight for. I wonder if the media response would have been just as lackluster if the government had de-funded a hands-on science lab to study the effects of climate change.

In 2014, the government of Niedersachsen then announced that they would resurrect an advanced biology laboratory program for high schools with the generic and vague title “Life Science Lab”. By removing the word “Gen” from its title which seems to trigger visceral antipathy among anti-GMO activists, de-emphasizing genome science and by also removing any discussion of GMOs from the curriculum, this new program would leave students in the dark about GMOs. Ignorance is bliss from an anti-GMO activist perspective because the void of scientific ignorance can be filled with fear.

From the very first day that I could vote in Germany during the federal election of 1990, I always viewed the Green Party as a party that represented my generation. A party of progressive ideas, concerned about our environment and social causes. However, the HannoverGEN incident is just one example of how the Green Party is caving in to ideologies, thus losing its open-mindedness and progressive nature. In the United States, the anti-science movement, which attacks teaching climate change science or evolutionary biology at schools, tends to be rooted in the right wing political spectrum. Right wingers or libertarians are the ones who always complain about taxpayer dollars being wasted and used to promote agendas in schools and universities. But we should not forget that there is also a different anti-science movement rooted in the leftist and pro-environmental political spectrum – not just in Germany. As a scientist, I feel that it is becoming increasingly difficult to support the Green Party because of its anti-science stance.

I worry about all anti-science movements, especially those which attack science education. There is nothing wrong with questioning special interests and ensuring that school and university science curricula are truly balanced. But the balance needs to be rooted in scientific principles, not political ideologies. Science education has a natural bias – it is biased towards knowledge that is backed up by scientific evidence. We can hypothetically discuss dangers of GMOs but the science behind the dangers of GMO crops is very questionable. Just like environmental activists and leftists agree with us scientists that we do not need to give climate change deniers and creationists “balanced” treatment in our science curricula, they should also accept that much of the “anti-GMO science” is currently more based on ideology than on actual scientific data. Our job is to provide excellent science education so that our students can critically analyze and understand scientific research, independent of whether or not it supports our personal ideologies.


Note: An earlier version of this article was first published on the 3Quarksdaily blog.

To Err Is Human, To Study Errors Is Science

The family of cholesterol lowering drugs known as ‘statins’ are among the most widely prescribed medications for patients with cardiovascular disease. Large-scale clinical studies have repeatedly shown that statins can significantly lower cholesterol levels and the risk of future heart attacks, especially in patients who have already been diagnosed with cardiovascular disease. A more contentious issue is the use of statins in individuals who have no history of heart attacks, strokes or blockages in their blood vessels. Instead of waiting for the first major manifestation of cardiovascular disease, should one start statin therapy early on to prevent cardiovascular disease?

If statins were free of charge and had no side effects whatsoever, the answer would be rather straightforward: Go ahead and use them as soon as possible. However, like all medications, statins come at a price. There is the financial cost to the patient or their insurance to pay for the medications, and there is a health cost to the patients who experience potential side effects. The Guideline Panel of the American College of Cardiology (ACC) and the American Heart Association (AHA) therefore recently recommended that the preventive use of statins in individuals without known cardiovascular disease should be based on personalized risk calculations. If the risk of developing disease within the next 10 years is greater than 7.5%, then the benefits of statin therapy outweigh its risks and the treatment should be initiated. The panel also indicated that if the 10-year risk of cardiovascular disease is greater than 5%, then physicians should consider prescribing statins, but should bear in mind that the scientific evidence for this recommendation was not as strong as that for higher-risk individuals.


Oops button - via Shutterstock
Oops button – via Shutterstock

Using statins in low risk patients

The recommendation that individuals with comparatively low risk of developing future cardiovascular disease (10-year risk lower than 10%) would benefit from statins was met skepticism by some medical experts. In October 2013, the British Medical Journal (BMJ) published a paper by John Abramson, a lecturer at Harvard Medical School, and his colleagues which re-evaluated the data from a prior study on statin benefits in patients with less than 10% cardiovascular disease risk over 10 years. Abramson and colleagues concluded that the statin benefits were over-stated and that statin therapy should not be expanded to include this group of individuals. To further bolster their case, Abramson and colleagues also cited a 2013 study by Huabing Zhang and colleagues in the Annals of Internal Medicine which (according to Abramson et al.) had reported that 18 % of patients discontinued statins due to side effects. Abramson even highlighted the finding from the Zhang study by including it as one of four bullet points summarizing the key take-home messages of his article.

The problem with this characterization of the Zhang study is that it ignored all the caveats that Zhang and colleagues had mentioned when discussing their findings. The Zhang study was based on the retrospective review of patient charts and did not establish a true cause-and-effect relationship between the discontinuation of the statins and actual side effects of statins. Patients may stop taking medications for many reasons, but this does not necessarily mean that it is due to side effects from the medication. According to the Zhang paper, 17.4% of patients in their observational retrospective study had reported a “statin related incident” and of those only 59% had stopped the medication. The fraction of patients discontinuing statins due to suspected side effects was at most 9-10% instead of the 18% cited by Abramson. But as Zhang pointed out, their study did not include a placebo control group. Trials with placebo groups document similar rates of “side effects” in patients taking statins and those taking placebos, suggesting that only a small minority of perceived side effects are truly caused by the chemical compounds in statin drugs.


Admitting errors is only the first step

Whether 18%, 9% or a far smaller proportion of patients experience significant medication side effects is no small matter because the analysis could affect millions of patients currently being treated with statins. A gross overestimation of statin side effects could prompt physicians to prematurely discontinue medications that have been shown to significantly reduce the risk of heart attacks in a wide range of patients. On the other hand, severely underestimating statin side effects could result in the discounting of important symptoms and the suffering of patients. Abramson’s misinterpretation of statin side effect data was pointed out by readers of the BMJ soon after the article published, and it prompted an inquiry by the journal. After re-evaluating the data and discussing the issue with Abramson and colleagues, the journal issued a correction in which it clarified the misrepresentation of the Zhang paper.

Fiona Godlee, the editor-in-chief of the BMJ also wrote an editorial explaining the decision to issue a correction regarding the question of side effects and that there was not sufficient cause to retract the whole paper since the other points made by Abramson and colleagues – the lack of benefit in low risk patients – might still hold true. Instead, Godlee recognized the inherent bias of a journal’s editor when it comes to deciding on whether or not to retract a paper. Every retraction of a peer reviewed scholarly paper is somewhat of an embarrassment to the authors of the paper as well as the journal because it suggests that the peer review process failed to identify one or more major flaws. In a commendable move, the journal appointed a multidisciplinary review panel which includes leading cardiovascular epidemiologists. This panel will review the Abramson paper as well as another BMJ paper which had also cited the inaccurately high frequency of statin side effects, investigate the peer review process that failed to identify the erroneous claims and provide recommendations regarding the ultimate fate of the papers.


Reviewing peer review

Why didn’t the peer reviewers who evaluated Abramson’s article catch the error prior to its publication? We can only speculate as to why such a major error was not identified by the peer reviewers. One has to bear in mind that “peer review” for academic research journals is just that – a review. In most cases, peer reviewers do not have access to the original data and cannot check the veracity or replicability of analyses and experiments. For most journals, peer review is conducted on a voluntary (unpaid) basis by two to four expert reviewers who routinely spend multiple hours analyzing the appropriateness of the experimental design, methods, presentation of results and conclusions of a submitted manuscript. The reviewers operate under the assumption that the authors of the manuscript are professional and honest in terms of how they present the data and describe their scientific methodology.

In the case of Abramson and colleagues, the correction issued by the BMJ refers not to Abramson’s own analysis but to the misreading of another group’s research. Biomedical research papers often cite 30 or 40 studies, and it is unrealistic to expect that peer reviewers read all the cited papers and ensure that they are being properly cited and interpreted. If this were the expectation, few peer reviewers would agree to serve as volunteer reviewers since they would have hardly any time left to conduct their own research. However, in this particular case, most peer reviewers familiar with statins and the controversies surrounding their side effects should have expressed concerns regarding the extraordinarily high figure of 18% cited by Abramson and colleagues. Hopefully, the review panel will identify the reasons for the failure of BMJ’s peer review system and point out ways to improve it.


To err is human, to study errors is science

All researchers make mistakes, simply because they are human. It is impossible to eliminate all errors in any endeavor that involves humans, but we can construct safeguards that help us reduce the occurrence and magnitude of our errors. Overt fraud and misconduct are rare causes of errors in research, but their effects on any given research field can be devastating. One of the most notorious occurrences of research fraud is the case of the Dutch psychologist Diederik Stapel who published numerous papers based on blatant fabrication of data – showing ‘results’ of experiments on non-existent study subjects. The field of cell therapy in cardiovascular disease recently experienced a major setback when a university review of studies headed by the German cardiologist Bodo Strauer found evidence of scientific misconduct. The significant discrepancies and irregularities in Strauer’s studies have now lead to wide-ranging skepticism about the efficacy of using bone marrow cell infusions to treat heart disease.


It is difficult to obtain precise numbers to quantify the actual extent of severe research misconduct and fraud since it may go undetected. Even when such cases are brought to the attention of the academic leadership, the involved committees and administrators may decide to keep their findings confidential and not disclose them to the public. However, most researchers working in academic research environments would probably agree that these are rare occurrences. A far more likely source of errors in research is the cognitive bias of the researchers. Researchers who believe in certain hypotheses and ideas are prone to interpreting data in a manner most likely to support their preconceived notions. For example, it is likely that a researcher opposed to statin usage will interpret data on side effects of statins differently than a researcher who supports statin usage. While Abramson may have been biased in the interpretation of the data generated by Zhang and colleagues, the field of cardiovascular regeneration is currently grappling in what appears to be a case of biased interpretation of one’s own data. An institutional review by Harvard Medical School and Brigham and Women’s Hospital recently determined that the work of Piero Anversa, one of the world’s most widely cited stem cell researchers, was significantly compromised and warranted a retraction. His group had reported that the adult human heart exhibited an amazing regenerative potential, suggesting that roughly every 8 to 9 years the adult human heart replaces its entire collective of beating heart cells (a 7% – 19% yearly turnover of beating heart cells). These findings were in sharp contrast to a prior study which had found only a minimal turnover of beating heart cells (1% or less per year) in adult humans. Anversa’s finding was also at odds with the observations of clinical cardiologists who rarely observe a near-miraculous recovery of heart function in patients with severe heart disease. One possible explanation for the huge discrepancy between the prior research and Anversa’s studies was that Anversa and his colleagues had not taken into account the possibility of contaminations that could have falsely elevated the cell regeneration counts.


Improving the quality of research: peer review and more

Despite the fact that researchers are prone to make errors due to inherent biases does not mean we should simply throw our hands up in the air, say “Mistakes happen!” and let matters rest. High quality science is characterized by its willingness to correct itself, and this includes improving methods to detect and correct scientific errors early on so that we can limit their detrimental impact. The realization that lack of reproducibility of peer-reviewed scientific papers is becoming a major problem for many areas of research such as psychology, stem cell research and cancer biology has prompted calls for better ways to track reproducibility and errors in science.

One important new paradigm that is being discussed to improve the quality of scholar papers is the role of post-publication peer evaluation. Instead of viewing the publication of a peer-reviewed research paper as an endpoint, post publication peer evaluation invites fellow scientists to continue commenting on the quality and accuracy of the published research even after its publication and to engage the authors in this process. Traditional peer review relies on just a handful of reviewers who decide about the fate of a manuscript, but post publication peer evaluation opens up the debate to hundreds or even thousands of readers which may be able to detect errors that could not be identified by the small number of traditional peer reviewers prior to publication. It is also becoming apparent that science journalists and science writers can play an important role in the post-publication evaluation of published research papers by investigating and communicating research flaws identified in research papers. In addition to helping dismantle the Science Mystique, critical science journalism can help ensure that corrections, retractions or other major concerns about the validity of scientific findings are communicated to a broad non-specialist audience.

In addition to these ongoing efforts to reduce errors in science by improving the evaluation of scientific papers, it may also be useful to consider new pro-active initiatives which focus on how researchers perform and design experiments. As the head of a research group at an American university, I have to take mandatory courses (in some cases on an annual basis) informing me about laboratory hazards, ethics of animal experimentation or the ethics of how to conduct human studies. However, there are no mandatory courses helping us identify our own research biases or how to minimize their impact on the interpretation of our data. There is an underlying assumption that if you are no longer a trainee, you probably know how to perform and interpret scientific experiments. I would argue that it does not hurt to remind scientists regularly – no matter how junior or senior- that they can become victims of their biases. We have to learn to continuously re-evaluate how we conduct science and to be humble enough to listen to our colleagues, especially when they disagree with us.


Note: A shorter version of this article was first published at The Conversation with excellent editorial input provided by Jo Adetunji.
Abramson, J., Rosenberg, H., Jewell, N., & Wright, J. (2013). Should people at low risk of cardiovascular disease take a statin? BMJ, 347 (oct22 3) DOI: 10.1136/bmj.f6123

Neutrality, Balance and Anonymous Sources in Science Blogging – #scioStandards

This is Part 2 of a series of blog posts in anticipation of the Upholding standards in scientific blogs (Session 10B, #scioStandards) session which I will be facilitating at noon on Saturday, March 1 at the upcoming ScienceOnline conference (February 27 – March 1, 2014 in Raleigh, NC – USA). Please read Part 1 here. The goal of these blog posts is to raise questions which readers can ponder and hopefully discuss during the session.


1.       Neutrality

Neutrality is prized by scientists and journalists. Scientists are supposed to report and analyze their scientific research in a neutral fashion. Similarly, journalistic professionalism requires a neutral and objective stance when reporting or analyzing news. Nevertheless, scientists and journalists are also aware of the fact that there is no perfect neutrality. We are all victims of our conscious and unconscious biases and how we report data or events is colored by our biases. Not only is it impossible to be truly “neutral”, but one can even question whether “neutrality” should be a universal mandate. Neutrality can make us passive, especially when we see a clear ethical mandate to take action. Should one report in a neutral manner about genocide instead of becoming an advocate for the victims? Should a scientist who observes a destruction of ecosystems report on this in a neutral manner? Is it acceptable or perhaps even required for such a scientist to abandon neutrality and becoming an advocate to protect the ecosystems?

Science bloggers or science journalists have to struggle to find the right balance between neutrality and advocacy. Political bloggers and journalists who are enthusiastic supporters of a political party will find it difficult to preserve neutrality in their writing, but their target audiences may not necessarily expect them to remain neutral. I am often fascinated and excited by scientific discoveries and concepts that I want to write about, but I also notice how my enthusiasm for science compromises my neutrality. Should science bloggers strive for neutrality and avoid advocacy? Or is it understood that their audiences do not expect neutrality?


2.       Balance

One way to increase objectivity and neutrality in science writing is to provide balanced views. When discussing a scientific discovery or concept, one can also cite or reference scientists with opposing views. This underscores that scientific opinion is not a monolith and that most scientific findings can and should be challenged. However, the mandate to provide balance can also lead to “false balance” when two opposing opinions are presented as two equivalent perspectives, even though one of the two sides has little to no scientific evidence to back up its claims. More than 99% of all climatologists agree about the importance of anthropogenic global warming, therefore it would be “false balance” to give equal space to opposing fringe views. Most science bloggers would also avoid “false balance” when it comes to reporting about the scientific value of homeopathy since nearly every scientist in the world agrees that homeopathy has no scientific data to back it up.

But how should science bloggers decide what constitutes “necessary balance” versus “false balance” when writing about areas of research where the scientific evidence is more ambivalent. How about a scientific discovery which 80% of scientists think is a landmark finding and 20% of scientists believe is a fluke? How does one find out about the scientific rigor of the various viewpoints and how should a blog post reflect these differences in opinion? Press releases of universities or research institutions usually only cite the researchers that conducted a scientific study, but how does one find out about other scientists who disagree with the significance of the new study?


3.       Anonymous Sources

Most scientific peer review is conducted with anonymous sources. The editors of peer reviewed scientific journals send out newly submitted manuscripts to expert reviewers in the field but they try to make sure that the names of the reviewers remain confidential. This helps ensure that the reviewers can comment freely about any potential flaws in the manuscript without having to fear retaliation from the authors who might be incensed about the critique. Even in the post-publication phase, anonymous commenters can leave critical comments about a published study at the post-publication peer review website PubPeer. The comments made by anonymous as well as identified commenters at PubPeer played an important role in raising questions about recent controversial stem cell papers. On the other hand, anonymous sources may also use their cover to make baseless accusations and malign researchers. In the case of journals, the responsibility lies with the editors to ensure that their anonymous reviewers are indeed behaving in a professional manner and not abusing their anonymity.

Investigative political journalists also often rely on anonymous sources and whistle-blowers to receive critical information that would have otherwise been impossible to obtain. Journalists are also trained to ensure that their anonymous sources are credible and that they are not abusing their anonymity.

Should science bloggers and science journalists also consider using anonymous sources? Would unnamed scientists provide a more thorough critical appraisal of the quality of scientific research or would this open the door to abuse?


I hope that you leave comments on this post, tweet your thoughts using the #scioStandards hashtag and discuss your views at the Science Online conference.

Background Reading in Science Blogging – #scioStandards

There will be so many interesting sessions at the upcoming ScienceOnline conference (February 27 – March 1, 2014 in Raleigh, NC – USA) that it is going to be difficult to choose which sessions to attend, because one will invariably miss out on concurrent sessions. If you are not too exhausted, please attend one of the last sessions of the conference: Upholding standards in scientific blogs (Session 10B, #scioStandards).


I will be facilitating the discussion at this session, which will take place at noon on Saturday, March 1, just before the final session of the conference. The title of the session is rather vague, and the purpose of the session is for attendees to exchange their views on whether we can agree on certain scientific and journalistic standards for science blogging.

Individual science bloggers have very different professional backgrounds and they also write for a rather diverse audience. Some bloggers are part of larger networks, others host a blog on their own personal website. Some are paid, others write for free. Most bloggers have developed their own personal styles for how they write about scientific studies, the process of scientific discovery, science policy and the lives of people involved in science. Considering the heterogeneity in the science blogging community, is it even feasible to identify “standards” for scientific blogging? Are there some core scientific and journalistic standards that most science bloggers can agree on? Would such “standards” merely serve as informal guidelines or should they be used as measures to assess the quality of science blogging?

These are the kinds of questions that we will try to discuss at the session. I hope that we will have a lively discussion, share our respective viewpoints and see what we can learn from each other. To gauge the interest levels of the attendees, I am going to pitch a few potential discussion topics on this blog and use your feedback to facilitate the discussion. I would welcome all of your responses and comments, independent of whether you intend to attend the conference or the session. I will also post these questions in the Science Online discussion forum.

One of the challenges we face when we blog about specific scientific studies is determining how much background reading is necessary to write a reasonably accurate blog post. Most science bloggers probably read the original research paper they intend to write about, but even this can be challenging at times. Scientific papers aren’t very long. Journals usually restrict the word count of original research papers to somewhere between 2,000 words to 8,000 words (depending on each scientific journal’s policy and whether the study is a published as a short communication or a full-length article). However, original research papers are also accompanied four to eight multi-paneled figures with extensive legends.

Nowadays, research papers frequently include additional figures, data-sets and detailed descriptions of scientific methods that are published online and not subject to the word count limit. A 2,000 word short communication with two data figures in the main manuscript may therefore be accompanied by eight “supplemental” online-only figures and an additional 2,000 words of text describing the methods in detail. A single manuscript usually summarizes the results of multiple years of experimental work, which is why this condensed end-product is quite dense. It can take hours to properly study the published research study and understand the intricate details.

Is it enough to merely read the original research paper in order to blog about it? Scientific papers include a brief introduction section, but these tend to be written for colleagues who are well-acquainted with the background and significance of the research. However, unless one happens to blog about a paper that is directly related to one’s own work, most of us probably need additional background reading to fully understand the significance of a newly published study.

An expert on liver stem cells, for example, who wants blog about the significance of a new paper on lung stem cells will probably need substantial amount of additional background reading. One may have to read at least one or two older research papers by the authors or their scientific colleagues / competitors to grasp what makes the new study so unique. It may also be helpful to read at least one review paper (e.g. a review article summarizing recent lung stem cell discoveries) to understand the “big picture”. Some research papers are accompanied by scientific editorials which can provide important insights into the strengths and limitations of the paper in question.

All of this reading adds up. If it takes a few hours to understand the main paper that one intends to blog about, and an additional 2-3 hours to read other papers or editorials, a science blogger may end up having to invest 4-5 hours of reading before one has even begun to write the intended blog post.

What strategies have science bloggers developed to manage their time efficiently and make sure they can meet (external or self-imposed) deadlines but still complete the necessary background reading?

Should bloggers provide references and links to the additional papers they consulted?

Should bloggers try to focus on a narrow area of expertise so that over time they develop enough of a background in this niche area so that they do not need so much background reading?

Are there major differences in the expectations of how much background reading is necessary? For example, does an area such as stem cell research or nanotechnology require far more background reading because every day numerous new papers are published and it is so difficult to keep up with the pace of the research?

Is it acceptable to take short-cuts? Could one just read the paper that one wants to blog about and forget about additional background reading, hoping that the background provided in the paper is sufficient and balanced?

Can one avoid reading the supplementary figures or texts of a paper and just stick to the main text of a paper, relying on the fact that the peer reviewers of the published paper would have caught any irregularities in the supplementary data?

Is it possible to primarily rely on a press release or an interview with the researchers of the paper and just skim the results of the paper instead of spending a few hours trying to read the original paper?

Or do such short-cuts compromise the scientific and journalistic quality of science blogs?

Would a discussion about expectations, standards and strategies to manage background reading be helpful for participants of the session?

Is It Possible To Have Excess Weight And Still Be Healthy?

Is it possible to be overweight or obese and still be considered healthy? Most physicians advise their patients who are overweight or obese to lose weight because excess weight is a known risk factor for severe chronic diseases such as diabetes, high blood pressure or cardiovascular disease. However, in recent years, a controversy has arisen regarding the actual impact of increased weight on an individual’s life expectancy or risk of suffering from heart attacks. Some researchers argue that being overweight (body mass index between 25 and 30; calculate your body mass index here) or obese (body mass index greater than 30) primarily affects one’s metabolic health and it is the prolonged exposure to metabolic problems that in turn lead to cardiovascular disease or death.



According to this view, merely having excess weight is not dangerous. It only becomes a major problem if it causes metabolic problems such as high cholesterol levels, high blood sugar levels and diabetes or high blood pressure. This suggests that there is a weight/health spectrum which includes overweight or obese individuals with normal metabolic parameters who are not yet significantly impacted by the excess weight (“healthy overweight” and “healthy obesity”). The other end of the spectrum includes overweight and obese individuals who also have significant metabolic abnormalities due to the excess weight and these individuals are at a much higher risk for heart disease and death because of the metabolic problems.

Other researchers disagree with this view and propose that all excess weight is harmful, independent of whether the overweight or obese individuals have normal metabolic parameters. To resolve this controversy, researchers at the Mount Sinai Hospital and University of Toronto recently performed a meta-analysis and evaluated the data from major clinical studies comparing the mortality (risk of death) and heart disease (as defined by events such as heart attacks) in normal weight, overweight and obese individuals and grouping them by their metabolic health.

The study was recently published in the Annals of Internal Medicine (2014) as “Are Metabolically Healthy Overweight and Obesity Benign Conditions?: A Systematic Review and Meta-analysis” and provided data on six groups of individuals: 1) metabolically healthy and normal weight, 2) metabolically healthy and overweight, 3) metabolically healthy and obese, 4) metabolically unhealthy and normal weight, 5) metabolically unhealthy and overweight and 6) metabolically unhealthy and obese. The researchers could only include studies which had measured metabolic health (normal blood sugar, blood pressure, cholesterol, etc.) alongside with weight.

The first important finding was that metabolically healthy overweight individuals did NOT have a significantly higher risk of death and cardiovascular events when compared to metabolically healthy normal weight individuals. The researchers then analyzed the risk profile of the metabolically healthy obese individuals and found that their risk was 1.19-fold higher than the normal weight counterparts, but this slight increase in risk was not statistically significant. The confidence intervals were 0.98 to 1.38 and for this finding to be statistically significant, the lower confidence interval would have needed to be higher than 1.0 instead of 0.98.

The researchers then decided to exclude studies which did not provide at least 10 years of follow up data on the enrolled subjects. This new rule excluded studies which had shown no significant impact of obesity on survival. When the researchers now re-analyzed their data after the exclusions, they found that metabolically healthy obese individuals did have a statistically significant higher risk! Metabolically healthy obese subjects had a 1.24-fold higher risk, with a confidence interval of 1.02 to 1.55. The lower confidence interval was now a tick higher than the 1.0 threshold and thus statistically significant.

Another important finding was that among metabolically unhealthy individuals, all three groups (normal weight, overweight, obese) had a similar risk profile. Metabolically unhealthy normal weight subjects had a three-fold higher than metabolically healthy normal weight individuals. The metabolically unhealthy overweight and obese groups also had a roughly three—fold higher risk when compared to metabolically healthy counterparts. This means that metabolic parameters are far more important as predictors of cardiovascular health than just weight (compare the rather small 1.24-fold higher risk with the 3-fold higher risk).

Unfortunately, the authors of the study did not provide a comprehensive discussion of these findings. Instead, they conclude that there is no “healthy obesity” and suggest that all excess weight is bad, even if one is metabolically healthy. The discussion section of the paper glosses over the important finding that metabolically healthy overweight individuals do not have a higher risk. They also do not emphasize that even the purported effects of obesity in metabolically healthy individuals were only marginally significant. The editorial accompanying the paper is even more biased and carries the definitive title “ The Myth of Healthy Obesity”. “Myth” is a rather strong word considering the rather small impact of the individuals’ weight on their overall risk.


Some press reports also went along with the skewed interpretation presented by the study authors and the editorial.


A BBC article describing the results stated:


It has been argued that being overweight does not necessarily imply health risks if individuals remain healthy in other ways.

The research, published in Annals of Internal Medicine, contradicts this idea.


This BBC article conflates the terms overweight and obese, ignoring the fact that the study showed that metabolically healthy overweight individuals actually do not have a higher risk.


The New York Times blog cited a study author:


“The message here is pretty clear,” said the lead author, Dr. Caroline K. Kramer, a researcher at the University of Toronto. “The results are very consistent. It’s not O.K. to be obese. There is no such thing as healthy obesity.”


Suggesting that the message is “pretty clear” is somewhat overreaching. One of the key problems with using this meta-analysis to reach definitive conclusions about “healthy overweight” or “healthy obesity” is that the study authors and editorial equate increased risk with unhealthy. Definitions of what constitutes “health” or “disease” should be based on scientific parameters (biomarkers in the blood, functional assessments of cardiovascular health, etc.) and not just on increased risk. Men have an increased risk of dying from cardiovascular disease than women. Does this mean that being a healthy man is a myth? Another major weakness of the study was that there was no data included on regular exercise. Numerous studies have shown that regular exercise reduces the risk of cardiovascular events. It is quite possible that the mild increase in cardiovascular risk in the metabolically healthy obese group may be due, in part, to lower levels of exercise.

This study does not prove that healthy obesity is a “myth”. Overweight individuals with normal metabolic health do not yet have a significant elevation in their cardiovascular risk. At this stage, one can indeed be “overweight” as defined by one’s body mass index but still be considered “healthy” as long as all the other metabolic parameters are within the normal ranges and one abides by the general health recommendations such as avoiding tobacco, exercising regularly. If an overweight person progresses to becoming obese, he or she may be at slightly higher risk for cardiovascular events even if their metabolic health remains intact. The important take-home message from this study is that while obesity itself can be a risk factor for increased risk of cardiovascular disease, it is far more important to ensure metabolic health by controlling cholesterol levels, blood pressure, preventing diabetes and important additional interventions such as encouraging regular exercise instead of just focusing on an individual’s weight.

Kramer CK, Zinman B, & Retnakaran R (2013). Are metabolically healthy overweight and obesity benign conditions?: A systematic review and meta-analysis. Annals of internal medicine, 159 (11), 758-69 PMID: 24297192

Replicability of High-Impact Papers in Stem Cell Research

I recently used the Web of Science database to generate a list of the most highly cited papers in stem cell research. As of July 2013, the search for original research articles which use the key word “stem cells” resulted in the following list of the ten most widely cited papers to date:


Human ESC colony – Wikimedia

1. Pittenger M et al. (1999) Multilineage potential of adult human mesenchymal stem cells. Science 284(5411):143-147

Citations: 8,157


2.  Thomson JA et al. (1998) Embryonic stem cell lines derived from human blastocysts. Science 282(5391):1145-1147

Citations: 5,565


3. Takahashi K and Yamanaka S (2006) Induction of pluripotent stem cells from mouse embryonic and adult fibroblast cultures by defined factors. Cell 126(4): 663-676

Citations: 5,034


4. Takahashi K et al. (2007) Induction of pluripotent stem cells from adult human fibroblasts by defined factors. Cell 131(5):861-872

Citations: 4,061


5. Donehower LA et al  (1992) Mice deficient for p53 are developmentally normal but susceptible to spontaneous tumours. Nature 356(6366): 215-221

Citations: 3,279


6. Al-Hajj M et al (2003) Prospective identification of tumorigenic breast cancer cells. Proceedings of the National Academy of Sciences 100(7): 3983-3988

Citations: 3,183


7. Yu J et al (2007) Induced pluripotent stem cell lines derived from human somatic cells. Science 318(5858): 1917-1920

Citations: 3,086


8. Jiang YH et al (2002) Pluripotency of mesenchymal stem cells derived from adult marrow. Nature 418(6893):41-49

Citations: 2,983


9. Orlic D et al (2001) Bone marrow cells regenerate infarcted myocardium. Nature 410 (6829):701-705

Citations: 2,961


10. Lu J et al (2005) MicroRNA expression profiles classify human cancers. Nature 435(7043): 834-838

Citations: 2,917


Three of the articles (Donehower et al, Al-Hajj et al and Lu et al) in this “top ten list” do not focus on stem cells but are actually cancer research papers. They were probably identified by the search because the authors may have made comparisons to stem cells or used stem cells as tools.The remaining seven articles are indeed widely known in the stem cell field.


The Science paper by Pittenger and colleagues in 1999 provided a very comprehensive description of mesenchymal stem cells (MSCs), a type of adult stem cell which is found in the bone marrow alongside hematopoietic stem cells (HSCs). Despite the fact that MSCs and HSCs are both adult stem cells in the bone marrow, they have very different functions. HSCs give rise to circulating blood cells, whereas MSCs primarily form bone, fat and cartilage as was nicely demonstrated by Pittenger and colleagues.

The article by Thomson and colleagues was published in 1998 in the journal Science described the derivation of human embryonic stem cells (ESCs) and revolutionized the field of stem cell research. While adult stem cells have a very limited capacity in terms of lineages they can turn into, ESCs are derived from the early blastocyst stage of embryonic development (within the first 1-2 weeks following fertilization) and thus retain the capacity to turn into a very wide range of tissues, such as neurons, heart cells, blood vessel cells or liver cells. This paper not only identified the methods for isolating human ESCs, but also how to keep them in culture and expand them as undifferentiated stem cells.

The Cell paper by Takahashi and Yamanaka in 2006 represented another major advancement in the field of stem cell biology, because it showed for the first time that a mouse adult skin cell (fibroblast) could be reprogrammed and converted into a truly pluripotent stem cell (an induced pluripotent stem cell or iPSC) which exhibited all the major characteristics of an embryonic stem cell (ESC). It was as if the adult skin cell was traveling back in time, erasing its identity of having been a skin cell and returning to primordial, embryonic-like stem cell. Only one year later, Dr. Yamanaka’s group was able to demonstrate the same phenomena for adult human skin cells in the 2007 Cell paper (Takahashi et al), and in the same year a different group independently confirmed that adult human cells could be reprogrammed to the iPSC state (Science paper by Yu et al in 2007). The generation of iPSCs described in these three papers is probably the most remarkable discovery in stem cell biology during the past decade. It is no wonder that each of these three papers have been cited several thousand times even though they were published only six or seven years ago, and that Dr. Yamanaka was awarded the 2012 Nobel prize for this pioneering work.

All five of the above-mentioned stem cell papers have one thing in common: the results have been repeated and confirmed by numerous independent laboratories all over the world. However, this does not necessarily hold true for the other two highly cited stem cell papers on this list.

The 2002 Nature paper by Jiang and colleagues from Dr. Verfaillie’s laboratory at the University of Minnesota proposed that the bone marrow contained a rather special subset of adult MSCs which had a much broader differentiation potential than had been previously recognized. While adult MSCs were thought to primarily turn into bone, cartilage or fat when given the appropriate cues, this rare new cell type – referred to as MAPCs (multipotent adult progenitor cells) – appeared to differentiate into a much broader range of tissues. The paper even showed data from an experiment in which these adult mouse bone marrow stem cells were combined with embryonic cells and gave rise to a chimeric mouse. i.e. a mouse in which the tissues were in part derived from standard embryonic cells and in part from the newly discovered adult MAPCs. Such chimerism suggested that the MAPCs were embryonic-like, contributing to the formation of all the tissues in the mice. At the time of its publication, this paper was met with great enthusiasm because it proved that the adult body contained embryonic-like cells, hidden away in the bone marrow, and that these MAPCs could be used to regenerate ailing organs and tissues without having to use ethically problematic human embryonic stem cells.

There was just one major catch. Many laboratories around the world tried to replicate the results and were unable to identify the MAPCs, and even when they found cells that were MAPCs, they were unable to confirm the embryonic-like nature of the cells. In a remarkable example of investigative journalism, the science journalists Peter Aldhous and Eugenie Reich identified multiple irregularities in the publications involving MAPCs and documented the inability of researchers to replicate the findings by publishing the results of their investigation in the New Scientist (PDF).

The second high profile stem cell paper which was also plagued by an inability to replicate the results was the 2001 Nature paper by Orlic and colleagues. In this paper from Dr. Anversa’s laboratory, the authors suggested that adult hematopoietic (blood-forming) stem cells from the bone marrow could regenerate an infarcted heart by becoming heart cells (cardiomyocytes). It was a rather bold claim, because simply injecting these blood-forming stem cells into the heart seemed to be sufficient to redirect their fate. Instead of giving rise to red and white blood cells, these bone marrow cells were generating functional heart cells. If this were the case, then every patient could be potentially treated with their own bone marrow and grow back damaged heart tissue after a heart attack. Unfortunately, it was too good to be true. Two leading stem cell laboratories partnered up to confirm the results, but even after years of experiments, they were unable to find any evidence of adult bone marrow stem cells converting into functional heart cells. They published their findings three years later, also in the journal Nature:

Murry CE et al (2004) Haematopoietic stem cells do not transdifferentiate into cardiac myocytes in myocardial infarcts. Nature 428(6983): 664-668

Citations: 1,150


Interestingly, the original paper which had made the claim that bone marrow cells can become functional heart cells has been cited nearly 3,000 times, whereas the refutation by Murry and colleagues, published in the same high-profile journal has been cited only 1,150 times. The vast majority of the nearly 3,000 citations of the 2001 paper by Orlic and colleagues occurred after it had been refuted in 2004! The 2001 Orlic et al paper has even been used to justify clinical trials in which bone marrow was obtained from heart attack patients and injected into their hearts. As expected after the refutation by Murry and colleagues, the success of these clinical trials was rather limited One of the largest bone marrow infusion trials in heart attack patients was recently published, showing no success of the therapy.

These claims of the two papers (Orlic et al and Jiang et al) were quite innovative and exciting, and they were also published in a high-profile, peer-reviewed journal, just like the other five stem cell papers. The crucial difference was the fact that their findings could not be replicated by other laboratories. Despite their lack of replicability, both papers had an enormous impact on the field of stem cell research. Senior scientists, postdocs and graduate students may have devoted a substantial amount of time and resources to developing projects that built on the findings of these two papers, only to find out that they could not be replicated. If there is a lesson to be learned, it is that we need to be rather cautious in terms of our enthusiasm for new claims in stem cell biology until they have been appropriately confirmed by other researchers. Furthermore, we need to streamline the replicability testing process so that we do not have to wait years before we find out that one of the most highly prized discoveries cannot be independently confirmed.


Update 7/24/2013: Peter Aldhous reminded me that the superb job of investigative journalism into the question of MAPCs was performed in partnership with the science writer Eugenie Reich, the author of a book on scientific fraud. I have updated the blog post to reflect this.

Blind Peers: A Path To Equality In Scientific Peer Review?

There is a fundamental asymmetry that exists in contemporary peer review of scientific papers. Most scientific journals do not hide the identity of the authors of a submitted manuscript. The scientific reviewers, on the other hand, remain anonymous. Their identities are only known to the editors, who use the assessments of these scientific reviewers to help decide whether or not to accept a scientific manuscript. Even though the comments of the reviewers are usually passed along to the authors of the manuscript, the names of the reviewers are not. There is a good reason for that. Critical comments of peer reviewers can lead to a rejection of a manuscript, or cause substantial delays in its publication, sometimes requiring many months of additional work that needs to be performed by the scientists who authored the manuscript. Scientists who receive such criticisms are understandably disappointed, but in some cases this disappointment can turn into anger and could potentially even lead to retributions against the peer reviewers, if their identities were ever disclosed. The cloak of anonymity thus makes it much easier for peer reviewers to offer honest and critical assessments of the submitted manuscript.

Unfortunately, this asymmetry – the peer reviewers knowing the names of the authors but the authors not knowing the names of the peer reviewers – can create problems. Some peer reviewers may be biased either against or in favor of a manuscript merely because they recognize the names of the authors or the institutions at which the authors work. There is an expectation that peer reviewers judge a paper only based on its scientific merit, but knowledge of the authors could still consciously or subconsciously impact the assessments made by the peer reviewers. Scientific peer reviewers may be much more lenient towards manuscripts of colleagues that they have known for many years and who they consider to be their friends. The reviewers may be more critical of manuscripts submitted by rival groups with whom they have had hostile exchanges in the past or by institutions that they do not trust. A recent study observed that scientists who review applications of students exhibit a subtle gender bias that favors male students, and it may be possible that similar gender bias exists in the peer review evaluation of manuscripts.

The journals Nature Geoscience and Nature Climate Change of  the Nature Publishing Group have recently announced a new “Double-blind peer review” approach to correct this asymmetry. The journals will allow authors to remain anonymous during the peer review process. The hope is that hiding the identities of the authors could reduce bias among peer reviewers.  The journals decided to implement this approach on a trial basis following a survey, in which three-quarters of respondents were supportive of a double-blind peer review. As the announcement correctly points out, this will only work if the authors are willing to phrase their paper in a manner that does not give away their identity. Instead of writing “as we have previously described”, authors write “as has been previously described” when citing prior publications.

The editors of Nature Geoscience state:

From our experience, authors who try to guess the identity of a referee are very often wrong. It seems unlikely that referees will be any more successful when guessing the identity of authors.

I respectfully disagree with this statement. Reviewers can remain anonymous because they rarely make direct references to their own work in the review process. Authors of a scientific manuscript, on the other hand, often publish a paper in the context of their own prior work. Even if the names and addresses of the authors were hidden on the title page and even if the usage of first-person pronouns in the context of prior publications was omitted, the manuscript would likely still contain multiple references to a group’s prior work. These references as well as any mentions of an institution’s facilities or administrative committees that approve animal and human studies could potentially give away the identity of the authors. It would be much easier for reviewers to guess the identity of some of the authors than for authors to guess the identity of the reviewers.


But even if referees correctly identify the research group that a paper is coming from, they are much less likely to guess who the first author is. One of our motivations for setting up a double-blind trial is the possibility that female authors are subjected to tougher peer review than their male colleagues — a distinct possibility in view of evidence that subtle gender biases affect assessments of competence, appropriate salaries and other aspects of academic life (Proc. Natl Acad. Sci. USA 109, 16474–16479; 2012). If the first author is unknown, this bias will be largely removed.


The double-blind peer review system would definitely make it harder to guess the identity of the first author and would remove biases of reviewers associated with knowing the identity of first authors. The references to prior work would enable a reviewer to infer that the submitted manuscript was authored by the research group of the senior scientist X at the University Y, but it would be nearly impossible for the reviewer to ascertain the identity of the first authors (often postdoctoral fellows, graduate students or junior faculty members). However, based on my discussions with fellow peer reviewers, I think that it is rather rare for reviewers to have a strong bias against or in favor of first authors. The biases are usually associated with knowing the identity of the senior or lead authors.

Many scientists would agree that there is a need for reforming the peer review process and that we need to reduce biased assessments of submitted manuscripts. However, I am not convinced that increasing blindness is necessarily the best approach. In addition to the asymmetry of anonymity in contemporary peer review, there is another form of asymmetry that should be addressed: Manuscripts are eventually made public, the comments of peer reviewers usually are not made public.

This asymmetry allows some peer reviewers to be sloppy in their assessments of manuscripts. While some peer reviewers provide thoughtful and constructive criticism, others just make offhanded comments, either dismissing a manuscript for no good reason or sometimes accepting it without carefully evaluating all its strengths and weaknesses. The solution to this problem is not increasing “blindness”, but instead increasing transparency of the peer review process. The open access journal F1000Research has a post-publication review process for scientific manuscripts, in which a paper is first published and the names and assessments of the referees are openly disclosed.  The open access journal PeerJ offers an alternate approach, in which peer reviewers can choose to either disclose their names or to stay anonymous and authors can choose to disclose the comments they received during the peer review process. This “pro-choice” model would allow reviewers to remain anonymous even if the authors choose to publicly disclose the reviewer comments.

Scientific peer review can play an important role in ensuring the quality of science, if it is conducted appropriately and provides reasonably objective and constructive critiques. Constructive criticism is essential for the growth of scientific knowledge. It is important that we foster a culture of respect for criticism in science, whether it occurs during the peer review process or when science writers analyze published studies. “Double blind” is an excellent way to collect experimental data, because it reduces the bias of the experimenter, but it may not be the best way to improve peer review. When it comes to peer review and scientific criticism, we should strive for more transparency and a culture of mutual respect and dialogue.

Critical Science Writing: A Checklist for the Life Sciences

One major obstacle in the “infotainment versus critical science writing” debate is that there is no universal definition of what constitutes “critical analysis” in science writing. How can we decide whether or not critical science writing is adequately represented in contemporary science writing or science journalism, if we do not have a standardized method of assessing it? For this purpose, I would like to propose the following checklist of points that can be addressed in news articles or blog-posts which focus on the critical analysis of published scientific research. This checklist is intended for the life sciences – biological and medical research – but it can be easily modified and applied to critical science writing in other areas of research. Each category contains examples of questions which science writers can direct towards members of the scientific research team, institutional representatives or by performing an independent review of the published scientific data. These questions will have to be modified according to the specific context of a research study.


1. Novelty of the scientific research:

Most researchers routinely claim that their findings are novel, but are the claims of novelty appropriate? Is the research pointing towards a fundamentally new biological mechanism or introducing a completely new scientific tool? Or does it just represent a minor incremental growth in our understanding of a biological problem?


2. Significance of the research:

How does the significance of the research compare to the significance of other studies in the field? A biological study might uncover new regulators of cell death or cell growth, but how many other such regulators have been discovered in recent years? How does the magnitude of the effect in the study compare to magnitude of effects in other research studies? Suppressing a gene might prolong the survival of a cell or increase the regeneration of an organ, but have research groups published similar effects in studies which target other genes? Some research studies report effects that are statistically significant, but are they also biologically significant?


3. Replicability:

Have the findings of the scientific study been replicated by other research groups? Does the research study attempt to partially or fully replicate prior research? If the discussed study has not yet been replicated, is there any information available on the general replicability success rate in this area of research?


4. Experimental design:

Did the researchers use an appropriate experimental design for the current study by ensuring that they included adequate control groups and addressed potential confounding factors? Were the experimental models appropriate for the questions they asked and for the conclusions they are drawing? Did the researchers study the effects they observed at multiple time points or just at one single time point? Did they report the results of all the time points or did they just pick the time points they were interested in?

Examples of issues: 1) Stem cell studies in which human stem cells are transplanted into injured or diseased mice are often conducted with immune deficient mice to avoid rejection of the human cells. Some studies do not assess whether the immune deficiency itself impacted the injury or disease, which could be a confounding factor when interpreting the results. 2) Studies which investigate the impact of the 24-hour internal biological clock on the expression of genes sometimes perform the studies in humans and animals who maintain a regular sleep-wake schedule. This obscures the cause-effect relationship because one is unable to ascertain whether the observed effects are truly regulated by an internal biological clock or whether they merely reflect changes associated with being awake versus asleep.


5. Experimental methods:

Are the methods used in the research study accepted by other researchers? If the methods are completely novel, have they been appropriately validated? Are there any potential artifacts that could explain the findings? How did the findings in a dish (“in vitro“) compare to the findings in an animal experiment (“in vivo“)? If new genes were introduced into cells or into animals, was the level of activity comparable to levels found in nature or were the gene expression levels 10-, 100- or even 1000-fold higher than physiologic levels?

Examples of issues: In stem cell research, a major problem faced by researchers is how stem cells are defined, what constitutes cell differentiation and how the fate of stem cells is tracked. One common problem that has plagued peer-reviewed studies published in high-profile journals is the inadequate characterization of stem cells and function of mature cells derived from the stem cells. Another problem in the stem cell literature is the fact that stem cells are routinely labeled with fluorescent markers to help track their fate, but it is increasingly becoming apparent that unlabeled cells (i.e. non-stem cells) can emit a non-specific fluorescence that is quite similar to that of the labeled stem cells. If a study does not address such problems, some of its key conclusions may be flawed.


6. Statistical analysis:

Did the researchers use the appropriate statistical tests to test the validity of their results? Were the experiments adequately powered (have a sufficient sample size) to draw valid conclusions? Did the researchers pre-specify the number of repeat experiments, animals or humans in their experimental groups prior to conducting the studies? Did they modify the number of animals or human subjects in the experimental groups during the course of the study?


7. Consensus or dissent among scientists:

What do other scientists think about the published research? Do they agree with the novelty, significance and validity of the scientific findings as claimed by the authors of the published paper or do they have specific concerns in this regard?


8. Peer review process:

What were the major issues raised during the peer review process? How did the researchers address the concerns of the reviewers? Did any journals previously reject the study before it was accepted for publication?


9. Financial interests:

How was the study funded? Did the organization or corporation which funded the study have any say in how the study was designed, how the data was analyzed and what data was included in the publication? Do the researchers hold any relevant patents, own stock or receive other financial incentives from institutions or corporations that could benefit from this research?


10. Scientific misconduct, fraud or breach of ethics

Are there any allegations or concerns about scientific misconduct, fraud or breach of ethics in the context of the research study? If such concerns exist, what are the specific measures taken by the researchers, institutions or scientific journals to resolve the issues? Have members of the research team been previously investigated for scientific misconduct or fraud? Are there concerns about how informed consent was obtained from the human subjects?


This is just a preliminary list and I would welcome any feedback on how to improve this list in order to develop tools for assessing the critical analysis content in science writing. It may not always be possible to obtain the pertinent information. For example, since the peer review process is usually anonymous, it may be impossible for a science writer to find out details about what occurred during the peer review process if the researchers themselves refuse to comment on it.

One could assign a point value to each of the categories in this checklist and then score individual science news articles or science blog-posts that discuss specific research studies. A greater in-depth discussion of any issue should result in a greater point score for that category.

Points would not only be based on the number of issues raised but also on the quality of analysis provided in each category. Listing all the funding sources is not as helpful as providing an analysis of how the funding could have impacted the data interpretation. Similarly, if the science writer notices errors in the experimental design, it would be very helpful for the readers to understand whether these errors invalidate all major conclusions of the study or just some of its conclusions. Adding up all the points would then generate a comprehensive score that could become a quantifiable indicator of the degree of critical analysis contained in a science news article or blog-post.



EDIT: The checklist now includes a new category – scientific misconduct, fraud or breach of ethics.

‘Infotainment’ and Critical Science Journalism

I recently wrote an op-ed piece for the Guardian in which I suggested that there is too much of an emphasis on ‘infotainment’ in contemporary science journalism and there is too little critical science journalism. The response to the article was unexpectedly strong, provoking some hostile comments on Twitter, and some of the most angry comments seem to indicate a misunderstanding of the core message.

One of the themes that emerged in response to the article was the Us-vs.-Them perception that “scientists” were attacking “journalists”. This was surprising because as a science blogger, I assumed that I, too, was a science journalist. My definitions of scientist and journalist tend to be rather broad and inclusive. I think of scientists with a special interest and expertise in communicating science to a broad readership as science journalists. I also consider journalists with a significant interest and expertise in science as scientists. My inclusive definitions of scientists and journalists have been in part influenced by an article written by Bora Zivkovic, an outstanding science journalist and scientist and the person who inspired me to become a science blogger.  As Bora Zivokovic reminds us, scientists and journalists have a lot in common: They are supposed to be critical and skeptical, they obtain and analyze data and they communicate their findings to an audience after carefully evaluating their data.  However, it is apparent that some scientists and journalists are protective of their respective domains. Some scientists may not accept science journalists as fellow scientists unless they are part of an active science laboratory. Conversely, some journalists may not accept scientists as fellow journalists unless their primary employer is a media organization. For the purpose of this discussion, I will try to therefore use the more generic term “science writing” instead of “science journalism”.

Are infotainment science writing and critical science writing opposites? This was one of the major questions that arose in the Twitter discussion. The schematic below illustrates infotainment and critical science writing.

Although this schematic of a triangle might seem oversimplified, it is a tool that I use to help me in my own science writing. “Critical science writing” (base of the triangle) tends to provide information and critical analysis of scientific research to the readers. Infotainment science writing minimizes the critical analysis of the research and instead focuses on presenting content about scientific research in an entertaining style. Scientific satire as a combination of entertainment and critical analysis was not discussed in the Guardian article, but I think that this too is a form of science writing that should be encouraged.

Articles or blog-posts can fall anywhere within this triangle, which is why infotainment and critical science writing are not true dichotomies, they just have distinct emphases. Infotainment science writing can include some degree of critical analysis, and critical science writing can be somewhat entertaining. However, it is rare for science writing (or other forms of writing) to strike a balance that is able to include accurate scientific information, entertainment, as well as a profound critical analysis that challenges the scientific methodology or scientific establishment, all in one article. In American political journalism, Jon Stewart and the Daily Show are perhaps one example of how one can inform, entertain and be critical – all in one succinct package. Currently, contemporary science writing which is informative and entertaining (‘infotainment’), rarely challenges the scientific establishment the way Jon Stewart challenges the political establishment.

Is ‘infotainment’ a derogatory term?  Some readers of the Guardian article assumed that I was not only claiming that all science journalism is ‘infotainment’, but also putting down ‘infotainment’ science journalism. There is nothing wrong with writing about science in an informative and entertaining manner, therefore ‘infotainment’ science writing should not be construed as a derogatory term. There are differences between good and sloppy infotainment science writing. Good infotainment science writing is accurate in terms of the scientific information it conveys, whereas sloppy infotainment science writing discards scientific accuracy to maximize hype and entertainment value. Similarly, there is good and sloppy critical science writing. Good critical science writing is painstakingly careful in the analysis of the scientific data and its scientific context by reviewing numerous other related scientific studies in the field and putting the scientific work in perspective. Sloppy critical science writing, on the other hand, might just single out one scientific study and attempt to discredit a whole area of research without examining context. Examples of sloppy critical science writing can be found in the anti-global warming literature, which hones in on a few minor scientific discrepancies, but ignores the fact that 98-99% of climate scientists agree on the fact that humans are the primary cause of global warming.

Instead of just discussing these distinctions in abstract terms, I will use some of my prior blog-posts to illustrate differences between different types of science writing, such as infotainment, critical science writing or scientific satire. I find it easier to critique my own science writing than that of other science writers, probably because I am plagued by the same self-doubts that most writers struggle with. The following analysis may be helpful for other science writers who want to see where their articles and blog-posts fall on the information – critical analysis – entertainment spectrum.


A.     Infotainment science writing

Infotainment science writing allows me to write about exciting or unusual new discoveries in a fairly manageable amount of time, without having to extensively review the literature in the field or perform an in-depth analysis of the statistics and every figure in the study under discussion. After providing some background for the non-specialist reader, one can focus on faithfully reporting the data in the paper and the implications of the work without discussing all the major caveats and pitfalls in the published paper. This writing provides a bit of an escapist pleasure for me, because so much of my time as a scientist is spent performing a critical analysis of the experimental data acquired in my own laboratory or in-depth reviews of scientific manuscripts and grants of either collaborators or as a peer reviewer. Infotainment science writing is a reminder of the big picture, excitement and promise of science, even though it might gloss over certain important experimental flaws and caveats of scientific studies.

Infotainment Science Writing Example 1: Using Viagra To Burn Fat

This blog-post discusses a paper published in the FASEB Journal, which suggested that white (“bad”) fat cells could be converted into brown (“good”) fat cells using Viagra. The study reminded me of a collision between two groups of spam emails: weight loss meets Viagra. The blog-post provides background on white and brown adipose tissue and then describes the key findings of the paper. A few limitations of the study are mentioned, such as the fact that the researchers never document weight loss in the mice they treated, as well as the fact that the paper ignores long-term consequences of chronic Viagra treatment. The reason I consider this piece an infotainment style of science writing is that there were numerous criticisms of the research study that could have been brought to the attention of the readers. The researchers concluded the fat cells were being converted into brown fat using only indirect measures without adequately measuring the metabolic activity and energy expenditure. It is not clear why the researchers did not extend the duration of the animal studies to show that the Viagra treatment could induce weight loss. If all of these criticisms had been included in the blog-post, the fun Viagra-weight loss idea would have been drowned in a whirlpool of details.

Infotainment Science Writing Example 2: The Healing Power of Sweat Glands

The idea of “icky” sweat glands promoting wound healing was the main hook. Smelly apocrine sweat glands versus eccrine sweat glands are defined in the background of this blog-post, and the findings of the paper published in the American Journal of Pathology are summarized.  Limitations of the study included little investigation of the mechanism of regeneration, whether cells primarily proliferate or differentiate to promote the wound healing and an important question: Does sweating itself affect the regenerative capacity of the sweat glands? Although these limitations are briefly mentioned in the blog-post, they are not discussed in-depth and there is no comparison made between the observed wound healing effects of sweat gland cells to the wound healing capacity of other cells. This blog-post is heavy on the “information” end, and it provides little entertainment, other than evoking the image of a healing sweat gland.


B.     Critical science writing

Critical science writing is exceedingly difficult because it is time-consuming and challenging to present critiques of scientific studies in a jargon-free manner. An infotainment science blog-post can be written in a matter of a few hours. A critical science writing piece, on the other hand, requires an in-depth review of multiple studies in the field to better understand the limitations and strengths of each report.

Critical Science Writing Example 1: Bone Marrow Cell Infusions Do NOT Improve Cardiac Function After Heart Attack

This blog-post describes an important negative study conducted in Switzerland. Bone marrow cells were injected into the hearts of patients in one of the largest randomized cardiovascular cell therapy trials performed to date. The researchers found no benefit of the cell injections on cardiac function. This research has important implications because it could stave off quack medicine. Clinics in some countries offer “miracle cures” to cardiovascular patients, claiming that the stem cells in the bone marrow will heal their diseased hearts. Desperate patients, who fall for these scams, fly to other countries, undergo risky procedures and end up spending $20,000 or $40,000 out of pocket for treatments that simply do not work. This blog-post is in the critical science writing category because it not only mentions some limitations of the Swiss study, but also puts the clinical trial into context of the problems associated with unproven therapies. It does not specifically discuss other bone marrow injection studies, but it provides a link to an editorial I wrote for an academic journal which contains all the pertinent references. A number of readers of the Guardian article raised the question whether one can make such critical science writing appear entertaining, but I am not sure how to incorporate entertainment into this type of an analysis.

Critical Science Writing Example 2: Cellular Alchemy: Converting Fibroblasts Into Heart Cells

This blog-post was a review of multiple distinct studies on converting fibroblasts – either found in the skin or the hearts – into beating heart cells. The various research groups described the outcomes of their research, but the studies were not perfect replications of each other. For example, one study that reported a very low efficiency of fibroblast conversion not only used cells derived from older animals but also used a different virus to introduce the genes. The challenge for a critical science writer is to decide which of these differences need to be highlighted, because obviously not all differences and discrepancies can be adequately accommodated in a single article or blog-post. I decided to highlight the electrical heterogeneity of the generated cells as the major limitation of the research because this seemed like the most likely problem when trying to move this work forward into clinical therapies. Regenerating a damaged heart following a heart attack would be the ultimate goal, but do we really want to create islands of heart cells that have distinct electrical properties and could give rise to heart rhythm problems?


C.     Science Satire

In closing, I just want to briefly mention scientific satire – satirical or humorous descriptions of real-life science. One of the best science satire websites is PhD Comics, because the comics do a brilliant job of portraying real world science issues, such as the misery of PhD students and the vicious cycle of not having enough research funding to apply for research funding. My own attempts at scientific satire take the form of spoof news articles such as “Professor Hands Out “Erase Undesirable Data Points” Coupons To PhD Students” or “Academic Publisher Unveils New Journal Which Prevents All Access To Its Content”. Science satire is usually not informative, but it can provide entertainment and some critical introspection. This kind of satire is best suited for people with experiences that allow them to understand inside jokes. I hope that we will see more writing that satirizes the working world of how scientists interpret data, compete for tenure and grants or interact with graduate students.


//[View the story “Reactions to the “Critical Science Journalism” piece in The Guardian” on Storify]

Transparency Is Not A One-Way Mirror

An editorial in the journal Nature published on April 24, 2013 announces an important new step in the scientific peer review process for manuscripts that are being submitted to Nature and other Nature research journals. Authors of scientific manuscripts will now be required to fill out a checklist before they can submit their work to the journal. The title of the editorial, “Announcement: Reducing our irreproducibility“, reveals the goal of this new step – addressing the problem of irreproducibility that is plaguing science. During the past year, Nature and its affiliated journals have repeatedly pointed out that the poor reproducibility rate of published research findings is a major challenge for science and that we need to develop new mechanisms to fix this problem. This new checklist may be one tiny step in the right direction. Its primary focus is the statistical reliability of the results in a submitted paper and asks authors to disclose details about the statistical analyses employed, sample size calculations, blinding and randomization. Manuscripts involving animals or human subjects are also required to disclose details about the approvals by the appropriate review boards or committees.


            Examples of the checklist questions are:

1. How was the sample size chosen to ensure adequate power to detect a pre-specified effect size? For animal studies, include a statement about sample size estimate even if no statistical methods were used.

5. For every figure, are statistical tests justified as appropriate? Do the data meet the assumptions of the tests (e.g., normal distribution)? Is there an estimate of variation within each group of data? Is the variance similar between the groups that are being statistically compared?


The authors are also reminded that they have to reveal complete statistical information in the figure legends and evidence that datasets have been submitted to public repositories for 1) Protein, DNA and RNA sequences, 2) Macromolecular structures, 3) Crystallographic data for small molecules and 4) Microarray data.

It is commendable that the Nature editors have recognized the importance of addressing the reproducibility issue in science, but I doubt that this checklist will make such a big difference. The cynical or perhaps overly honest answer to how many biologists determine sample size is not by a pre-specified sample size calculation. Instead, they might just go ahead and perform some arbitrary number of experiments with a sample size of n=5 or so, and then adjust the sample size by increasing it, if the initial results are not statistically significant until they achieve the equally arbitrary and near-mystical p-value thresholds of p<0.05 or p<0.01. This checklist will remind authors of the importance of keeping track of statistical and methodological details, and disclosing them in the manuscript. Such transparency in terms of methods and analyses is sorely needed. This will make it easier for other laboratories to attempt to replicate the published paper, but it is not clear how revealing these details will affect the chances that the results are indeed reproducible. Will the editors perhaps not review a manuscript if the checklist reveals that the authors only studied one strain of mice? Will sample sizes of n=5 not be acceptable even if the p-value is <0.01?

This brings us to another crucial point in the debate about reproducibility of scientific results. Prestigious journals such as Nature rarely review manuscripts that are deemed to be of limited significance or novelty to their readership. In fact, the vast majority of manuscripts submitted to high profile journals such as Nature of Science are rejected at the editorial level without ever undergoing a thorough peer review process. On the other hand, when editors get a personal call from high profile investigators, they may be more likely to send out a paper for review, because the publication of the paper could increase the often maligned “impact factor” of the journal.

Attempts to improve transparency and reliability of published research should not only target scientists, but also target the editorial and peer review process. Instead of the sending out a rather cryptic “Sorry, your paper is not interesting enough for us to review”, shouldn’t editors also complete a checklist that documents how they reached their decision? A checklist that addresses questions such as:

Was the acceptance/rejection of this manuscript based primarily on the scientific rigor or the number of expected citations per year?

How were the anonymous reviewers of this manuscript selected? How many of the chosen reviewers had been suggested by the authors? 

Did the authors directly interact with the editors to influence their decision whether or not to send a manuscript out for review?

Transparency is not a one-way mirror. Scientists need to become more transparent, but the editorial and review process should also be more transparent.


Image credit: Hall of Mirrors at Versailles (Image by Myrabella – Creative Commons License via Wikimedia)