The Replicability Crisis in Cancer Research

A few months ago, we discussed the replicability issues associated with high-impact publications in stem cell research on this blog. Some of the most exciting and most widely cited papers using adult stem cells could not be replicated during subsequent studies, and resulted in a lot of conflicting data and frustration among stem cell scientists. However, this is not just an affliction of stem cell research, but it also applies to many other areas of research, such as cancer biology.


Here is an excerpt from an article I wrote for about the issues of reproducibility in pre-clinical cancer research:

The cancer researchers Glenn Begley and Lee Ellis made a rather remarkable claim last year. In a commentary that analyzed the dearth of efficacious novel cancer therapies, they revealed that scientists at the biotechnology company Amgen were unable to replicate the vast majority of published pre-clinical research studies. Only 6 out of 53 landmark cancer studies could be replicated, a dismal success rate of 11%! The Amgen researchers had deliberately chosen highly innovative cancer research papers, hoping that these would form the scientific basis for future cancer therapies that they could develop. It should not come as a surprise that progress in developing new cancer treatments is so sluggish. New clinical treatments are often based on innovative scientific concepts derived from pre-clinical laboratory research. However, if the pre-clinical scientific experiments cannot be replicated, it would be folly to expect that clinical treatments based on these questionable scientific concepts would succeed.

 Reproducibility of research findings is the cornerstone of science. Peer-reviewed scientific journals generally require that scientists conduct multiple repeat experiments and report the variability of their findings before publishing them. However, it is not uncommon for researchers to successfully repeat experiments and publish a paper, only to learn that colleagues at other institutions can’t replicate the findings. This does not necessarily indicate foul play. The reasons for the lack of reproducibility include intentional fraud and misconduct, yes, but more often it’s negligence, inadvertent errors, imperfectly designed experiments and the subliminal biases of the researchers or other uncontrollable variables.

 Clinical studies, of new drugs, for example, are often plagued by the biological variability found in study participants. A group of patients in a trial may exhibit different responses to a new medication compared to patients enrolled in similar trials at different locations. In addition to genetic differences between patient populations, factors like differences in socioeconomic status, diet, access to healthcare, criteria used by referring physicians, standards of data analysis by researchers or the subjective nature of certain clinical outcomes – as well as many other uncharted variables – might all contribute to different results.

You can read the complete article here at It is important to note that the the replicability issues were identified in pre-clinical research, i.e. lab bench studies and were not the result of varying responses between patients that often plague clinical research. Pre-clinical cancer researchers use molecular and cellular techniques that are commonly used in neuroscience, immunology, stem cell biology and many other areas of the life sciences. Therefore, all of these areas of biological research may have similarly poor rates of replicability.

This highlights the importance of conducting studies that attempt to replicate previously published data. The recent award of $1.3 million to the Reproducibility Initiative by the Laura and John Arnold Foundation is small, yet important step in the right direction, because funding for replicability testing is very hard to obtain.


Immune Cells Can Remember Past Lives

The generation of induced pluripotent stem cells (iPSCs) is one of the most fascinating discoveries in the history of stem cell biology. John Gurdon and Shinya Yamanaka received the 2012 Nobel Prize for showing that adult cells could be induced to become embryonic-like stem cells (iPSCs). Many stem cell laboratories now routinely convert skin cells or blood cells from an adult patient into iPSCs. The stem cell properties of the generated iPSCs then allow researchers to convert them into a desired cell type, such as heart cells (cardiomyocytes) or brain cells (neurons), which can then be used for cell-based therapies or for the screening of novel drugs. The initial conversion of adult cells to iPSCs is referred to as “reprogramming” and is thought to represent a form of rejuvenation, because the adult cell appears to lose its adult cell identity and reverts to an immature embryonic-like state. However, we know surprisingly little about the specific mechanisms that allow adult cells to become embryonic-like. For example, how does a blood immune cell such as a lymphocyte lose its lymphocyte characteristics during the reprogramming process? Does the lymphocyte that is converted into an immature iPSC state “remember” that it used to be a lymphocyte? If yes, does this memory affect what types of cells the newly generated iPSCs can be converted into, i.e. are iPSCs derived from lymphocytes very different from iPSCs that are derived from skin cells?

There have been a number of recent studies that have tried to address the question of the “memory” in iPSCs, but two recent papers published in the January 3, 2013 issue of the journal Cell Stem Cell provide some of the most compelling proofs of an iPSC “memory” and also show that this “memory” could be used for therapeutic purposes. In the paper “Regeneration of Human Tumor Antigen-Specific T Cells from iPSCs Derived from Mature CD8+ T Cells“, Vizcardo and colleagues studied the reprogramming of T-lymphocytes derived from the tumor of a melanoma patient. Mature T-lymphocytes are immune cells that can recognize specific targets, depending on what antigen they have been exposed to. The tumor infiltrating cells used by Vizcardo and colleagues have been previously shown to recognize the melanoma tumor antigen MART-1. The researchers were able to successfully generate iPSCs from the T-lymphocytes, and they then converted the iPSCs back to T-lymphocytes. What they found was that the newly generated T-lymphocytes expressed a receptor that was specific for the MART tumor antigen. Even though the newly generated T-lymphocytes had not been exposed to the tumor, they had retained their capacity to respond to the melanoma antigen. The most likely explanation for this is that the generated iPSCs “remembered” their previous exposure to the tumor in their past lives as T-lymphocytes before they had been converted to embryonic-like iPSCs and then “reborn” as new T-lymphocytes. The iPSC reprogramming apparently did not wipe out their “memory”.

This finding has important therapeutic implications. One key problem that the immune system faces when fighting a malignant tumor is that the demand for immune cells outpaces their availability. The new study suggests that one can take activated immune cells from a cancer patient, convert them to the iPSC state, differentiate them back into rejuvenated immune cells, expand them and inject them back into the patient. The expanded and rejuvenated immune cells would retain their prior anti-tumor memory, be primed to fight the tumor and thus significantly augment the ability of the immune system to slow down the tumor growth.

The paper by Vizcardo and colleagues did not actually show the rejuvenation and anti-tumor efficacy of the iPSC-derived T-lymphocytes and this needs to be addressed in future studies. However, the paper “Generation of Rejuvenated Antigen-Specific T Cells by Reprogramming to Pluripotency and Redifferentiation” by Nishimura and colleagues in the same issue of Cell Stem Cell, did address the rejuvenation question, albeit in a slightly different context. This group of researchers obtained T-lymphocytes from a patient with HIV, then generated iPSC and re-differentiated the iPSCs back into T-lymphocytes. Similar to what Vizcardo and colleagues had observed, Nishimura and colleagues found that their iPSC derived T-lymphocytes retained an immunological memory against HIV antigens. Importantly, the newly derived T-lymphocytes were highly proliferative and had longer telomeres. The telomeres are chunks of DNA that become shorter as cells age, so the lengthening of telomeres and the high growth rate of the iPSC derived T-lymphocytes were both indicators that the iPSC reprogramming process had made the cells younger while also retaining their “memory” or ability to respond to HIV.

Further studies are now needed to test whether adding the rejuvenated cells back into the body does actually help prevent tumor growth and can treat HIV infections. There is also a need to ensure that the cells are safe and the rejuvenation process itself did not cause any harmful genetic changes. Long telomeres have been associated with the formation of tumors and one has to make sure that the iPSC-derived lymphocytes do not become malignant. These two studies represent an exciting new development in iPSC research. They not only clearly document that iPSCs retain a memory of the original adult cell type they are derived from but they also show that this memory can be put to good use. This is especially true for immune cells, because retaining an immunological memory allows rejuvenated iPSC-derived immune cells to resume the fight against a tumor or a virus.


Image credit: “Surface of HIV infected macrophage” by Sriram Subramaniam at the National Cancer Institute (NCI) via National Institutes of Health Image Bank