Every now and then, there’s a report that totally changes the scope of a field. When it’s good, it can feel like the promised land is right around the corner. But when it’s bad, it feels like the sky is falling, and the stock prices with it. A few months ago, a “bad news” bombshell seemingly dropped on the CRISPR genome editing field when a paper claimed to show a major flaw with the technology, to the tune of a one-day drop of about $200M in the valuation of the three publicly-traded CRISPR companies (Editas, CRISPR Therapeutics, and Intellia)*.The problem was that none of this was warranted - the paper didn’t actually change the field, only its perception. The study in question, “Unexpected Mutations after CRISPR-Cas9 editing in vivo”, conducted by scientists at Stanford and The University of Iowa, carried a particularly fatal experimental flaw - a faulty control. The experiment went like this: the researchers used CRISPR embryo editing to create genetic mutations in a mouse embryo and then allowed that embryo to develop, at which point they sequenced its genome. At the same time, they sequenced the genome of another mouse that they thought had the same genetic background as the CRISPR-modified mouse, such that any DNA sequence differences they observed between the two mice they could ostensibly attribute to CRISPR.This comparison showed more than 1,000 (!!!) genetic differences between the mice, a truly surprising result. Keep in mind that if the CRISPR editing was totally specific, the two mice would only have one genetic difference, so every change on top of that is presumably an unintended (read: bad) off-target mutation. No one has ever claimed that CRISPR genome editing is perfectly specific, but 1,000 off-target mutations is a few orders of magnitude more than anyone in the field would have reasonably predicted. And beyond that, the off-targets seemed to mostly occur at genomic sites that didn’t really resemble the on-target sequence, meaning that the off-target mutations were completely unpredictable and could in principle happen at any gene in the genome. All that would be extremely problematic, because if CRISPR is causing that many random off-target mutations all across the genome, it’s almost certainly never going to be viable as a therapeutic since its risk-reward ratio will always be way too out of whack.As it turns out, this was much ado about nothing. As was initially speculated by statements from the CRISPR companies themselves, there was something very wrong with the report, as subsequent analysis from multiple groups (including one co-authored by my mentor, Keith Joung) found that the CRISPR-edited mouse and the control mouse weren’t actually comparable to one another. The control mouse, while related to the CRISPR mouse, didn’t actually have the same genetic background, and the many genetic differences between the mice were almost certainly just the normal genetic variations that exist between two related individuals of the same species.The immediate coverage of the paper pretty much presented its findings as fact, and presumably that’s what caused the drop in stock valuation. As more and more responses called into question its findings, and coverage started to reflect this doubt, the stocks inevitably recovered, and in the end this is just going to be a minor blip in the overall CRISPR story. For their part, the original authors are standing by their results. One rebuttal can be found here, but as you can probably tell, I’m not buying it. But for those who care about CRISPR - investors, futurologists, or just fans - who aren’t necessarily in a position to dive deep into CRISPR papers on their own, I think this is a pretty interesting case study to illustrate some of trust problems in the CRISPR field and the difficulties you might run into when you’re trying to get a handle on the truth. So to help with this, I’ve come up with a quick list of things you need to consider when you’re reading about CRISPR that I’ve heard a number** of people start to call “The Angstman Rules”:

The Angstman Rules for Reading about CRISPR

• Don’t be surprised by anything.

If you are at all connected to the genome editing field, you know that competition to publish is fierce among scientists. But what you may not realize is there’s also competition for journals to publish CRISPR papers, to the point that there’s a general sense that sometimes less-than-stellar papers can sneak through publication if they have to do with CRISPR or genome editing on their “splash” factor alone. The splashier and more surprising the paper, the more skeptical you should be about it.

• If something does seem surprising, wait for confirmation.

Obviously, pioneering CRISPR splashes are sometimes real. But when they’re real, they’re always corroborated by a second independent study that follows. The good news with CRISPR is that there’s almost always someone eagerly working to duplicate or build off of surprising results immediately after their publication, or sometimes even before the initial study comes out, so confirmation never takes very long. And if the paper is wrong, like the one I profile above, the field is often even more eager to correct the scientific record. So if you’re invested in CRISPR, either financially or emotionally, I’d caution you to not make any decisions on the basis of one paper. Just wait for the inevitable follow up - it won’t take very long.

• Initial coverage might not be reliable or critical enough.

The first reports of any major CRISPR development are going to come from sources looking to get clicks (who isn’t these days?), and they often do so by filling their articles with a healthy dose of optimism and promise and short on criticism. And if it affects CRISPR stock prices, you’re going to see pieces reporting on the financial angle that are likely to take paper’s conclusions at face-value without too much additional research. Don’t let a reporter who doesn’t do their due diligence make your conclusions for you.

• If someone finds a dramatic-yet-obvious problem with CRISPR, they’re probably doing something wrong.

I’m not absolutely sure about this, but besides umbrella topics like climate change and cancer, CRISPR might just be the most studied single topic in science right now. If a research group comes out of left field with a dramatic claim that CRISPR is doing something drastically different than what was previously known, it probably disagrees with a dozen other published studies and it almost definitely won’t end up being right. This is especially true for the topic of off-target effects, which is one of the major areas of focus for CRISPR engineers and has been studied extensively (though not exhaustively).

CRISPR Outlook and Accountability

As you can probably tell, I’m pretty excited about CRISPR genome editing and its outlook as a therapeutic. No, it’s not perfect, but I also think it’s so well-studied that at this point any major bombshells that drop on this field need to be looked at with a very skeptical eye. With CRISPR, now more than ever, scientific accountability is something it would be wise to be vigilant about when trying to read and interpret new findings.*On May 30th, Editas dropped from $15.62 per share to $13.75, CRISPR Therapeutics from $15.04 to $13.62, and Intellia from $13.27 to $11.38.**One is technically a number.Disclosure statement: Editas has licensed some of my intellectual property and they employ several of my former colleagues and friends.Note from the author: While I hold the views of my colleagues in high regard, the opinions expressed in this blog are mine alone and not necessarily representative of those held by my laboratory or departmental affiliations.