Comments on “Doing being rational: polymerase chain reaction”

Comments

Really really appreciate the

Romeo Stevens's picture

Really really appreciate the careful demonstration of ethnomethodology. I suppose the bonus points are to recurse and do the same for this post. This leads to an insight about paranoia or OCD event horizons i.e. when the process of checking for correctness (ultimately, safety) of a process is itself regarded as potentially incorrect and the base case for this cognitive recursion is missing or corrupted.

Bringing Back Memories

Over a decade ago I was a undergraduate researcher in college doing experimental work and remember explicitly wondering if, instead of having papers, scientists should just record everything they do in the lab and publish that. A video nabs all of the details that a paper never could. I also remember being quite frustrated with attempting to use techniques from papers, as they never gave enough details to actually re-create what they had done.

I’ve heard of programmers Twitch streaming their work. I’ve never looked into this directly, but a quick search shows that something like this exists. Might be interesting.

Great post but...

zer0's picture

It’s the status of such mistakes that makes PCR a formally rational activity, and not a “merely reasonable” one (in the terminology of The Eggplant).6 If you miss or double a reagent, the results of your PCR are incorrect.

Whether the waffles are bad, or inedible, is—in ethnomethodological terms—a matter of hermeneutic, accountable interpretation and negotiation. (Your family gets a say, and will definitely have opinions.) Whether your performance of PCR is incorrect is—almost always—an objective fact.

Could you expand on what makes a ‘merely reasonable’ activity different from a formally rational one? I agree that if you miss a step/mess up your reagent prep, the results of the PCR will be ‘incorrect’. But isn’t this also a matter of ‘hermeneutic, accountable interpretation and negotiation’?

Rationality in practice

Romeo—interesting insight, thank you!

Benjamin—Check out JoVE, a peer-reviewed video journal for scientific methods. (Unfortunately available only to institutions by subscription.)

zero—Yes, ultimately the distinction between reasonableness and rationality is somewhat nebulous (although useful). Everything is always at least somewhat squishy. However, there’s pretty much a fact of the matter about whether or not you remembered to add the magnesium chloride, and if you didn’t, your results are just wrong.

molecular biology collide with the granularity of living things

Ed Giniger's picture

Molecular biology is unquestionably a rich subject for the sort of analysis you are doing, and I can’t imagine any actually thoughtful practitioner disputing your description. One of the most common turns of phrase you will hear in any molecular biology lab is to say “thus and such is how this procedure works in my hands” (emphasis added). It is an absolutely essential qualifier to nearly anything one does in the lab. The protocol I swear by is often the protocol you swear at. I can list any number of procedures that work every time for me but never for my colleagues, while they have their own variants that work entirely reliably for them, but are entirely unusable for me. In the world of molecular biology, it is extraordinarily difficult for two people to get the same result simply by following the same series of written-out steps. There is a nearly endless list of trivial reasons for two people to get different results: holding a tube differently so heat transfer from the investigator to the sample is slightly different; shaking a sample a little more or less vigorously as one balances the need to mix components with the oxidative damage produced by air bubbles in a biological reaction; differences in how quickly one pipettes and therefore the amount of reagent that either fails to enter the tube or fails to be expelled (or that sticks to the outside rather than draining off). I assume that similar stories could be told about any experimental process; any process in which one’s rational expectations have to interact with the realities of the world outside.

As an aside, David, if you want a naïve, unrehearsed video of molecular biology practice, you may recall that you once took video of me in my lab at UCSF doing steps in a DNA subcloning project when you were toying with the idea of building a general-purpose lab robot. Unquestionably, if you were to look at it again, you would see an intermixture of steps that are written down nowhere (since they were so utterly routine), steps left-out as compared with all of the published protocols (since, in my hands, they were either irrelevant or frankly counterproductive), as well as steps using memory aids such as itemized reagent lists to ensure that I used the correct amounts of all reagents – even though I had done this procedure untold thousands of times before.

Sorting by title

James's picture

Offered more for amusement rather than substantive commentary:

Just tonight, as I was contemplating how I sort my Nintendo Switch game collection, I hit on what I think is a simple example where no general, systematic solution seems likely to work:

Do you sort “The Legend of Zelda” games under L, T, or Z?

If you were sorting strictly alphabetically, they would be under T. But of course we often (though not always) skip the leading article in a title. So that would put them under L. Except that I and pretty much everyone else think of them primarily as “Zelda” games instead of “Legend of Zelda” games, and so they actually go under Z. This is just something you have to know.

(It doesn’t hurt that the second game was called Zelda II instead of “The Legend of Zelda II”, but if anything that only complicates the matter instead of simplifying it.)

Thinking on paper

Peter Corbett's picture

I must have been 14 or so and was working on some maths in class. Things were going fine with my equations until at some point everything stopped making sense; things got odd and then downright broken. I spent a while going through what I’d done, trying to find out where the strangeness was. Eventually I found the answer - a place where I had written a “2” and read a “z”, and all of the equations that followed from that error had gone wrong. So I went back and re-did things reading the “2” correctly this time and it all made sense again.

(Oh, wait, that’s not just thinking on paper, repair too. Doubtless you’ve got something lined up to discuss the “this is nonsense, I must have made a mistake somewhere” moments.)

Various nebulousnesses

Ed—Thanks for this account! Thank you also very much for having been an informal experimental subject :)

I’m somewhat surprised by how unreproducible you describe methods as being. I’d be interested to understand how biologists deal with the problems this must cause.

James—Nice example! You might find interesting Bowker and Star’s Sorting Things Out: Classification and Its Consequences, which is about this sort of thing.

Peter—Yes! I’m planning to do an example with a sign error somewhere in the middle of a long algebraic simplification. Same idea. This might be even better!

One thing that is interesting, if it is long, is the gradually dawning feel that something must have gone wrong, because the terms are looking increasingly “odd,” as you said. There’s no definitive moment of being able to definitely say it’s wrong, but eventually you just say “this looks ugly, I must have screwed up somehow.”

what is "reproducibility" in biology

Ed Giniger's picture

Experimental biology is not as bad as it sounds. Partly, of course, this is the point of error bars! But more seriously, it has to be understood that reproducibility means different things in different disciplines. In many (not all) parts of physics, where the boundary conditions can be controlled better and the variables are better defined, it is reasonable to expect “reproducing” a result to be quantitative to very high numerical precision. In biology, the “variables” tend themselves to be extremely complex processes and the boundary conditions have a habit of getting up and walking away. If an answer is in some sense “correct” we can expect trends to be reproduced; we can expect principles to be reproduced, we can expect relationships to be reproduced. In many cases, however, expecting exact values to reproduce is more about physics-envy than it is about rigorous biology. As you may know, the physicist-turned-biologist Max Delbruck long ago enunciated a ‘principle of limited sloppiness’: be very sloppy and you’ll never get anything but garbage, of course, but if you really want to know if you’re on the right track in biology, be willing to try an envelope of conditions around the ones in the protocol. Results that are true tend to be robust to small variation; results that are so brittle that no variation can be tolerated are apt to be telling you more about your assay system than they are about the subject you think you are studying. This is also why, in biology, if a first-rate molecular biologist really wants to know if a result is true, s/he doesn’t just repeat the experiment, they ask whether an entirely unrelated experiment supports the same fundamental idea. An artifact can be every bit as reproducible as a true result, but if two different approaches (genetics AND biochemistry; structural analysis AND mutational analysis; computation AND imaging) tend toward the same conclusion, then it is far more likely to be a meaningful statement about how the biological process works.

Now, you can say that this sounds a lot like building on sand; that one can get very far down the rabbit hole with confirmation bias, thinking you are learning things, when in fact you are piling fantasies on fantasies. And you would be right – there is unquestionably a danger here. We have all gone wrong this way at some point (and some never manage to realize it and crawl back out). Not to mention that I suspect this is a way of doing science that would be quite unrecognizable to most philosophers of science. But I’m not sure it is as different from other forms of science as we like to pretend. You’ve read Feynman – the best physics is often done by looking at the aesthetics of the ideas and equations first; at whether they feel right; and only later filling in the blanks to make them rigorous. Maybe biologists (at least the thoughtful ones) are just more honest about what they are doing.

Biology is different

Thanks, Ed, this was very interesting. It articulates several points I think I understood tacitly but hadn’t quite put together. Especially this:

if a first-rate molecular biologist really wants to know if a result is true, s/he doesn’t just repeat the experiment, they ask whether an entirely unrelated experiment supports the same fundamental idea. An artifact can be every bit as reproducible as a true result, but if two different approaches (genetics AND biochemistry; structural analysis AND mutational analysis; computation AND imaging) tend toward the same conclusion, then it is far more likely to be a meaningful statement about how the biological process works.

The main thing that seems to have happened in the philosophy of science in the past ~40 years is that some of them have realized that “philosophy of science” was really “philosophy of fundamental physics” and nothing else in science works that way. Especially, biology doesn’t.

A good intro text on this is Peter Godfrey-Smith’s Philosophy of Biology (2014).

One way of summarizing it is that fundamental physics finds laws, which are mostly absent in biology. Biology finds mechanisms, which aren’t the same thing.

I like your your summary line

Ed Giniger's picture

I like your your summary line - I think it captures the idea. I think I’ll use it on my students.

Navigation

You are reading a metablog post, dated November 15, 2019.

This was the most recent metablog post.

The previous metablog post was An exercise: meta-rational phenomena.

This page’s topic is Rationalism.

General explanation: Meaningness is a hypertext book (in progress), plus a “metablog” that comments on it. The book begins with an appetizer. Alternatively, you might like to look at its table of contents, or some other starting points. Classification of pages by topics supplements the book and metablog structures. Terms with dotted underlining (example: meaningness) show a definition if you click on them. Pages marked with ⚒ are still under construction. Copyright ©2010–2019 David Chapman. Some links are part of Amazon Affiliate Program.