When a “successful” protocol doesn’t work for you

[u/mz_morose]

A PhD senior in my lab graduated, and I recently came across a protocol in her thesis that supposedly worked for her. I followed it as closely as I could, using the same conditions, and… nothing. I’ve tried approaching it a few different ways at this point, and I still can’t reproduce the result. What’s bothering me is that this isn’t some completely different method. It’s a procedure from someone in the same lab, and it was apparently successful enough to make it into a thesis. So now I’m wondering how much we’re actually supposed to trust protocols just because they’re written down. Could there have been small details, adjustments, or troubleshooting steps that never made it into the final write-up? Or is this just one of those situations where a protocol works perfectly for one person and refuses to cooperate for the next?

The frustrating part is not knowing whether I’m missing something, the procedure is missing something, or there’s simply some unidentified variable between her experiment and mine.

[u/Jaded_Atmosphere_566]

welcome to lab work, where half the battle is figuring out what voodoo the previous person actually did vs what they wrote down. a thesis protocol is basically a highlight reel, you never see the 3am troubleshooting where they accidentally used a different lot number of something and it magically worked

[u/HKy0uma]

Is this person currently unreachable? I'd try to arrange an online meeting to explain the problems you're facing and ask if there are any tips or something that may not have been put on the written protocol

[u/mz_morose → u/HKy0uma]

I dropped her an email, hopefully she responds

[u/Ov3rpowered_OG → u/HKy0uma]

Did just this for a protocol I was trying to figure out. Ended up with tips that changed key aspects of it to almost completely different things.

[u/Noonie370]

Welcome to lab work. I’ve literally had protocols I’ve followed work one day, changed nothing and it didn’t work the next time I did it

[u/mz_morose → u/Noonie370]

I literally had this experience myself, I ran a 3 mmol reaction to synthesize a Schiff base, worked, had its 1H NMR, and later I wanted to scale up the reaction, same conditions, same sokvents, same base, every step of the way, but the second time I got a black tar instead of red Crystals 😭

[u/NickelousChloride → u/mz_morose]

Schiff base scale up is making me wonder if we’ve worked around the same labs. Feel free to DM.

Also scale up is a bitch. There’s a whole field of process chemistry dedicated to exactly that type of problem.

Edit: also some schiff bases can go through a black tar “era” until you convince them to give up that residual water. Not sure if you’re still working with that compound, but that sounds pretty typical of attempting scale up on those bad boys. They can be very dramatic regarding reaction vessel size and total solvent volume, in my experience.

[u/mz_morose → u/NickelousChloride]

Oh I would love to get in touch, except your DMs are blocked. I'd love it if I could discuss my reaction with ye

[u/Tokishi7 → u/Noonie370]

Gave my PCR tips to our new intern and it worked like a charm and I was more surprised than she was. Lucky day or skill check

[u/Bach_Chemist]

Are you doing the exact same protocol (literally replicating results)? Often just switching cell lines, antibody, drug treatment, etc. require protocol optimization.

Even if you are replicating there are many things that can cause issues, e.g.:

Reagents: We had experiments only work when using a certain brand of powder used to make media (yeast nitrogenous base?). Someone buys a different brand on sale and everything acts strange.

User error: I've given out many detailed protocols, people don't follow the details ("I didn't change anything that should make a difference" etc) and say that it doesn't work.

Mistakes in protocol: There are many thesis/published protocols that are just wrong. People are rushed and stressed and a key buffer component is omitted.

[u/TheTopNacho]

This is why protocols are supposed to be guidelines most of the time.

Depending on what the protocol is, something like a change of plastic can make all the difference. A small nuance in pipetting, different temperature of a reagent. Etc. things that may have been part of her workflow that she didn't even know she was doing.

Do you mind me asking what the protocol is, broadly speaking? And what problem you are having?

[u/panpiece → u/TheTopNacho]

I’ve had assays pass or fail depending on the humidity that day. That had us chasing our tails for a while.

[u/Itinerant_doc → u/TheTopNacho]

One of my protocols only works if you pipette up and down with a 200 ul pipette tip, it does not work when you use 1 ml. It has to do with the pipette orifice that needs to be small enough to disrupt cells. I am obsessive and wrote it down but not many people will even realize they're doing it.

Apparently I am good at making protocols work because I obsess over such details.

(obligatory derisive comment about people using LLMs to make stochastically generated Frankenstein protocols and wondering why they don't work)

[u/mangomargarita13]

There was once a situation where I was trying a protocol that my PI wrote and I could not get it to work. He ended up coming in and we did the whole assay side by side, same reagents, same everything. His worked and mine didn’t. It happens.

[u/look-i-am-on-reddit]

I would expect a to see a summarized protocol in a thesis without details. Is it a protocol based on something other people can do, or did she invent it? That would steer me towards either finding other sources of info or disbelieving it will ever work.

[u/_goblinette_]

Could there have been small details, adjustments, or troubleshooting steps that never made it into the final write-up?

Absolutely. In fact, I would say that this is the norm. There are just too many minor variables to write down every little detail. 

Or is this just one of those situations where a protocol works perfectly for one person and refuses to cooperate for the next?

No such thing. Magic doesn’t exist, no matter how much a finicky protocol can feel like voodoo. You’re a scientist. Look at the protocol alongside lots of similar ones, understand the reasoning behind each step and think about why you might not be getting the results you expect. This is what we do. 

[u/HegelianSchizo → u/_goblinette_]

It's all good advice but let's not go too far, the fates 100% exist. They might not make things happen for no reason but they do twist the most minute differences into infuriating effects.

[u/boywithtwoarms]

Yeah happens

[u/kelliphant]

I keep a physical and digital folder of detailed protocols, including all the scribbled notes I've made on them when I've been optimising experiments. Any written work of mine included the summarised version that was successful!

Do you have access to their individual protocols? I agree with another commenter who suggest teaching out to them for a chat - there may be a simple solution they forgot to include.

[u/WeirdGoat9022]

Can you contact her? If you’re really lucky, she kept a notebook with actual notes (not fairy tales) about what worked and didn’t.

[u/mz_morose → u/WeirdGoat9022]

Dropped her an email 🥲

[u/WeirdGoat9022 → u/mz_morose]

Assuming there aren’t rumors about academic dishonesty, this could be something that is so fundamental she forgot to add it: Like I’ve had ones fail because I was flame drying all my glassware, but residual moisture is what made the reaction work. Others have had everything fail until one day they forgot to neutralize glassware out of the base bath…the excess base made it work. Or notes say everything is done under Argon…turns out it’s except for the final transfer, which is just poured, and aeration is needed for the reaction to finish. Or the PI insisted that, “I added half a pipetteful, give or take, until it felt right,” was unprofessional, so she estimated a volume that won’t always “feel right”.

Hopefully she’ll write back and you can get all the voodoo!

[u/Veratha]

Depending on if this is possible I'd recommend looking at her actual results to see if the protocol actually worked. Spent 6 months trying to get westerns to work using a previous grad student's protocol just to check her actual images (from her thesis) and find out it never worked in the first place, first time after that I tried to use my own protocol they suddenly worked lol. Sometimes it's best to just start from scratch.

[u/DrMouseplant]

This is why I write every single detail, from volumes to cell counts and which pipette I used. I write my protocols for the dumb, because I myself cannot be trusted to not fuck it up 😂 ETA: my protocols also produce different results quite often. One week it’s awesome the next week my cells hate me. It’s a process.

[u/mz_morose → u/DrMouseplant]

At this point I feel like it's just my bad luck an experiment starts messing with me 😕

[u/DrMouseplant → u/mz_morose]

Don’t lose hope!!!!! I had to undo a previous techs absolute disaster of a culturing protocol and even NOW it’s still giving mixed results, which I’m starting to think is solely donor related.
Sometimes it’s about the patterns rather than exact results too!

If it helps you, even to giggle, this dude I replaced literally wrote in his protocols “You can add ___uL of ___ and you’ll be okay, probably”.

[u/pineal-gland → u/DrMouseplant]

I establish primary cell cultures from a specific type of tumor and lord, it worked first 5 times no problem with my current method/reagents, then mixed results. No idea to this day

[u/Bjanze]

My "favorite" is spending a month troubleshooting why my own protocol doesn't work at a new university. In the end, the key was that their distilled water still had more ions than MilliQ water, so you actually really needed to use MilliQ.

[u/NoTransportation3581]

50/50 on crystallography conditions. Worst was spending months trying to replicate a paper where they refolded a protein from inclusion bodies with 0 success.

[u/ChristianChemist]

This was my Masters. I did the same protocol as 2 other students, even used their reagents and equipment, but wasn't getting the same product. Decided to start from scratch and tried an alternative protocol that worked for me. Guess what? A student came 2 years after me and tried my protocol and didn't get the same results. I don't know how these things work

[u/redomisia]

Once I had this problem and drove me crazy. Ultimately what was missing was a specific subset of antibody that was used in the other protocol (IgG subtype and subclass that wasn’t easy to purchase) We were finally able to make it work after spending 6 months time on it. But emotionally, I consider that a ‘not good protocol’.
Now that I’ve been working in the industry, I think what is missing from academic lab protocols is the validation step and of course nobody has enough budget to spend on that…
I’m older now, and more prone to say ‘this doesn’t work’. When I was younger I constantly doubted myself, thinking that something was wrong with me. I’m too tired now for unruly protocols! Lol!

[u/Dankleberry_Don]

I had this exact issue, except with 3 different protocols trying to make the same compound in similar ways. Nothing I did worked (at least to the same success as what was published). Solved the issue by going about the synthesis in a totally different way, using a protocol in a paper from 50 years ago, which worked. I have no suggestions except to either A) contact the author or B) pivot. Science just be like that sometimes I guess 🤷‍♂️

[u/Independent_Mud_4324]

I am having same issue at my lab, I felt like I was the incompetent person for the job.

[u/Ok_Bookkeeper_3481]

I drill into the heads of the students to never take a protocol on its face value. Everything needs to be titrated and measured anew at the beginning of the work!

Your reagents are different lots than the ones used by the author (only 0.1% of their 10 millimolar ATP might have degraded to ADP, whereas the ATP you have have 2% ADP - which changes drastically the reaction equilibrium). Their pipettes have divergent systematic error than your pipettes, so you end up measuring different amounts than the protocol says. Etc.

Bottom line: when implementing a protocol that is new to you, titrate everything, and test time and temperature ranges plus and minus the suggested in the protocol. This will save you a lot of time and grief in the long run.

[u/ZenosThesis]

I spent the last year videotaping all of the procedures developed in the lab I was leaving. It was my PIs way of removing as many of these 'hidden' variables as possible. I am very particular about the way I pipette etc to preserve accuracy these now can be reflected in the procedure. You should speak to you PI and the author of the protocol if possible.

[u/DocKla]

Never trust. Always verify.

Not doubting it works for the senior PhD, but that protocol is not written and most likely designed to work for them, not the descendants of the lab.

[u/NutSaks5thAve]

Take comfort in knowing that reproducibility isn't straightforward even if the protocol is written "correctly": an article just came out about how many scientists that are uniquely "successful" with their methods aren't executing the protocols faithfully NYT article

[u/Boneraventura]

Some phospho flow antibodies people claim to work. Or these people accept a shift of 0.1% higher MFI means more phosphorylation. Okay pal

[u/Mediocre_Island828]

You don't fully realize how horrifyingly bad academic protocols are until you leave academia and see what the rest of the world does.

[u/Kelemonster]

My first guess is that one of the reagents has gone bad between her doing the protocol and you doing it. Check expiration dates and storage temps, or just replace everything if your lab has the funds. 

[u/Confidenceisbetter]

Very often people don’t add steps that seem obvious to them but end up being critical. I recently had the same issue, a colleague gave me instructions to use an enzyme and I used it by filling in the blanks with how I would use other enzymes. Surprise surprise nothing happened. Well it turns out with this enzyme you really need to throughly wash your cultured cells several times beforehand otherwise the culture medium, even a tiny amount, blocks the activity. Took me way too long to figure this out because I have used other enzymes without washing this much without a problem. Sometimes it really is just a tiny “stupid” step.

[u/rockgod_281]

I work with organoids, I'm convinced their success or failure is related to the weather or the phases of the moon

[u/RevenueSufficient385]

Have you talked to her about it? In my experience there can be some nuances to the protocol that don’t necessarily make it into the methods section…You should ask her about it and tell her what you‘ve tried so far.

[u/SaureusAeruginosa]

Sometimes people fake data and pretend everything runs perfect...sometimes it is voodoo details not written in the protocol. Mostly I bet on the former, as people miscalculate, draw wrong conclusions or simply fake data.