Tuesday, 28 March 2017

The Random Goldfish - When confirmation bias met affective bias

In the previous post, I explored how confirmation bias can lead us to believe that something is causing a problem when really it isn't.  Although sometimes unexpected, this kind of news is not unwelcome.  Finding out that fever doesn't really cause febrile convulsions is a surprise to many, but usually a good one.  Telling people that their treatments don't work is much less popular.

Anyone who has been a clinician for a reasonable amount of time knows what it feels like to share good and bad news with someone.  When I am teaching about paediatrics in primary care, it sometimes feels like I am saying that very few treatments actually work.  This feels having something taken away form us.  The real headline is that children make themselves better in the vast majority of clinical scenarios.  It is our job as clinicians to do as much nothing as possible while looking for opportunities to give effective treatments.

One question that I am often asked is, "If these treatments don't work, why do people use them?" Good question.  I tend to assume that my sample cohort of co-workers is representative of clinicians and I work with good, conscientious people who want to give children the best treatment possible. Since I don't believe that clinicians are generally unintelligent, malicious or lazy, I will take a risk and say that there must be powerful forces at work if any of us are giving ineffective medicines to children.  The problem that leads us all to use ineffective treatments is our very desire to make children better and parents happy.

If you want something really badly and do something in an attempt to make it come about, when the thing happens, we are likely to believe that this was cause and effect.  This approach would work in a fixed environment,  For example, consider a person trying to solve a puzzle (like a Rubik's Cube).  The puzzle isn't going to solve itself, so if the person tries many different strategies and then something works, they have solved the puzzle.  This assumes that the puzzle has not been solved by chance, which in this case is extremely unlikely.

Now consider someone with a different problem.  He wants a picture of his goldfish next to the castle in the goldfish bowl.  He tries shining a light to get the goldfish to move into position.  He tries tapping the glass, placing some food and tries making waves in the tank.  Whatever he does just before the goldfish moves into position must have done the trick right?  Wrong.

Well, much of paediatrics is like that.  Childhood symptoms often fluctuate or resolve.  We want our treatments to work.  We want to make children better and parents to be happy.  These factors are the perfect ingredients for us to wrongly believe that what we did worked.  Sometimes though, the goldfish just moves.  Because it's a goldfish.

Enough about goldfish. Let's use a really common example of how confirmation bias leads us and our patients to believe in an effect that is not real.

If I tell someone that antibiotics will cure a child's throat infection within a week, you can imagine how that sounds plausible.  Then, the initial belief in my statement will be reinforced when the child does indeed get better.  The true believer does not consider that the alternative is just as plausible - that all (uncomplicated) throat infections get better with time.  You know that guy who still has the viral sore throat he caught when he was two years old?  No?  Neither do I.

Paediatrics is a branch of medicine where most illnesses will resolve with time and many symptoms that could be attributed to a treatable cause.  But we and the parents both want the problem to be treatable.  The problem is, you're too nice.  You want to help and you want everyone to leave happy. This is called affective bias (the thing that reinforces confirmation bias). The end result is that we can easily believe that we are treating a problem, when in fact it gets better on its own.

Now, bias gets a bad name in medicine, but I would like to defend bias.  Clinicians could never learn or make decisions without bias.  We would be permanently uncertain and unable to choose. Without confirmation bias, we would never notice a pattern.  Without affective bias we would never take the parents seriously or care about the child.

Bias is good.  There, I've said it.

Bias is your friend, but friends can be fickle.  The thing about friends is that despite their faults, they're still your friends.  It is good to know what to expect from them.  That way, you don't feel surprised when they do the thing that they always do.  In the case of bias, your friend wants to mislead you and get you do do things you shouldn't do.

So, what are the best examples of the confirmation bias of presumed effect in paediatrics?  I've already mentioned antibiotics for upper respiratory tract infection.  There are many more, but lets just look at one in detail as an example:

Confirmation bias (presumed effect) - 
Example number 2: Treatments for gastro-oesophageal reflux disease in infants
If there was ever a paediatric condition that fitted the brief for this subject, it is feeding problems in babies.  The symptoms that babies present with are so often simply withing normal limits for infancy.   Did you know that straining is a thing that 1 in 6 babies do and that it is called dyschezia, not constipation?  If you add regurgitation of milk (aka 'reflux'), colic and other common gastrointestinal complaints, most babies have some sort of symptom that we could treat if we chose to during the first few months of life.  With a few exceptions, these are normal for being a baby, and will resolve in time.

The cardinal sign of a self resolving problem is that there are treatments available without good evidence of efficacy.  (Cough medicines for children are a good example of this.)  By way of contrast, there are very few treatment strategies for the management of pneumonia.  I'm guessing that you probably use antibiotics.

The evidence for the available treatments for reflux disease is not good.  In many cases the evidence is that they have little effect.  Rather than posting several dozen references, I am simply going to signpost you to the NICE guideline for Gastro-oesophageal reflux disease in children and young people. (1)  In the full document there is an extensive literature review which makes for interesting reading.  The bottom line is that the evidence is usually lacking.  The evidence that we do have form research points toward little or no effect for alginates, H2 agonists and PPIs.  By their own admission, much of the advice in the guideline depends on the experience of the experts involved in the guideline writing process.

Of course the problem is that there are infants with genuine pathology.  These children often begin their visits to healthcare professionals with non-specific presentations which easily fit the bill for what is 'normal for infancy'.  If we are honest with ourselves, the niggling doubt that the child might have a significant problem is one of the factors that pushes us in the direction of pulling out our prescription pads.  After all, it won't look as bad when the child turns out to have a problem later if we were busy trying treatments instead of reassuring the parent that these symptoms are usually part of normal infancy.

Avoiding unnecessary treatments is gold standard care.  Alginates are the most frequently used medications for reflux symptoms but in my experience this treatment runs a high risk of causing constipation.  This is far from ideal if you are trying to make life easier for baby and parent alike.   Motility drugs have repeatedly been associated with dangerous cardiac side effects and PPIs have been shown to increase the risk of respiratory tract infection. (2)  The take home message from this is that, although medication is an option, we need to be sure that a treatment is really likely to be better than watchful waiting.

The NICE guidelines focus on consideration of how the infant is affected - severe distress or red flags (faltering growth, feed refusal etc.).  It is also important to consider other possible diagnoses such as UTI.

I said that there were quite a few problems that have a similar story.  Going through all of them in detail would take a long time, but here is a list of some of the treatments that lend themselves to this combined bias effect:
The evidence for all of these treatments in children is that they work rarely (antibiotics for URTI, inhalers for cough alone, suspected CMPA based on colic alone) or never (cough syrups, simethicone or lactase for colic in babies).  All of these clinical scenarios share a common theme- the likelihood that the symptom will resolve in time.

So lets come back to bias.  Confirmation bias will cause us to believe that a treatment is effective while affective bias will make us want to give something even when there is little or no benefit. "But earlier, you said that bias is good.  You said that bias is my friend!" you might well say.  I stand by that.  As long as you know how you expect your friends to behave, any misbehavior can be managed and they can still be your friends.

For confirmation bias, we need to have good evidence to justify treatment that is used for symptoms when the natural course is resolution with time.  For example, I don't need evidence to back up my belief that morphine works for pain when a child has a broken leg.  If the child feels less pain afterwards, it has nothing to do with a goldfish effect.  Conversely, I should want evidence that a treatment is effective for any of the symptoms listed above.

For affective bias, we need to harness our desire to be nice to parents and children.  That means not wasting their time with ineffective treatments or worse still causing new problems. Sometimes, doing nothing is the nicest thing that you can do.  Managing to treat where appropriate and avoid unnecessary treatment is the holy grail of paediatrics.  Ultimately, the child is the patient and we need to only give them medication that is more likely to help than harm.  At least, that's what we should be trying to do despite our biases.

Edward Snelson
Amateur Medical Errorist

Disclaimer - I am too biased to be taken seriously, even by myself.

  1. NG1 - Guideline for Gastro-oesophageal reflux disease in children and young people, NICE
  2. Orenstein et al., Multicenter, double-blind, randomized, placebo-controlled trial assessing the efficacy and safety of proton pump inhibitor lansoprazole in infants with symptoms of gastroesophageal reflux disease, J Pediatr. 2009 Apr;154(4):514-520

Friday, 3 March 2017

Why has no one told me this before? Confirmation bias - It lies to you. It lies to everyone. What has it been telling you about children? (Part 1)

Recently, after I had explained why something was a medical myth, a colleague in Primary Care looked at me and with genuine exasperation said, "Why has no one told me this before?"

Good question.  The answer to this is complicated.  A lot of the time there is a big 'Emperor's New Clothes' factor.  Declaring a myth to be untrue requires someone to burst the bubble and it is not always the case that someone is listening or that anyone wants to change their belief.

Myths start for various reasons but only persist if they are fed.  For example, very few people actually believe in the existence of fairies.   Seeing a fairy or hearing from someone who claimed to see one might change that.   People do however believe that mice like cheese without any good evidence to support this.  Where does this belief come from?  Surely we can't all have taken Tom and Jerry cartoons at face value?  Since there was never any reason to doubt what we were told we continued to believe it. Well, it turns out that, whatever the basis for the belief, it is wrong when tested scientifically (Yes, this has been researched and published!).

The best ingredients for a myth are plausibility and confirmation.  Take the old chestnut about not being allowed to use adrenaline with anaesthetic in fingers as an example.  This myth originated when lidocaine and adrenaline were commonly mixed with various things to aid anaesthesia and asepsis.  The mix often contained cocaine, procaine and boric acid.  When skin necrosis developed in fingers, the cause was not isolated, but the idea that a vasoconstrictor (adrenaline) was the cause was credible.  In fact, the other ingredients were probably to blame.  Thus a myth  has persisted for roughly a century was created by a plausible theory and repeated episodes which seemed to confirm this theory. (1)

Confirmation bias comes in several forms.  It affects how we search for, interpret and retain information.  They have been responsible for misleading us about quite a few things in paediatics.  There are so many, in fact, that it would be ambitious to put them all in a single post.  Instead I will divide them roughly into two groups - those where we have been misled about cause and those where we are misled about effect.

  • We tend to consider what the cause of something is when we witness an event.  
  • We concentrate on an effect when we think we can influence events.

For now, I am going to run through some examples of presumed cause.  Lets start with the things that you may have been told are caused by something else, but probably are not.  It works like this:

Of course sometimes, the presumed cause is real.  We have confirmation bias for a reason and in most cases it is teaching us, not lying to us.  Assumption has a bad name for itself, but is a necessary part of how we work and learn.
(No disrespect to Mrs. Sullivan, who taught me that to assume makes and ass of you and me.  Mrs. Sullivan was an English teacher and the spelling mnemonic is valid even if the statement is completely wrong in the context of exploratory learning.)

In certain circumstances, the reality is very different from our assumptions.  This is usually due to a factor that is not as obvious as the two that we have associated.

There are several examples of this below.  The one that often surprises many people is finding out that it is a fallacy that fever causes febrile convulsions.  I know, right?  I mean it's in the name and everything!  It makes sense that fever causes febrile convulsions since a child develops a fever and then has a convulsion.  We even see a correlation between febrile convulsion and fever that comes on particularly quickly (or so we think).

The only problem is that the evidence goes against this being true.  When children are treated for their fever, it seems that they have the same number of fits.  (2) So what is the cause of the fits?  Probably badness.  Badness is the stuff that infections make which causes the fever, the flu symptoms and all that.  You know, chemicals and stuff.  So even when we treat the symptoms of the infection, badness still causes the seizure to occur.  We can't get rid of viral badness.  In most cases we just make children feel better until they make themselves well.

How does this change our practice?  When I found this out, it completely changed my approach to children who had suffered a febrile seizure.  I no longer worry that treatment needs to be focused on the fever rather than the child's wellness.  Most importantly, I now tell parents that the seizure was not preventable.  Often, the parent blames themselves for failing to treat the fever adequately.  They need to know that this convulsion was not their fault.

Next up is the apparent epidemic of allergy to amoxicillin.
We have to work this one backwards from the evidence.  Approximately 95% of children who have a label of amoxicillin allergy have no allergy when tested or challenged. (3) The explanation for this poor correlation is that children of a certain age frequntly develop a rash (which is often urticarial) while ill with a virus.  Viral and bacterial infections are difficult to tell apart, so it is not uncommon for a child to be given antibiotics while unwell with a viral illness.  When a culprit is sought for the rash, the antibiotics may be blamed, though the reality was that the virus caused it.

Finding this out completely changed my practice.  By careful case selection, I take every opportunity to undiagnose penicillin allergy.

Next up: another much maligned medicine - Ibuprofen.   Ibuprofen is often avoided in children who have history of wheeze.  I suspect that this is one of the biggest cases of (wrongly) presumed cause currently in paediatric practice.  You may have been told that ibuprofen causes wheeze or that ibuprofen should be avoided in children with a history of wheeze.  Well, it turns out that this is another myth that has persists despite being disproved.

Once again, the association in space and time of the medicine and the symptoms leads to a very rational fear that it is the ibuprofen causing the wheeze.  When large groups are studied, it seems that Ibuprofen may even be protective against wheeze. (4)  I'll just leave that one with you for a minute...

So after that bomb shell, something a little more palatable but still interesting.  Growing pains are not caused by (wait for it........) growing.  In fact no one knows what causes children to have growing pains.
Feel free to file this under 'how does that change my practice?'  I just think that it is interesting that we feel the need to have an explanation for a symptom which has no known cause and no effective treatment - a bit like colic really!

Next up is something a bit more meaty.  Based on the sessions that I do for GPs here and there, I would approximate that roughly three quarters of primary care clinicians are aware that there is a concern about using ibuprofen for children with chickenpox.  I also know that the basis for this concern is poorly understood.

The truth is that this concern was raised based on a cluster of cases of children who developed severe complications of secondary infection about 20 years ago (5).  No causal link has ever been convincingly shown and the fact that huge numbers of children continue to have ibuprofen in this context makes me think that more robust evidence would have emerged if there was genuine cause and effect.

Invasive streptococcal infection during varicella infection is something that all clinicians should know about.  It is also true that most children who have chickenpox are not very unwell and so paracetamol should be all that is needed.

So why does this matter?  It matters when someone is blamed for something based on poor evidence.  So, let's be clear.  The Emperor appears to be naked, but if anyone else can see that he's got clothes on, I am prepared to be convinced.

Edward Snelson
Non-steroidal guardian of the year 2014-2016

Disclaimer- I would never use any of the treatments listed above.  For many years now I have only used fairy magic to treat my patients and any prescribed medication is a pretence.  No one can prove to me that fairies don't exist. 

  1. Bradon et al, Do Not Use Epinephrine in Digital Blocks: Myth or Truth?, Plastic & Reconstructive Surgery, February 2001
  2. A Sahib El-Radhi, W Barry, Do antipyretics prevent febrile convulsions?, ADC, Volume 88, Issue 7, 2003
  3. Caubet JC et al., The role of penicillin in benign skin rashes in childhood: a prospective study based on drug rechallenge, J Allergy Clin Immunol. 2011 Jan;127(1):218-22.
  4. Kanabar et al., A review of ibuprofen and acetaminophen use in febrile children and the occurrence of asthma-related symptoms, Clinical Therapeutics, Volume 29, Issue 12, December 2007, Pages 2716-2723
  5. Zerr DM. et al., A case-control study of necrotizing fasciitis during primary varicella, Pediatrics. 1999 Apr;103(4 Pt 1):783-90.