Skip to content

Archive for

9
Oct

Acupuncture verses Coin Toss

I ran across a lady in my area on Google Plus recommending a chiropractor/acupuncturist:  Dr. Max Norris of Ozark Herb & Spice. She said anyone suffering should try him out. Browsing his website, he lists quite a lot of conditions that he can cure through acupuncture, including:

  • Migraine headaches
  • Stress
  • Diabetes
  • High blood pressure
  • ADD
  • ADHD
  • PMS
  • Menopause
  • Fibromyalgia
  • Anxiety
  • Insomnia
  • Depression
  • Vertigo
  • Tinnitus
  • Epilepsy
  • Arthritis

That’s quite an exhaustive list. But how well do these claims hold up to scientific scrutiny? Being limited in time, I decided to focus on just one claim, depression. I asked Dr. Norris what sort of studies had been done for this. He provided me list of 12 studies backing up the claim. I don’t have time to read up on 12 studies, so I asked which one had the best evidence. Since I never got a response, I decided to pick the very first one.

The Efficacy of Acupuncture in the Treatment of Major Depression in Women. Psychological Science, September, 1998; Vol. 9, No. 5: pp. 397-401.

In this study, the experimenters broke people up into three groups.

  • Specific treatment – these people would receive acupuncture specifically designed to treat depression.
  • Non-specific treatment – these patients would receive real acupuncture, but the treatment would targeted towards something besides depression.
  • Wait list – These people would remain on a wait-list for 8 weeks, and after that receive 8 weeks of treatment.

So, it seems like they made a reasonable effort to blind the study and have control groups to measure against. That’s good.

Results from the abstract:

Following treatments specifically designed to address depression, 64% of the women (n = 33) experienced full remission. A comparison of the acute effect of the three 8-week treatment conditions (n = 34) showed that patients receiving specific acupuncture treatments improved significantly more than those receiving the placebo-like nonspecific acupuncture treatments, and marginally more than those in the wait-list condition.

According to the abstract, they received a 64% success rate out of 33 people.  However, If you read further into the study, it says:

“Patients were randomly assigned to one of three conditions (specific treatment: n = 12, nonspecific treatment: n = 11, or wait list: n = 11) and to one of the acupuncturists).”

So, actually, there were 38 total in the entire experiment. 5 dropped out, leaving 33 total. Only 12 were in the group that got the real treatment.

That sample size seems pretty low. Don’t scientific studies need large sample sizes to have accurate results? Are the results in this study really better than flipping a coin? I’m no scientist, and I don’t understand statistics. Being a programmer, I decided to write a program to test this. This is coded in python. It flips a coin however many times you tell it to and outputs the results.

from __future__ import division
import random

num_heads = 0
num_tails = 0
times_flipped = 0
num_flips = 12

for i in range (num_flips):
    result = random.randrange(2)
    if result == 0:
        num_heads += 1
    else:
        num_tails += 1

percentage_heads = num_heads / num_flips * 100
percentage_tails = num_tails / num_flips * 100

print str(num_flips) + ” iterations”
print str(num_heads) + ” heads,  ” + str(num_tails) + ” tails”
print “%.2f%% heads”  % percentage_heads + “, %.2f%% tails”  % percentage_tails

I ran through 10 tests. Here are the results:

12 iterations
6 heads,  6 tails
50.00% heads, 50.00% tails

12 iterations
7 heads,  5 tails
58.33% heads, 41.67% tails

12 iterations
5 heads,  7 tails
41.67% heads, 58.33% tails

12 iterations
4 heads,  8 tails
33.33% heads, 66.67% tails

12 iterations
7 heads,  5 tails
58.33% heads, 41.67% tails

12 iterations
4 heads,  8 tails
33.33% heads, 66.67% tails

12 iterations
9 heads,  3 tails
75.00% heads, 25.00% tails

12 iterations
7 heads,  5 tails
58.33% heads, 41.67% tails

12 iterations
8 heads,  4 tails
66.67% heads, 33.33% tails

12 iterations
4 heads,  8 tails
33.33% heads, 66.67% tails

Wow! Over half the time, we get results of 64% or more on one side or the other, purely through simulating a random coin toss. I’m not trained in statistics, but this would seem to me to show that, although the 64% success rate seems good, it’s really no better than random chance.

What happens if we increase our sample size of coin tosses to 10,000? See below:

10000 iterations
4929 heads,  5071 tails
49.29% heads, 50.71% tails

10000 iterations
4999 heads,  5001 tails
49.99% heads, 50.01% tails

10000 iterations
4991 heads,  5009 tails
49.91% heads, 50.09% tails

10000 iterations
4955 heads,  5045 tails
49.55% heads, 50.45% tails

10000 iterations
5041 heads,  4959 tails
50.41% heads, 49.59% tails

10000 iterations
4970 heads,  5030 tails
49.70% heads, 50.30% tails

10000 iterations
5037 heads,  4963 tails
50.37% heads, 49.63% tails

10000 iterations
4963 heads,  5037 tails
49.63% heads, 50.37% tails

10000 iterations
4961 heads,  5039 tails
49.61% heads, 50.39% tails

10000 iterations
5074 heads,  4926 tails
50.74% heads, 49.26% tails

As you can see, when the number of coin tosses is 10,000, not once do we get a result higher than 64%. In fact, we don’t even get a result of 51%. Maybe there is something to this large sample size thing.

I’m in no way a scientist, of course. I’m not even good at math. So, I remain humble and ready to admit mistakes if anyone finds any. But it seems to me, with my uneducated opinion, that the results of this study are no better than random chance. If I find the time and am not shown to be completely incompetent in the results above, then I’ll explore the other studies listed.

Link:

http://128.196.99.80/JJBAReprints/Allen%20Schnyer%20Hitt%20Psych%20Science%201998.pdf