Continuitis, Dichotomania and the Tetrachoric Coefficient of Correlation

Guernsey McPearson

Karl Pearson suffered from galloping continuitis. To him, the very idea of a dichotomy was offensive. Show him a binary characteristic and he would imagine it continuous. Show him a two by two table and he would imagine it bivariately continuous. That being so, what the data were in need of was a correlation coefficient and he was the man to provide it... and a name for it. Pearson's tetrachoric coefficient of correlation is based on the idea that binary outcomes are really continuous ones that have somehow been binaried. (They met a medical advisor on a dark night most likely. More of this anon.) How did this work? Well this column is not famed for detailed exposition, the broad themes, the great controversies, the deep philosophical issues, these are our subject matter but since you asked...

What you didn't ask? The cheek of it! Don't try to pretend to me that you know all about tetrachoric coefficients of correlation. I've interviewed a good many of the younger generation and I know what they know about: bootstraps, BUGS and anything else that involve throwing random numbers at problems, but tetrachoric coefficients of correlation ? I don't think so. So sit up and pay attention.

We need to estimate a bivariate Normal distribution. Without loss of generality we can let the two means be zero and the two standard deviations be one. We can then use each of the two margins of the table to estimate the two cut-points. For example we might have the following table describing readers of this article

 

 

 

Has Previously Heard of Karl Pearson

 

 

 

No

Yes

Total

Has previously heard of the tetrachoric correlation coefficient

No

73

17

90

Yes

2

8

10

 

Total

75

25

100

 

 

Now you may think that this is a pretty insulting table. Do I think that only 25%of the readers of SPIN have heard of Karl Pearson or do I perhaps think that a particularly cretinous subset read GMcP? I am using tight criteria here, however. To have 'heard' of Karl Pearson I require the reader to know a) that there are two famous statisticians called Pearson b) that Karl Pearson is Egon Pearson's father c) That only one of them is involved in the Neyman-Pearson lemma and it is not Karl d) To have heard of the Neyman-Pearson lemma e) To have heard of the Pearson product moment correlation and f) to know that the Pearson involved in this is not Egon. I leave aside, through pure generosity of spirit, the small matter of the Pearson system of frequency curves not to mention Pearson and Hartley's Biometrika tables let alone Clopper-Pearson intervals. And don't get me started on the Kendalls. Now quite honestly when I look at it like this, I would be hard put to name 25 members of PSI who would fit the bill: Professor Ivor Gripe, to be sure, but does he have time to read SPIN these days? However, notice also the cunning of my example. You begin to see, do you not, that having 'heard' of Karl Pearson is a criterion that admits of a subtle continuum rather than a mere vulgar dichotomy. But I digress.

So 25% have heard of KP. This means, as anybody possessed of the Hartley and Pearson tables or a personal computer, can tell, that our cut-point for this margin is, 0.674, 75% of the standard Normal distribution being below this point and 25% above it. Similarly the cut-point for the other margin is 1.282, as we all know from all those power calculations we do involving 90%. Now the fun starts. Given these marginal values, the 2 x 2 table is uniquely characterised by the number in any one of the four cells, as anyone who has ever calculated a Pearson chi-square knows. (Which Pearson is "chi-square Pearson" is point g, which inexplicably got missed from the list above.) There are an infinity of bivariate standard Normal distributions with these marginal probabilities. However, only one of them yields 8% in the southeast corner and that is the one with correlation coefficient 0.74. (I leave it to the reader as an exercise to find the root of the equation involving the double integration of the bivariate Normal to confirm that this is so. Alternatively you can use proc freq of SAS and the plcorr option.)

So much for continuitis which, in any case, is not what this article is about. It is about dichotomania. Dichotomania is an obsessive compulsive disorder to which medical advisors in particular are prone. This is the opposite of continuitis, or Pearson's syndrome as it is sometimes known. Show a medical advisor some continuous measurements and he or she immediately wonders. "Hmm, how can I make these clinically meaningful? Where can I cut them in two? What ludicrous side conditions can I impose on this?" Well, the last bit is made up by me, but it must be happening in the dichotomaniac's id if not in the ego.

I had such a ghastly encounter only the other day. I was assigned to work on the latest hypertension project. I can tell you that my own blood pressure was done no favours by learning that I was to work with that compulsive categoriser Dr Angina Cutter. Sure enough we were into the definition of responders in no time at all.

"So, Guernsey" she said trilling in what she no doubt imagined was a delightful manner, "the definition of a responder is any patient who either reduces their diastolic blood pressure from a value of 95 or more at baseline to one of less than 90 or who has a 10% reduction from baseline". "Or both, presumably", I said. "Yes, of course". "So let me get this straight", I added, "If the blood pressure reduces from 95 to 89, the patient is a responder but if it reduces from 101 to 91 the patient is not." "Well yes, don't you see, 91 is too high." "But", I persisted, "if it reduces from 105 to 94 then the patient is a responder." "Well, of course," she replied, "that's a very meaningful reduction".

But at Pannostrum Pharmaceuticals they don't call me the ace inhibitor for nothing. I was soon back with a couple of diagrams. Figure 1, below shows the "response region " for this measure.

 

 

 

 

 

 

 

 

 

 

 

"What a delightful step feature on hypertension hill," I said, "are you sure that you would not like a couple more, say one a little higher up and one lower down." "Are you being sarcastic?," she replied. "The criterion I propose has been used in the scientific literature and is well accepted". "Well not by me, "I added. "Let us suppose for argument's sake that the mean DBP is 100 at baseline in the treatment group, the standard deviation is 10, the treatment effect is to reduce DBP by 10 and the correlation coefficient between baseline and outcome is 0.7." (I meant, of course, the Pearson product moment correlation coefficient, not the tetrachoric one.) "Then this is what the probability of response looks like as a function of the baseline value. Note the delightful ski-jump feature on pressure piste."

 

 

 

"Now I know that you are being sarcastic," she replied, "however, I know for a fact that this criterion has been used not just in the medical literature but also in the statistical literature."

Game, set and match to Dr Cutter I am afraid. To say that she had me completely stumped does not do justice to the situation. My jaw nearly hit the floor. I know that amazement is a continuous emotion admitting, of course, of subtle gradation but I was definitely dichotomised as "staggered". I expect such nonsense in The Speculum and the Albion Physician's Enquirer but in statistical journals? Surely not. But so it turned out to be. Papers in The Journal of Theoretical Applied Statistics and Very Fashionable Clinical Statistics turned out to have analysed some trials in which this criterion had been used. The statistician authors made no comment at all on the suitability of these measures but got into the real meat of the matter: how to produced efficient analyses of binary data when auxiliary information on compliance was available using nearly nested structural almost missing at random mean response models.

So, to return to Dr Cutter. "How," I said gloomily, "do you propose that I should analyse this response measure". "Oh," she said, "I wouldn't presume to advise you on that. I do think that it is important that everybody sticks to their area of expertise in drug development, don't you? You know, statisticians with analysis and physicians with measurement." "On the other hand, if it's pointers you are looking for, I was talking the other day to a friend of mine who's a sociologist. He was recommending a particular approach to binary data that's very popular in that research community." "Not, numbers needed to treat?," I said.

Dr Cutter paused. "No," she said, wrinkling her pretty little nose in a rather charming manner, clearly making an effort to remember something very difficult. "It was some sort of funny correlation coefficient. Tetra something, I think."

Back to Guernsey McPearson's Prose Selection

Back to Guernsey McPearson's Homepage

Go to Stephen Senn Homepage