Thursday, December 3, 2015

My response to Dr. James

What follows is an email correspondence that nicely represents the different positions Dr. James and I take on the issue.


Dear Dr. Koka,

I really appreciate the respectful way you disagreed with me, and I agree with you that it is probably time to let the issues rest in the realm of respectful disagreement. I never expected to make anyone comfortable with my analysis, least of all physicians and hospitals. Harm during hospital care is an uncomfortable topic. One comment I received on the paper from a physician was something like "How good do you expect us to be? Maybe 1% is the best we can do." I don't know the answer to that.  I think physicians and patients working together can substantially reduce the preventable harm.

What I would say to your physician colleagues that do not like my assumptions - then please go ahead and survey the data and draw your own conclusions, and then publish them in a peer-reviewed journal. Better yet, do a comprehensive study of preventable harm in hospitals that has the scope and depth to at last get irrefutable data on preventable harm in U.S. hospitals. Then we can track improvements as those unfold.

I wish you well in your efforts to heal those with heart troubles. I think there can be nothing better in life than healing another human being.

Best Regards,

john james


-----Original Message-----
From: Anish Koka 
Sent: Nov 30, 2015 7:07 PM
To: John James dwbates@bics.bwh.harvard.edu
Subject: Re: Journal of Patient Safety article

Dear Dr. James,

Thank you again for your response. I do appreciate your time.  I don't think I'm trying to impose my beliefs on the data.  I guess you can say I am trying to strictly interpret the data, but that is what evidence based medicine is. I am fairly certain you are comfortable with your assumptions, which is why you made them in the first place.  I have yet to find a physician who takes care of patients who is comfortable with the assumptions you have made.  

My comments below, but  I think we will have to agree to disagree.  

Again, I do appreciate your time, and I do applaud your efforts to shed light on this very important issue

Respectfully,

Anish Koka


On Mon, Nov 30, 2015 at 6:13 PM, John James <john.t.james@earthlink.net> wrote:
Dear Dr. Koka,

My comments below on your thoughts. I think you are trying too hard to demand precision and impose your beliefs upon the data available. No one has performed a single study that would represent the nation as a whole. You seem to be stuck on the Landrigan study as somehow representative and it is not. There is certainly more than one way to handle the available data, but in my paper I made it clear which choices I made and why. I might point out that the number 178,000 you suggest below would make preventable adverse events the third leading cause of death.

Best Regards,

john james

-----Original Message-----
From: Anish Koka 
Sent: Nov 30, 2015 1:18 PM
To: John James dwbates@bics.bwh.harvard.edu
Subject: Re: Journal of Patient Safety article

Dear Dr. James,

Thank you for the response. I sent a letter to the editor as I was hoping to have a peer review your approach as well.
Your response does clarify one matter I was confused with, but does reaffirm my challenge to your estimate.  The clarification that I didn't grasp in my initial email was that you assumed the total preventable harm rate to be equivalent to the lethal event harm rate. I do not feel this is a reasonable assumption.  While I understand that science and evidence based medicine must necessarily have some assumptions contained within them, I do feel your assumptions are a leap too far, especially for a paper that is intended to provide an 'evidence based' estimate of patient deaths due to medical error. 

My response to you with regards to the major issues are as follows:

1. The preventable harm rate is 69%

Only one paper has a preventable death rate noted (Landrigan).  The OIG study does not list preventable deaths, just a total preventable harm rate.
You have to assume that the Classen paper would have shown a similar rate of preventable harms.  You then have to assume that the preventable death rate is equivalent to the total preventable harm rate.  The classen authors write that all harms may in some way be preventable?  Since they have no way of telling they chose not to say so in their paper.  I don't think its fair for you to take that statement as a blessing to use all their harms as preventable.  Your attempt was to create an evidence based approach for harm data (this I applaud), but you are creating evidence in this case.  You excluded a number of papers because they didn't you the GTT, you could have excluded the Classen paper in coming up with a preventable harm rate. The actual evidence for preventable harms is below.  There are no assumptions in the numbers below.  That is what the evidence says.  If your interpretation of classen et al.,  were to hold, why not say all the harms were in fact preventable?  The OIG and landrigan paper specifically don't believe all adverse events that happen in hospitals are preventable.  Regardless of your belief or my belief, that is what the evidence says.  Again, the Classen folks could have said in their conclusion that they felt all harms were preventable.  They did not, and you shouldn't make that assumption for them.

Landrigan paper:
588 total harms,  364 preventable (63%)
OIG Medicare analysis
128 total harms, ~56 preventable (44%)

(364+56)/(128+588) = 58%

Assuming that.. (34.4 million x .58. x .0089) = 178,310.  Of course this assumes the preventable harm rate is equivalent to the preventable death rate.

PREVENTABILITY IS SUBJECTIVE AND AS I SAID, I COULD NOT IGNORE THE  OPINION OF THE WRITERS OF THE CLASSEN PAPER. CLEARLY, THEY BELIEVED THAT THE VAST MAJORITY OF ADVERSE EVENTS WERE PREVENTABLE. AS I SAID IN MY ORIGINAL RESPONSE, EVEN IF ONE TOOK THE MIDDLE VALUE OF 44, 63, AND >>63, THE RESULT IS HARDLY DIFFERENT THAN MY ORIGINAL ESTIMATE. LET ME EMPHASIZE THAT THIS IS AN ESTIMATE. YOU ARE ALSO TRYING TO TREAT EACH PAPER AS AN INDIVIDUAL SOURCE AND THIS IS NOT JUSTIFIED. ONE MUST LOOK AT THE AGGREGATE OF THE DATA.

It remains a fact that the Classen paper does not comment on preventable harms.  I also can't seem to find your quote in the article.  I searched the text for the same, with no results.  I assume this was personal communication?  I know you disagree but I certainly would not assume 100%.  I agree that this does not have a sig. effect on the number.  But of course, I don't think much of that number since I take issue with assuming total preventable harm rate is the same as preventable death.
AT THE RISK OF MORE CONTENTION, I SHOULD POINT OUT THAT I COULD HAVE WEIGHTED THE THREE LARGE PAPERS EQUALLY INSTEAD OF HEAVILY WEIGHTING THE LANDRIGAN PAPER. THAT WOULD BE OIG (2010), CLASSEN AND LANDRIGAN. IF I HAD DONE THAT, THE AVERAGE PERCENTAGE OF ADVERSE EVENTS WOULD HAVE BEEN 1.4 + 1.1 + 0.6 = 1.03%, WHICH IS A GOOD BIT HIGHER THAN THE WEIGHTED AVERAGE I USED OF 0.89%.


If you weighted the 3 papers equally that would be completely wrong, since the landrigan paper is 2-3 x larger than the other papers.
 
2. The lethal preventable harm rate is equivalent to the total preventable harm rate

In your article you assume the lethal preventable harm rate is equivalent to the total preventable harm rate.  You say in your response that you think the lethal preventable event rate is higher than the total preventable harm rate.  What basis do you have to make that assumption? The landrigan paper is the only paper that allows a comparison of this, and only in this one case does this happen to be about the same.  As a busy clinician in the hospital, I feel that of all the people dying in hospitals, much less than 63% are preventable.  Again, that doesn't give me the right to write a paper about it.  I'm just one guy, and I could be completely wrong.  But you assume this based on your feeling, and have written a evidence based paper on this.

THE LANDRIGAN PAPER DOES STATE THAT THE OVERALL PREVENTABILITY IS 63%, WHICH IS WHAT I USED. FROM THEIR TABLE ONE CAN CALCULATE THAT THE PREVENTABILITY RATE IS 9/14 = 64% FOR LETHAL EVENTS.
You, elsewhere, have taken issue using one paper as being globally representative.  However, in this case, you have no issues with using the one papers lethal preventable event rate and extrapolating to the other 3, and of course to the 34 million folks.

PLEASE, WE ARE NOT TALKING ABOUT ALL THE PEOPLE DYING IN A HOSPITAL. WE ARE TALKING ABOUT DEATHS IDENTIFIED USING THE GLOBAL TRIGGER TOOL AS APPLIED TO WHATEVER INFORMATION MAY OR MAY NOT BE CONTAINED IN THE MEDICAL RECORD. OBVIOUSLY MANY PEOPLE PEOPLE DIE IN HOSPITALS IN WAYS THAT WOULD NEVER BE FLAGGED BY THE GTT. I ALSO MADE IT CLEAR THAT MANY DO NOT ACTUALLY DIE IN HOSPITALS. THEIR DEATHS ARE HASTENED BY FAILURE TO USE EVIDENCE BASED MEDICINE. THE MURTHA EXAMPLE ATTEMPTED TO MAKE THAT CLEAR TO YOU. 

I agree completely with this comment.  But then how do we know what the number is?  This smells of something I was taught in computer science class all the time -- Garbage inputs lead to garbage outputs. 
I ASSUME YOU ARE WELL AWARE THAT CARDIOLOGISTS OFTEN DO NOT FOLLOW EVIDENCE-BASED MEDICINE IN THEIR PRACTICES. I HAVE MESSAGES FROM CARDIOLOGISTS FRUSTRATED BECAUSE THEIR COLLEAGUES IGNORE THE NEED FOR CAREFUL ELECTROLYTE MANAGEMENT IN THE FACE OF HEART ARRHYTHMIA. I READ IN THE LITERATURE THAT FAR TOO MANY CARDIAC CATHS ARE PERFORMED - AGAINST THE EVIDENCE. ONE STUDY I RECALL SHOWED THAT IF TROPONIN IS USED AS AN INDICATOR OF TISSUE DAMAGE, THEN A LARGE PERCENTAGE OF CARDIAC CATHS CAUSE SOME SUBCLINICAL TISSUE DAMAGE. THERE IS ONE NOTORIOUS EXAMPLE OF HUNDREDS OF CARDIAC BYPASS SURGERIES DONE ON PATIENTS THAT NEVER NEEDED THEM. MANY DIED.

Again, while all of this does happen, one needs to know how often this happens. What are the numbers?  How often are lethal arrhythmias happening because of inadequate electrolyte repletion , how often are inappropriate bypass surgeries being done? How often do people die?  You assume this happens commonly..again I don't know for sure, it doesn't feel like any of this is happening on a daily basis...but I would not deign to write an article based on that feeling.  With no intention to agitate/upset you I would suggest your underlying belief of the frequency of these events is shading the assumptions you make.
I THINK YOU NEED TO VIEW THIS AT A TOP LEVEL. WHAT I ASSERT IS THAT SOMEWHAT MORE THAN 1% OF THE TIME WHEN A PERSON IS HOSPITALIZED, THEIR LIFE IS SIGNIFICANTLY SHORTENED BY INAPPROPRIATE (NON-EVIDENCE BASED) CARE IN HOSPITALS. THAT DOES NOT MEAN THAT THEY DIE IN THAT HOSPITAL. PLEASE ALSO NOTE THAT MY ESTIMATE INCLUDES HOSPITAL ACQUIRED INFECTIONS, WHICH THE CDC ESTIMATED KILL 100,000 PERSONS EACH YEAR IN 2007.
Why do you think that?  A better way to think about this is that out of a 1000 patients that are presenting to hospitals in need of help, 4 are dying due to a preventable error (Landrigan).  Using your numbers (0.58 x .0089), 5 out of a 1000 patients presenting in need of help are dying due to a preventable error.  And while much can be done to prevent, for example, infections in the hospital, there are some infections that will not be preventable.  Some patients who get central lines, despite appropriate, evidence based care, will have infections.  Some patients who get put on antibiotics to save their lives from sepsis, will get Clostridium difficile, and some may die from this.  

3. The evidence for lethal preventable deaths.

You note that you can't just use one paper to make an estimate.  Why not? IT IS NOT REPRESENTATIVE OF THE NATION AS A WHOLE.  If only one paper has the answers you seek, and the other papers are not appropriate, you use whatever you can.  Regardless of whether you use one paper or four papers, you are talking about extrapolating data from ~3000 patients to 34 million patients.  It would be better to use only papers that allow you to make reasonable assumptions.  Again I would have used the Landrigan paper to be pure.  If you wanted to assume that the lethal harm rate was equivalent in the OIG analysis, I guess that may be done, but you're starting to walk on planks not supported by evidence.
Simply put, If you want to know what the lethal event rate in a population is, multiply that by the population.  

In this case, the most evidence based number is to take the landrigan papers lethal event rate (9/2341) .384% x 34,400,000 (medicare admissions 2010). That equals ~ 130,000.  

AS I HAVE SAID, ONE MUST LOOK AT ALL THE STUDIES AVAILABLE, AND THE LANDRIGAN PAPER, AS I POINTED OUT IN MY JPS PAPER, IS LIKELY TO BE A LOW ESTIMATE OF THE NATION AS A WHOLE.

I understand that one paper is a poor representation.  Obviously my argument is that pooling together, and aggregating/extrapolating inappropriately isn't a good representation either.  By the way, while you state that the north carolina folks may be better than the nation part of the conclusion of the landrigan paper is that no significant improvement has been seen in harm rates over the years (2002-2007)
4. The factor of 2.

There is little question that this approach does not capture errors not captured in the medical record, errors happening after leaving the hospital, or errors of misdiagnosis, nondiagnosis.  But again, the question, and your charge was to come up with an evidence based estimate.  Where is the evidence for your factor of 2, 3, or 4? WEISMANN IS THE BEST THERE IS. Perhaps I feel its 1, or 1.2, or 1.3 or 10?  who knows?? Again, I surmise from your response that you think the factor should be even higher.  I will submit readily to not knowing what that number is.  It doesn't 'feel' like medical errors are the 3rd leading cause of death, but again I wouldn't write a paper based on that feeling.  MEDICAL ERRORS ARE OFTEN A CONTRIBUTING FACTOR IN PATIENT DEATH; OF COURSE, A MEDICAL ERROR ALONE IS UNLIKELY TO KILL A HOSPITALIZED PATIENT - THEY ARE IN THE HOSPITAL BECAUSE THEY DO HAVE AN ILLNESS IN MOST CASES. 
In essence, you have used a feeling to come up with that number.  THAT IS NOT TRUE. THE EVIDENCE IS LIMITED, BUT THERE IS SOME EVIDENCE THAT MEDICAL RECORDS ARE MISSING INFORMATION THAT WOULD ALLOW A REVIEWING PHYSICIAN TO SEE THAT AN ADVERSE EVENT HAD OCCURRED. YOU PRESUME THERE IS NO EVIDENCE AND THAT IS NOT TRUE. This has a place in an op-ed, but not in a paper that should require some type of evidence to back up this claim.

IN MY PAPER I POINT OUT THE BASIS FOR THE ESTIMATE THAT THE GTT MISSES AT LEAST HALF OF THE ACTUAL LETHAL, PREVENTABLE, ADVERSE EVENTS. ONE CANNOT IGNORE THE LIMITATIONS OF THE SEARCH TOOL AND THE LACK OF VERACITY OF THE MEDICAL RECORDS. THE WEISMANN PAPER SUPPORTS MY CHOICE. YOU MAY HAVE CHOSEN TO IGNORE THIS FACTOR, BUT I COULD NOT. I KNOW THAT ERRORS OF OMISSION, CONTEXT AND DIAGNOSIS ARE LARGE AND NOT TYPICALLY DETECTED BY THE GTT, AND I KNOW THAT MEDICAL RECORDS ARE OFTEN NOT A TRUE REPRESENTATION OF WHAT HAPPENED WHEN THERE IS AN ADVERSE EVENT. PLEASE READ THE WEISMANN AND DUNLAY PAPERS. 
I don't see an evidence based estimate for the factor of 2.  I read the Weisman paper.  This was a paper that used post discharge interviews of patients to find events not in the medical record.  So necessarily, these were all patients that survived.  I still don't see how you come up with your factor based on this.  Again, I could say its 10, and you wouldn't (by your logic) say I was wrong.

As, an aside... I reviewed the serious adverse events deemed preventable in this paper.  Let me just give you the first one.. DVT develops after open heart surgery.  If the patient was receiving appropriate DVT prophylaxis after their open heart surgery, was it really preventable? I guess if the open heart surgery was unindicated? But who knows??  After I read this paper, I emailed Dr. landrigan to ask him if the clinical summaries on the 9 deaths in his study were available somewhere.  He has not responded :/

Respectfully,

Anish Koka

No comments:

Post a Comment