Defending small data

I read with interest a recent editorial that opined on the poor evidence for screening in cancer trials. The evidence was judged poor because apparently no screening trial has demonstrated a clear reduction in all-cause mortality, only disease-specific mortality.  One example discussed in the analysis reviews the data for colon cancer screening and notes that, while there were a statistically significant lower number of deaths related to colon cancer in the screened group, the total mortality in the two groups was no different.  The authors posit that the study is either underpowered for total mortality or that the screened patients may have more deaths due to the 'downstream effects' of screening.  The provocative conclusion by many a tweet and retweet is that cancer screening has not been shown to save lives. Apparently the path to progress in medicine now must be paved by studies with millions of patients.  I understand the desire for more and more data, but I see danger in the sanctimonious protestations of those who can only find truth within the confines of a million-person, randomized control trial.  This approach ignores the history of advances in clinical medicine, most of which live far outside of the boundaries of million-strong randomized clinical trials.

The history of interventional cardiology is particularly instructive.  The first catheterization of the human heart was performed by a surgical resident named Werner Forsmann in 1929 - on himself.  He thought it was a good idea to catheterize his own left antecubital vein, pass a ureteral tube into his right atrium, and then walk to the radiology department to document his accomplishment.  Forsman was fired for his troubles.  His work was rejected by the scientific establishment of the time and he languished for two decades - at times having difficulty finding gainful employment, and ultimately giving up research- until 1956, when he was finally recognized for his felicitous discovery.  The technique did not find widespread use and was mostly the provenance of physiology labs until a German physician named Andreas Grunzig decided to make an attempt at opening chronically narrowed coronary arteries using balloons and catheters.

The first coronary angioplasty was performed by Grunzig September 16, 1977 using a home-made catheter on a 38 year old with disabling angina.  The procedure was remarkably successful.  The patient remained free of angina and had no residual stenosis on a repeat catheterization at the 10 year anniversary of the seminal event. Cynics would say that Grunzig ushered in an era that saw the rapid rise in unnecessary coronary interventions, and they would be right.  But it was techniques developed by people like Grunzig who made it possible for a Seattle cardiologist named Marcus Dewood to take patients in the throes of a heart attack and inject contrast into the coronary tree to show that heart attacks were a result of blood clots that formed in arteries that fed the heart.  Soon after, cardiologists began blowing up balloons to open up thrombosed arteries in patients having heart attacks and primary angioplasty - arguably the single biggest leap forward in the treatment of cardiovascular disease - was born.

The epidemiology of cardiovascular disease, as a result, has seen dramatic change over the last half century.  Interestingly, the rate of acute myocardial infarctions has been fairly constant over time, but cardiovascular mortality has dropped significantly.
While the decline in cardiovascular mortality is at times attributed to the success of primary prevention and medical therapy, the data for primary prevention in this arena is relatively weak. That does not mean that tobacco cessation and aspirin do not work, it's just hard to show what exactly the size of these effects are.  A study in the New England Journal tried to adjudicate the number of deaths prevented by specific risk factors using assumptions that would kindly be called shaky.  Excerpted below is the example given in the paper.

The paper assumes from a metanalysis that a 20mmHg drop in blood pressure will reduce deaths by 50%. This is not a statement that can be made broadly for a variety of reasons. The starting blood pressure matters, it would be wrong to suggest all deaths prevented were cardiovascular deaths, and it is quite possible that aggressive blood pressure control in certain populations (diabetics) resulted in poorer outcomes.  The faulty assumptions are rescued, of course, with the use of an exponential function, because Euler's number fixes everything.  This same paper also attributed a reduction in deaths to interventions (revascularization for chronic stable angina) that have never been shown to reduce mortality!

I realize it has become dangerous to use one's clinical experience to inform one's views.  While I have no quarrel with evidence, the reality is that the longer I practice, the more I realize that clinical scenarios rarely fit even the best designed clinical trials.  In light of the underwhelming evidence with regards to primary prevention and cardiovascular mortality, my thoughts on the impressive reduction in cardiovascular mortality are driven by a group of patients that I cannot forget.  These were patients who had large 'missed' myocardial infarctions.  I can still vividly remember racing to place a central line in a middle-aged Cambodian man who had arrived in the CCU more than 24 hours after the onset of chest pain.  He had suffered a proximal LAD infarct  - an occlusion in an artery that fed a very large part of his heart.  As I rapidly added medications in an attempt to support a failing heart, a cardiac marker was called from the lab - a cardiac troponin that was over 400.  Cardiac troponins are released into the blood stream as cardiac muscle dies.  The higher the troponin, the more cardiac muscle that has died.  The vast majority of troponins that I get called for are under 1.  A troponin of 400 was inconceivable to me at the time.  My patient didn't have a chance - he was dead within 12 hours.

I had other patients with extensive heart attacks that could not be revascularized - they all did poorly, driving it home to to me in a way that reading about it never could - that the time to revascularization was the single most important metric that predicted a good outcome.  So when I look at the graphs that show a marked drop off in cardiovascular mortality that accelerated in 1980, I do not believe that it was coincidence that this also marked the beginning of the era of coronary intervention for heart attacks. Again, this is not to suggest primary prevention does not matter, rather there is little that clinically has seemed to have as direct of a clinical impact on patient outcomes than revascularization to limit the size of an infarct.

I wonder how Forsmann and Grunzig would have survived in the current climate that looks down on any clinical data that has fewer than a million patients.  Grunzig demonstrated results of coronary angioplasty in his first four patients at the American Heart Association meeting in 1977.  He was interrupted by applause, and was so stunned he was unsure if he could proceed.  I suspect today he would have been rewarded by denigrating tweets from the 'elites' among us asking for more data.
These apostates worshiping at the temple of data see themselves as gladiators championing the interests of patients.  They are right and wrong.  The current messy path medicine forges ahead on involves hurting patients.  Grunzig used primitive catheters he made in his kitchen and patients in those early years had complications that sometimes required emergent bypass, and at other times, resulted in death.  But here's the thing, I don't think Grunzig did anything wrong - he simply did the best he could with the tools and the knowledge available to him.  Without Grunzig, there would have been no DeWood, and no progress to a world of interventional cardiologists on call to open up an occluded artery within 90 minutes of a heart attack.

While well intentioned, the purists that seek to use big data to protect us flirt with therapeutic nihilism.  Forcing all progress through the funnel of a million strong clinical trial is a bar too high, a bar that protects us from medicine itself.  No one benefits from that.

  

Treating chest pain with a cup of tea

It is very early.  I am running to the 'clinical decision unit' (CDU) to see a patient of mine sent in the night before from a local skilled nusing facility.  Also known as clinical observation units,  'obs' units, or short stay observation units, these units were designed to help decompress busy emergency rooms and divert unnecessary, expensive inpatient admissions.  The units are typically adjacent to emergency departments, and usually are run by emergency physicians.  My particular patient was admitted due to an episode of chest pain at her facility.  A brief conversation the prior night with the emergency room staff revealed chest pain that clinically was not typical for any of the feared diagnoses of a heart attack, pulmonary embolism or an aortic dissection.  An electrocardiogram and cardiac enzymes were also initially unremarkable.  Regardless, the patient was elderly and had multiple other comorbidities, and was somewhat confused.  I recommended a short stay to allow anything malignant to declare itself.

And so, here I was, at the observation unit, digging through pages upon pages of printed gibberish that clearly had achieved the nirvana stage of meaningful use (for those wondering, that's after stage 3).  Ironically, the most useful piece of information lay in a handwritten progress note describing the episode. I could see why the patient had been brought here to be further evaluated, but after 18 hours of negative biomarkers, electrocardiograms, and no recurrence of symptoms, I felt comfortable letting her go back to where she came from.  I told the ER staff... who cancelled her stress test.  A stress test? Yes, a stress test had been ordered prophylactically.  We practice in a climate where every bad outcome has the potential for litigation - malpractice lawyers would have a field day with the case of anyone going to the ER, being discharged without some type of cardiac imaging study, and having a heart attack.  My recommendation to discharge the patient shifts the liability of an adverse outcome from the ER squarely on to my shoulders, and thus, poof goes the stress test.

As I get ready to move to the next patient on my list, I'm stopped by one of the ER physicians. There's a 34 year old woman with chest pain in the CDU as well.  She also had a brief episode of chest pain the day prior.  She had no other medical history, her electrocardiogram was benign, multiple biomarkers were negative.  Her symptoms had resolved.  She was scheduled to have a CT scan of her coronaries done.  Could I take a look?

6 million patients arrive at emergency departments complaining of chest pain.  About seventeen percent of them have a real coronary event.  Discovering which one of these patients actually has a real coronary event is one of the great clinical challenges that faces physicians.  Traditionally, doctors have used the clinical history, an electrocardiogram, and cardiac biomarkers to make a decision about hospitalization and further cardiac testing.  The teaching when I was in training was to categorize patients: low risk patients needed reassurance, intermediate risk patients needed some type of a stress test, and high risk patients needed a cardiac catheterization.  The hard part, of course, was drilling down to the patient in front of you in the emergency department and getting that label right. The data with this approach was  not perfect.  The chances of missing a heart attack were 1 in a 1000 (0.1%).  Review of these cases by an expert, experienced cardiologist in the majority of these missed diagnoses revealed no errors; the patients just had very subtle findings that the traditional approach was unable to resolve.

Two developments coincided over the last ten years that started to change the landscape of the approach to chest pain patients.  The first development was the push by payers to reduce cost - for chest pain patients that meant avoiding admissions to the hospital.  Payers started to refuse payments for those patients deemed too low risk for admission to the hospital.  Spooked by this, observation units, and protocols to increase patient throughput through these units flourished.  It was within this Wild West-like atmosphere that John Wayne, in the form of the coronary computed tomography angiography (CCTA), arrived.  CCTA allowed high resolution images of the coronary tree within minutes, and promised an easy straight talkin' yes/no answer with regards to coronary plaque. A negative test effectively ruled out a coronary event, and as a side benefit, also could provide information about clots in the lung and abnormalities of the aorta.  One test that took minutes could obviate the need for a hospital admission, and shorten the amount of time spent in the ER because you didn't need to wait for the old cardiac stress test that was only done during daytime hours.  It also seemed to satisfy everyone's appetite for zero risk -  the promise was that a negative test meant you were absolutely not going to have a heart attack.  The clinical history in this brave new world was irrelevant.  After all, the traditional approach that focused on talking to patients missed one out of a 1000 patients.  Now we could be perfect, and it would save money!  The time for irrational exuberance was upon us.

Not surprisingly, coronary CTA's proliferated in ERs.  I noticed that patients who ordinarily would get admitted to the hospital, or would merit a cardiology consultation were instead being sent home from the ER after a negative study.  The calls from the ER didn't stop though; they changed.   Now the calls were for positive coronary CTAs, which some times forced additional testing.  After all, the if the CT scan suggested there was something wrong with your heart...it was exceedingly difficult to reassure the patient or myself with only a lowly stethoscope that everything was actually ok.

Clearly,there was no free lunch when it came to CCTA. There was improved efficiency in the ED, but there was an increase in downstream testing and radiation exposure with no decrease in the overall costs of care.  Even more troubling, I started noticing patients that I ordinarily would have suggested no testing were also being put in the imaging pathway.  The question no longer was if you needed a test, but what test you should get.

In a study (Foy et. al.) comparing no testing with multiple other cardiac imaging modalities that included stress tests and CCTA, low risk patients who received no further cardiac testing had no difference in outcomes.  Unfortunately, that wasn't even the worst part.  The unfortunate low risk chest pain cohort that did get additional cardiac testing ended up with additional, invasive testing.  It was estimated that for every 27 patients who underwent a myocardial perfusion imaging stress test, 1 patient got an unnecessary cardiac catheterization.  For every seventy-one patients who underwent a CCTA, 1 patient would require an unnecessary cardiac catheterization.  Since there was no difference in outcomes between the two groups, the additional invasive testing was all risk with no benefit.



As the figure above demonstrates, an approach that involved CCTA for all low risk patients with chest pain would result annually in 50,000 unnecessary cardiac catheterizations.  Cardiac catheterizations have become incredibly safe, but do involve snaking a catheter into the small arteries that come off of the aorta and feed the heart. Do this enough, and bad things are bound to happen.  In contemporary practice the mortality associated with cardiac catheterization is 1 in a 1000.  A rare event, to be sure, but that would mean an estimate of 50 healthy people dying annually due to having an unnecessary procedure.  To top it off with a giant maraschino cherry, there is nothing to suggest a CT scan-for-all approach would even reduce the rate of missed heart attacks.  Recall that Foy et. al., showed no difference in hospitalization for MI regardless of whether low risk patients got some type of cardiac stress/imaging test or not.

While Foy's paper is a retrospective review that is far from definitive, it does fit in with what I see in clinical practice.  The well-intentioned desire to cut costs by avoiding inpatient admissions, and improve safety by reducing missed coronary events has not lowered costs, and has not made patients any safer.  There certainly are trials that will point to the impressive negative predictive value of coronary CTAs, and the impressive close-to-zero event rate with these tests.  These studies are hopelessly flawed.  Many of the patients in these trials should have warranted no further testing, and would have been better served with my mother's masala chai - a much better tasting and soothing alternative.

I did take a look at the young woman with chest pain.  I could not bring myself to order a CCTA or a stress test.  I did recommend a nice cup of masala chai.  She remains alive and well to this day.

The problem with choosing Medicare Advantage plans: the abbreviated version

Mrs. Cassidy slowly walks into my office one busy afternoon.  Mrs. Cassidy has some serious style.  She is wearing a deep orange dress with a bright blue blazer.  There aren't too many folks that can pull that outfit off, but she can. She has a wide slow smile, and she speaks with a slow southern drawl that belies her southern roots. This was supposed to be a routine follow up visit for a 67 year old woman with a history of a mechanical mitral valve replacement and coronary disease.  Unfortunately, she tells me a story that is concerning for angina.  I think she needs a stress test. I quickly look at her insurance, and I let out a somewhat audible groan.  She has a Medicare Advantage (MA) plan.  I explain to Mrs. Cassidy that we will need to go through an extra step to pre-certify her stress test. She expresses surprise and asks me what she should do.  I will tell you what I told her, but first, let me tell you why.

Medicare Advantage Plans: A brief history

Medicare Advantage plans represent the federal government's experiment with delivering care to seniors through private insurance companies.  The experiment started as an alternative to the pay-for-whatever-the-doctor-orders model of traditional medicare.  Traditional medicare is fee-for-service, which means there are payments made for any and all services delivered.  If I think a patient needs a stress test, the patient just goes and gets the test..no questions asked.  This can lead to some perverse incentives and results in a lot of services being delivered.  Like the parent that abdicates the responsibility of disciplining their child to a reform school, the federal government hoped private insurance companies would mete out the tough love needed to reign in costs.

Tough love meant I was going to have to get 'permission' from the insurance company to do a stress test on Mrs. Cassidy.  If my staff was unable to get the test authorized with the information I gave them, I would have to get on the phone with an insurance company physician to plead my case.  This seems like a win for the federal government from a cost standpoint.  Surprisingly, it's not.

The federal government pays private insurance companies a fixed amount for each enrollee in their plan.  In the beginning (prior to 1997), the federal government would pay 95% of what they would have paid had a patient enrolled in traditional Medicare.  This saved the government money, but resulted in MA plans making up only 13% of the total Medicare market.  In an effort to expand access and have more people enrolled in MA plans, the federal government increased reimbursement to MA plans in 2003 in a bill best known for introducing Medicare prescription drug coverage.

Enrollees in MA plans now cost the taxpayer $1000 more than if the patient had stayed in traditional Medicare, and enrollment in MA plans has risen rapidly to cover almost 30% of all Medicare eligible seniors.

The government has been active in trying to rectify the payment asymmetry by tying reimbursement to risk scores and value.  This has not proven to be without its problems.

Scoring risk: You are higher risk if they say you are

In 2004, the Medicare program began to tie payments to private plans to beneficiary risk score.  Risk scores are based on diagnoses coded during the year prior to the payment year.  Insurance companies responded by investing resources in ensuring 'appropriate' coding.  The patients didn't change.  But if the patient enrolled in a MA plan, they became higher risk.  The evidence clearly showed insurers were upcoding to raise payments.

Measuring value: Harder than it sounds.

Payments were also tied to value -based payments to reward MA plans that provided high quality care. Quality in the MA program is graded on a 5 star scale that is determined from a weighted scale comprised of variables that include adherence to best practice processes (adults should get flu shots) and outcomes.   An example of an outcome measure is blood pressure targets.  In 2010, the MEDPAC report on quality used blood pressures < 130/80 in diabetics as an outcome measure.  Unfortunately, 2010 was also the same year that the ACCORD trial examining aggressive blood pressure targets in diabetics was released: Targeting a systolic blood pressure target of < 120mmHg compared to < 140mmHg did not reduce the composite endpoint of fatal/non fatal cardiovascular events.  The world of hypertension was so spooked by this trial that the panel convened to create a national guideline using this trial as evidence that lower was not always better when it came to blood pressure.  So it is entirely possible, that in 2010, 5 star plans were those plans that did harm to diabetic patients by having lower blood pressure targets.

In brief, the federal government's experiment in risk-adjusted, value-based payments to private insurance companies has only served to shift health care dollars to insurance companies at increased cost to the taxpayer.

Between January 1st and February 14th each year, if you happen to be enrolled in a Medicare Advantage plan, you can leave your plan and return to original Medicare.  John F Kennedy famously said, "Ask not what your country can do for you, ask what you can do for your country".  Mrs. Cassidy, do what's right for your country.  Choose traditional Medicare.