What can academia learn from the actuarial profession about promoting critical thinking? What other mechanisms are available to improve critical thinking in academia?
As I become more exposed to academia in general and health services research in particular, I’m becoming more concerned about the apparent prevalence of groupthink which short circuits critical thinking. What can academia learn from the actuarial profession about promoting critical thinking and professional standards? What other mechanisms are available to improve critical thinking in academia?
Health services research (HSR) is the scholarly investigation of health care access, quality, costs, and outcomes. We ask, “What works? For whom? At what cost? Under what circumstances?” I would say that AcademyHealth is the flagship membership organization for HSR, although there are many other organizations that focus on specific populations or specific parts of the health care system. I consider AcademyHealth my home organization as a health services researcher because I am a generalist; I’m more interested in the overall functioning and financing of the whole system and less interested in specific populations or specific types of medical care.
If we are all thinking alike and failing to engage with all viewpoints and all relevant evidence, we are not making good use of our training as researchers and scholars, and we may miss ideas that could improve health and health care.
Generally, it is my impression that most health services research published in the peer-reviewed scholarly literature is of good quality, asking relevant questions, employing appropriate methods, and drawing conclusions that are supported by the research. However, I have noticed a lack of critical thinking outside the realm of peer reviewed articles. I recently attended a conference hosted by an organization I was not very familiar with. I did not expect it to be as scholarly as the AcademyHealth Research Meeting, but what I found was appalling. Despite a veneer of being somewhat academic, with several sessions focused on data and many of the speakers being M.D.s or Ph.D.s, I was shocked by the lack of critical thinking. Speakers who talked about social determinants of health primarily focused on topics related to access and issues that impede access to care such as transportation concerns. No one even hinted that research needs to be done to understand the impact of such services on patient health outcomes, let alone mentioned any results of such research. And little to no attention was given to cost benefit analysis. I was given the impression that any discussion of cost to the system was to be discouraged if not flat out ignored. Some speakers focused on “we have partnerships with hospitals” without providing any details about what that means or how it has been shown to impact patient health. It’s as if “we are collaborating” is the measure of success. One speaker’s final slide, in part, said, “We can make healthcare more affordable by making it better.” My initial impression was that this sounded like something a twelve-year-old would say; it sounded wishy-washy and unserious. What is “better” and what evidence supports the claim that making healthcare “better” can make it more affordable? If anything, attempts to improve healthcare in some way tend to make care more expensive, not more affordable. Then I took a deep breath and said to myself that she’s probably going to answer all these questions in what she says about the slide. She did not. She never said anything about what she meant by “better” and she never talked about affordability.
I began to wonder why this group exhibited such a startling lack of critical thinking; in particular, groupthink seemed to be the norm, everyone seemed to look at things the same way. During the course of the conference and networking events, I noticed that many attendees seemed to know each other well. I suspect that they have worked in this part of healthcare and attended this conference for many years. Perhaps peer pressure or the desire for social acceptance plays a role. However, it is one thing to work towards collaboration in the workplace but it should never trump dialogue focused on scientific merit. Without the latter the social climate within many non-profits and bureaucracies may end up reinforcing poor scholarship. Perhaps public funding of the organizations represented at the conference creates a lack of accountability for effectiveness; if all that is required to keep funders happy is to show you are collaborating or you are connecting patients to services, I suppose it could be expected that such superficial process measures are all that you focus on. However, such an atmosphere can only serve to corrode academia especially if it implies that Ph.D.s do not take their degree and their training seriously. Of course, this one conference may be an outlier; it may not be indicative of the state of critical thinking and scholarship in HSR more broadly. But this is not the only time I’ve observed that everyone seems to think alike.
I have noticed over the past few years that it is not unusual for blog posts and other non-academic communications in HSR to make assumptions about how health care is best delivered without appearing to subject them to much scrutiny. In particular, they tend to take the view that more and more health care should be provided by or controlled by the government without considering other views or evaluating evidence to the contrary. For instance, recent AcademyHealth blog posts (click here and here) seem to assume that we are necessarily better off with more government spending related to health and that all “public health advocates” and “those who care about health” agree. While the author does mention the importance of using evidence in decision making, he also glosses over the lack of evidence supporting certain favored talking points. Social determinants of health is a good example. Where is the evidence taking us from a growing “awareness of the importance of social determinants” to the conclusion that initiatives to address such determinants “need to be reinforced and better integrated with health systems?” Don’t we want to know whether and under what conditions various initiatives improve health for certain populations? Shouldn’t we examine an initiative’s resulting health improvements in light of their financial and other costs? How else can we ensure we are being good stewards of taxpayer money? If we are all thinking alike and failing to engage with all viewpoints and all relevant evidence, we are not making good use of our training as researchers and scholars, and we may miss ideas that could improve health and health care.
If a presenter at an actuarial conference made assertions that they had no evidence to support, I would expect actuaries to question them on the spot. I believe this culture arises because our responsibility to the public and the rigors of our training are top of mind for us.
Perhaps I view how Ph.D.s should approach scholarship differently than other scholars. I have been an actuary for many years. Being an actuary is a profession. Not only must we get through a rigorous and challenging exam process to become credentialed actuaries, but we must also maintain professional standards by complying with numerous actuarial standards of practice and the code of professional conduct. We must complete continuing education requirements annually, and we have a disciplinary body that adjudicates cases of alleged failure to adhere to professional standards. In addition, a big part of the actuarial profession and how actuaries view themselves revolves around our responsibility to the public. Insurance company annual statements must contain actuarial certifications regarding reserve adequacy to safeguard against insolvency, for instance. Perhaps my long tenure in a profession influences my view of Ph.D. scholarship. Health services research is probably not a profession and AcademyHealth members hold Ph.D.s in various disciplines, so what works for the actuarial profession may not work for health services research in general.
Despite this, I would suggest that there are aspects of actuarial professional culture that should be embraced by health services research. In particular, actuaries tend to hold each other’s feet to the fire. For example, if a presenter at an actuarial conference made assertions that they had no evidence to support, I would expect actuaries to (respectfully) question them on the spot. I believe this culture arises because our responsibility to the public and the rigors of our training are top of mind for us.
The other mechanism that comes to mind as a possible means to improve critical thinking in academia is peer review. What does peer review offer in terms of promoting quality scholarship? While peer review provided by academic journals certainly should add an element of accountability to research standards, it has its limitations. Some say that academic research is in the midst of a crisis; a large majority of peer-reviewed studies are not replicable; authors commit fraud by manipulating or falsifying data or the peer review process (examples here and here). This indicates to me that peer review is not enough to ensure that health services research meets some minimal standard of research rigor. In addition, peer review does not guide how Ph.D.s conduct themselves in other situations or what they say in other venues.
We should aim to enhance how we as health services researchers are perceived by illustrating that we take Ph.D. level scholarship and research seriously whenever we are representing ourselves as Ph.D.s.
As I mentioned, I have been shocked at the groupthink and poor critical thinking exhibited by Ph.D.s at an industry conference. Given that I have not yet attended many such conferences, the fact that I’ve encountered this already suggests that it may be common. I propose that the best remedy at hand to improve the state of Ph.D. scholarship in HSR is to increase the amount and level of dialogue. I believe more dialogue, starting at the leadership level and permeating throughout the organization, could enhance how AcademyHealth is perceived by illustrating that we take Ph.D. level scholarship and research seriously whenever we are representing ourselves as Ph.D.s. More dialogue about underlying assumptions and philosophies would strengthen us by encouraging us to engage with each other to discuss and question our evidence and views.
In summary, while it seems that some areas of health services research suffer from a lack of critical thinking, I am hopeful that more and better dialogue, perhaps inspired by the professional culture of actuaries, will be embraced in HSR and at AcademyHealth. If each of us works a little harder to think more broadly, consider other points of view, question the conventional wisdom, and occasionally play devil’s advocate with our colleagues, the level of scholarship of health services research will increase. Given how important health and health care are for everyone, we should not settle for anything less.