Accuracy of Psychiatrists’ Assessment of Medication Adherence in an Outpatient Setting

Background: Many patients have uncontrolled psychiatric symptoms because they are not taking their medication as prescribed. Psychiatrists may have difficulty accurately assessing medication adherence, which is important because it helps guide them in how they prescribe. If nonadherence is the cause of uncontrolled symptoms, then strategies to improve adherence are advised. However, if nonadherence is not the cause, then the usual course of action would be to intensify or modify the medication regimen. Knowing whether nonadherence is a factor at the time of an appointment could help guide clinical decision making in real-time. Methods: A cohort of established patients in an outpatient mental health treatment clinic at a large health network was studied from November 2018 to August 2019. Blood drug levels of several oral antipsychotic medications were obtained and placed in the following three categories: below, within, or above the therapeutic range of published cutoff points. Treating physicians answered Likert-scale questions regarding their assessment of patient adherence. Subsequently, blood drug levels were compared to the psychiatrists’ assessment of adherence using a Kappa coefficient. Results: Sixty-four patients being prescribed antipsychotic medications were analyzed. A total of 87.5% of treating psychiatrists thought their patients were always adherent or adherent greater than 50% of the time. However, based on blood levels, 14% of the 42 patients at target FDA dosing for their medication and diagnosis were below the therapeutic range. The Kappa coefficient was used to find the level of agreement between the treating psychiatrist’s perception of patient adherence and the blood drug level. It was determined to be 0.14 which is consistent with no agreement between the two measurements. Conclusions: Treating psychiatrists inaccurately estimated oral antipsychotic medication adherence based on clinical impression alone. Making an objective measure of adherence available at the time of an appointment could help psychiatrists recognize nonadherence in real-time and inform prescribing decisions.


Introduction
Nonadherence with oral antipsychotic medications is commonly cited as a cause of partially to fully uncontrolled psychiatric symptoms [1,2]. Treating psychiatrists should make decisions in real-time about how to best handle such symptoms. They should consider if the cause of the symptoms is related to nonadherence versus lack of efficacy so that they can best decide whether to increase the dosage of a medication or change the medication. It remains questionable and unclear how accurate psychiatrists are about predicting whether their patients are adherent based on clinical assessment alone. However, a prior study demonstrated that psychiatrists in an emergency room setting were not accurate in predicting whether patients were compliant with their antipsychotic medications [3]. Additionally, in studies with other disciplines, primary care physicians were not accurate at predicting adherence to blood pressure medications [4]. This study sought to assess how accurate treating psychiatrists were at identifying their patients' medication adherence by comparing their clinical impression with blood medication level in an outpatient setting.

Materials And Methods
The goal of the study was to retain blood samples from individuals receiving Risperidone, Aripiprazole, Clozapine, Olanzapine, and Quetiapine therapy for research purposes in the development of the company's diagnostic tests. Patients at the institution's outpatient psychiatric practices who were being prescribed the psychotropic drugs of interest-based on a review of their electronic medical record were approached to see if they would be willing to have their blood drawn. Patients were compensated with a $25 gift card for their participation. Saladax Biomedical, Inc. performed the analysis of medication drug levels using their validated immunoassays, and results were obtained within several hours. Subsequently, treating psychiatrists of the enrolled patients were asked to fill out a two-question survey based on a prior study by Meddings et al. regarding their perception of their patient's adherence to the medication as well as how difficult the lack of adherence made it to control their patient's symptoms [4].
Descriptive statistics were used to summarize the study sample. The two survey questions were summed to create a continuous adherence score, after reverse-scoring the first survey question, "How often does this subject adhere to the psychiatric medication regimen?" Psychiatrist perception of patient adherence was defined by using responses to the second survey question, "How much does a lack of adherence make it difficult to control this subject's symptoms?" If the physician responded to this survey question as "A lot" or "A great deal" the patient was considered non-adherent. The Chi-square test was used to determine if there was an association between physician perception of adherence and therapeutic range. The Kappa coefficient was used to find the level of agreement between the psychiatrist's perception of patient adherence and drug blood level. Statistical significance was set at p=0.05. Statistical analysis software (SAS, Cary, NC, USA) was used to conduct statistical analysis.

Results
Sixty-four patients were included in the study. General demographics of the study participants are included in Table 1.  Medication dosages for each patient were reviewed to assess whether the patient was at or below the target doing based on FDA prescribing guidelines for their diagnosis [5]. In instances when a dosing range was given, the minimum dose of the range was used as the target dose. This step was performed to ensure that low dosing was not the reason for a subtherapeutic blood drug level rather than nonadherence. For Risperdal, a minimum dose of 4mg for schizophrenia/schizoaffective and 1mg for bipolar disorder or major depressive disorder was required to be a target. For Aripiprazole, a minimum dose of 10mg for schizophrenia/schizoaffective and 15mg for bipolar disorder was required. For Clozapine, a minimum dose of 300mg for schizophrenia/schizoaffective was required. For Quetiapine, a minimum dose of 150mg for schizophrenia/schizoaffective, 300mg for bipolar depression, and 400mg for bipolar mania was required. For Olanzapine, a minimum dose of 10mg for either schizophrenia/schizoaffective or bipolar disorder was required. Using these FDA guidelines, 42 (66%) patients were at target dosing of their medications. There was no maximum limit imposed on dosing as the purpose was to ensure that low dosing was not the reason for subtherapeutic blood drug levels. See Table 2 for details.  Sixty-four surveys were filled out by nine different treating psychiatrists at three different outpatient psychiatric practices within the health network. One participant had two surveys completed because he was on multiple psychiatric medications and provided two blood samples. One participant's survey was not filled out because his psychiatrist practiced outside the network. Most treating psychiatrists thought that their patients were always or frequently adherent to their psychiatric medication regimen. Similarly, most treating psychiatrists did not find that a lack of adherence made it difficult to control their patient's symptoms. See Table 3 for details.

TABLE 3: Physician survey results
For the 42 patients at target medication dosing based on FDA guidelines for their diagnosis, six (14%) were found to have blood drug levels below established cutoffs [6,7]. Based on these cutoffs the minimum blood level to be considered "therapeutic" was 20ng/mL for Risperdal, 100ng/mL for Aripiprazole, 350ng/mL for Clozapine, 100ng/mL for Quetiapine, and 20ng/mL for Olanzapine. Only the blood levels of the 42 patients at target medication dosing were used for comparison with the psychiatrists' perception of adherence because patients on subtherapeutic dosing of the medication may have had a subtherapeutic level due to low dosing rather than nonadherence.
Of the six patients with target medication dosing but subtherapeutic blood medication levels, five were thought to have been adherent by their treating psychiatrist. Of the 36 patients with target medication dosing and therapeutic blood medication levels, two were thought to have been nonadherent. Of note, adherence was defined as those patients who were thought to take their medication frequently or always. Non-adherence was defined as those patients who were thought to take their medication occasionally, rarely, or never. Using these definitions, there was no statistical difference between the two groups. See Table 4

TABLE 4: Physician perception of adherence and classification of the therapeutic range for patients at or above target dose
The Kappa coefficient was also determined for all drugs combined and for each individual drug. A Kappa coefficient measures the extent of agreement between two measures, which in this case was the physician's perception of patient adherence (defined as being either adherent or non-adherent), and medication blood level (defined as below therapeutic range or at/above therapeutic range). When looking collectively at all 42 blood drug levels, the Kappa coefficient was 0.14, which is consistent with no agreement between the two measures. When broken down by individual medication, only Risperdal showed almost perfect agreement. Olanzapine showed no agreement and Quetiapine showed minimal agreement. Aripiprazole and Clozapine could not be calculated because no patients fell into the non-adherent definition. See Table 5 for details.

Discussion
The overall findings suggest that treating psychiatrists were not accurate at determining adherence to oral antipsychotic medication based on clinical examination alone given the low Kappa coefficient (0.14). This finding could provide support for the need for point-of-care testing of blood medication levels to help with timely prescribing decisions.
It is notable that most treating psychiatrists in this study thought that their patients were adherent. Several factors could explain this finding. The study participants were chronic, established, outpatients. Most of the psychiatrists had been treating their patients for at least 5 to 10 years. To remain enrolled in the outpatient practices, patients must be compliant with appointments or they are at risk of having their case closed. This expectation alone would suggest some basic level of adherence with treatment. Also, the study was voluntary, and patients had to agree to participate, which would also suggest a more compliant population.
With all the features of this study population, it might have been expected that treating psychiatrists would have had a good ability to assess adherence. However, this was not the case. Therefore, these findings would suggest that other settings, such as the emergency room or inpatient psychiatric units where patients are not known to the treating psychiatrists, would be even more challenging environments to assess adherence and make accurate prescribing decisions. These environments would likely benefit even more strongly to access to point-of-care testing of antipsychotic drug levels.
There are several limitations to this study. The most notable is its small sample size. Only the 42 samples from patients who were at target dosing for their medication and diagnosis could be utilized. If all samples had been included, then the results may have been confounded by low dosing rather than nonadherence as blood medication levels are dose-dependent. An additional limitation is the assumption that knowing whether a patient is an adherent will help with symptom control. For example, some patients will still exhibit symptoms even while on medication, and some symptoms, such as negative symptoms, are not as responsive to medications. Further studies would be needed to see if timely access to blood medication levels impacts treatment outcomes.

Conclusions
Treating psychiatrists in an outpatient setting were not accurate in determining oral antipsychotic medication adherence based on clinical impression alone. Making an objective measure of adherence available in real-time during a patient encounter may help providers decipher whether symptoms are a result of nonadherence versus ineffectiveness and help to inform prescribing decisions.

Additional Information Disclosures
Human subjects: Consent was obtained by all participants in this study. Lehigh Valley Health Network IRB issued approval STUDY00000165. The study was approved on Nov 6, 2018 by the Lehigh Valley Health Network IRB. Animal subjects: All authors have confirmed that this study did not involve animal subjects or tissue. Conflicts of interest: In compliance with the ICMJE uniform disclosure form, all authors declare the following: Payment/services info: Saladax Biomedical provided funded for study personnel and also ran testing of serum drug levels. Financial relationships: All authors have declared that they have no financial relationships at present or within the previous three years with any organizations that might have an interest in the submitted work. Other relationships: All authors have declared that there are no other relationships or activities that could appear to have influenced the submitted work.