Diagnostic Accuracy of Point-of-Care Tests Measuring Glycosylated Haemoglobin (HbA1c) for Glycemic Control: A Field Study in India

Objectives This study was performed to estimate diagnostic accuracy of the two commercially available point-of-care tests to identify poor glycemic control defined by HbA1c levels, with high-performance liquid chromatography (HPLC) as a reference. Settings The study was carried at two locations, general medical outpatient department of a teaching medical college in Bhopal (urban), and a primary health care centre in a rural area in the state of Madhya Pradesh, India. Participants All individuals with diabetes mellitus who presented to the health care facility for assessment of glycemic control. We compared HbA1c estimated from two index tests (Hemocue Hb 501, Sweden; SD Biosensor, South Korea) from capillary blood samples with HPLC performed from venous blood, as a reference standard. Primary and secondary outcome measures Diagnostic properties of index tests such as sensitivity, specificity, positive and negative predictive value and diagnostic accuracy for identifying poor glycemic control were primary outcome measures. Lin’s concordance correlation coefficient (CCC) was secondary outcome measure. Results Out of 114 patients, all received reference standard - 103 patients received Hemocue A1C test and 110 patients received SD Biosensor test. Overall both the index tests had similar diagnostic accuracy estimates. The area under the Receiver Operating Curve for SDA1c device was 0.935 (95% CI 0.886-0.983), and for Hemocue device was 0.938 (95% CI 0.893-0.984). The Hemocue device HbA1c value of above 7.0 (positive) correctly predicted poor glycemic control 92% times (81.58% for SD device). There were 4 vs. 11 device failures and 14 vs. 12 failures with SD and Hemocue, respectively. Ambient air temperatures were no different for the device test failures. Conclusions Commercially available point-of-care tests evaluated in this study are comparable and an acceptable alternative to HPLC-based measurements for the assessment of glycemic control. Tests and device failure rates of both the index tests are similar.


Introduction
Diabetes is a worldwide health concern characterized by elevated blood glucose levels caused by inadequate pancreatic insulin synthesis and/or decreased insulin sensitivity [1]. The prevalence of diabetes rapidly rising in India increased from 26.0 million in 1990 to 65.0 million in 2016 [2]. The total number of people with diabetes is projected to rise from 31.7 million in 2000 to 79.4 million in 2030 [3]. Early detection of diabetes and its consequences allows for prompt and effective preventive therapy [4], which helps to prevent the development or progression of macrovascular and microvascular illness. The key to avoiding diabetes problems is to keep blood glucose levels within normal limits in addition to lifestyle modifications, blood pressure control, and dyslipidemia management, etc. [5]. The reference measure of glycaemic management in people with diabetes is determining the glycated haemoglobin A1c (HbA1c) levels. It reflects the glycemic level throughout the previous two to three months [6]. The use of HbA1c in the management of diabetes has been well established since the Diabetes Control and Complications Trial (DCCT) and the UK Prospective Diabetes Study (UKPDS) trial [7][8]. American Diabetes Association (ADA) in 2010 updated their guidelines for the diagnosis of diabetes, taking HbA1c into account. Since then, cut-offs of 6.5 percent have been utilised as a handy marker for both diagnosis and follow-up of diabetic patients [9][10].
Various methods are available for the measurement of HbA1c levels, separate HbA1c from other types of hemoglobin using charge differences (e.g., ion-exchange high-performance liquid chromatography [HPLC], electrophoresis or isoelectric focusing), structural difference (e.g., affinity chromatography or immunoassay), or chemical analysis (e.g., photometry and spectrophotometry) [11][12][13][14]. More than 30 different methods have been described to estimate HbA1c. Due to the inconsistency in the technique used and reporting format, ADA recommends the International Federation for Clinical Chemistry (IFCC) units (mmol/mol) and derived the National Glycohemoglobin Standardization Program (NGSP) units (%) using the IFCC-NGSP master equation. The methods certified by the NGSP, a US-based organization, relate individual techniques to HbA1c values as obtained in the landmark DCCT trial [15]. The most commonly used techniques are ion-exchange high-performance liquid chromatography (HPLC), capillary chromatography, borate affinity assay, and immunoassay; HPLC is considered the gold standard [11]. However, it has greater technical needs, including equipment, expenditure, and longer turnaround times. In the past few years, the concept of point-of-care (POC) HbA1c testing has emerged [16]. Commonly used POC, HbA1c assays use the principles of borate affinity or immunoassay. In borate affinity assays glycated hemoglobin binds to borate resins, with the potential to overestimate HbA1c, as the technique is not HbA1c specific [11]. In contrast, immunoassay uses anti-HbA1c antibodies against the glycated N-terminal of a beta-hemoglobin chain. While this technique is attractive, interference with other hemoglobin chains remains a concern.
Previous HbA1c diagnostic accuracy studies were conducted in high-income countries that have more controlled environmental and logistic conditions for the storage of test kits and operation of the equipment. Ours is the first study conducted in public healthcare settings of a tropical middle-income country. The current study was designed to answer two research questions: a) Among individuals with diabetes mellitus, point-of-care (POC) HbA1c measurement devices (based on immunoassay or borate affinity principle), as compared to HPLC as a reference standard, are accurate for assessing optimal glycemic control (HbA1c ≤ 7%); b) In a community-based setting in a rural area, the use of a POC device to estimate the HbA1c level is feasible. We aimed to compare the accuracy and feasibility of two commercially available point-of-care devices. This article was previously posted to the Preprints preprint server on January 29, 2019.

Design and setting
This was a cross-sectional diagnostic accuracy and feasibility study. The study was conducted at two locations, the medical outpatient department at the All India Institute of Medical Sciences Bhopal, Madhya Pradesh (Urban), and Centre for Rural Health AIIMS (CRHA) located at the Primary Health Centre at Chiklod, Madhya Pradesh (Rural). Individuals with diabetes mellitus who sought care at both facilities were evaluated for glycemic control.

Participants
The study sought to include all individuals with diabetes mellitus who presented to the health care facility for the assessment of glycemic control. The individuals were previously diagnosed with diabetes (based on ADA criteria: fasting plasma glucose of 126 mg/dL or above or post-meal plasma glucose of 200 mg/dL or above or HbA1c level of 6.5% or above) at least three months before the date of inclusion in the study. Written informed consent was obtained from all the eligible participants. For logistic reasons, the participants were sampled from these facilities once weekly for the study duration.

Ethics issues and permissions
The study design was approved by the Institutional Human Ethics Committee of All India Institute of Medical Sciences, Bhopal (Ref: IHEC-LOP/2015/IM0056). Participant Information Sheet in Hindi language was provided to each participant. All participants provided written informed consent prior to initiation of any study procedures. Participants were enrolled after obtaining written informed consent. The results of biochemical investigations were communicated to the participants and were appropriately managed or referred.

Study procedures
All eligible and consenting participants were administered a questionnaire to collect information about demographics, duration of diagnosis of diabetes mellitus, current therapies, and any history of hemoglobinopathy. Subsequently, the following samples for the index test and reference standard were collected within a 10-minute interval of each other. We first collected a 2-mL venous sample in an ethylenediaminetetraacetic acid (EDTA) tube, which was immediately stored between 4 and 80°C in an icepack-containing vaccine carrier. These samples were transported to the laboratory on the same day in a temperature-controlled environment for HPLC-based HbA1c estimation (Reference Standard). The test was performed in a National Accreditation Board for Testing Calibration Laboratories (NABL) accredited laboratory. Then we collected capillary blood samples using a finger-prick. We collected one drop of capillary blood (approximately 5 μL) for each of the index tests: a Borate affinity-based point-of-care test (Hemocue A1c 501, Sweden) and an immunoassay-based point-of-care test (SD A1c care Bioassay, South Korea). The characteristics of the two index tests are presented in Table 1. Point-of-care tests were performed in front of the patient, using standard techniques as per the manufacturer's guidelines. Briefly, in the borate affinity meter (Hemocue A1c 510) a blood sample was collected using a reagent pack (which draws appropriate volume into the reagent pack), and the reagent pack is inserted into the cartridge. The cartridge was then inserted into the machine, and the results are displayed on the meter in three minutes. In the immunoassay meter (SD A1c care) a drop of blood (about 5 μL) is collected in a reagent tube, and allowed to mix for a minute. A test strip is inserted into the meter, and 5 μL of the blood-reagent mixture is applied to the sample port on the strip. The results were obtained in three minutes. Both the index tests were conducted, and interpreted, blinded to the results of the reference standard. The HPLC results were available only the next day.

Characteristics
HemoCue® HbA1c   For all the tests we have collected the following four variables to know the feasibility of the point-of-care HbA1c assays: i) Device failure events -the device does not get powered on, or is unable to perform a measurement. ii) Test failure events -the device gives an error message or the testing cartridge is prematurely ejected and no meaningful output is received. Test failure events are expected to result in wastage of the testing strips or cartridges. We also measured the ambient room temperature of the facility where the tests were carried out, using a temperature logger device (Lascar Electronics, Erie, PA, USA).

Sample size
We have calculated sample size by using Buderer's formula for determining sensitivity [17]. Assuming the proportion of glycemic control (HbA1c ≤ 7%) to be about 40% in the participants, for a desired sensitivity of at least 90%, we needed a sample size of at least 111 participants for a relative precision of 10% of sensitivity (i.e., 95% CI for 90% point estimate to be between 81%-99%).

Statistical analysis
We have summarized nominal variables with frequency and percent and numerical variables with mean and standard deviation. Concordance in estimated HbA1c values was visualized by creating Bland-Altmann plots (mean of HbA1c by index and reference test vs difference in HbA1 between index and reference test) and scatter plots. For quantifying concordance, we have estimated Lin's concordance correlation coefficient, Pearson's correlation coefficient, and Area Under ROC Curve (AUC). Lin's concordance correlation coefficient (CCC) is used to examine the agreement between two continuous measurements. Like Pearson's correlation coefficient, CCC ranges from -1 to +1. However, if one method of measurement differs systematically from other methods then we will get a higher value of correlation coefficient, but concordance is not ensured. Therefore, Lin's CCC is a better measurement when we aim to assess agreement or concordance. For qualitative agreement, using a cut-off of 7% we estimated traditional measures of diagnostic accuracy (Sensitivity, Specificity, Positive and Negative predictive values, and Likelihood ratios).
We measured the precision of our estimates by calculating 95% confidence intervals. Data analysis was done using SPSS software (IBM SPSS Statistics for Macintosh, Version 26.0., IBM Corp., Armonk, NY) and R software.

Results
The study was conducted between April and September 2016. A total of 114 patients (63 from urban and 51 from rural facility) were included in the study. Most patients were middle-aged (mean age 53.4 years (SD 11.5)), with an age range of 18 to 80 years. A total of 45 (39.5%) participants were women. The characteristics of the study population are detailed in Table 2.

Discussion
In the current study, we demonstrated that both techniques for point-of-care estimation of HbA1c levels were comparable to the reference standard. The current study is one of the largest and only such work from the developing world. We encountered more device failures with the Hemocue device and comparable test failures with both devices. This provides us with important insights into the performance of point-of-care tests in actual field conditions.
Our diagnostic study included those participants who would have received the tests in actual clinical practice. Most participants in our study were middle-aged men with uncontrolled T1DM or T2DM. These demographics are comparable to previous studies in terms of age but had a lower representation of women [16]. The mean HbA1c level by reference standard in our study was 8.03% (SD 2.02), higher than in previous studies [18]. These differences are a reflection of poor glycemic control in a developing country setting.
Since there was a greater variation in the HbA1c values in our study, it has a potential to bias diagnostic accuracy estimates towards the null, as compared to the studies that included participants with lower mean HbA1c values. Despite this only 2.3% and 4.8% values were outside the two-standard deviation in the Bland-Altmann plot for SD and Hemocue A1c device, respectively. This variation is comparable to previously reported studies. Previous HbA1c POC studies are listed in Table 4.  These differences in the diagnostic accuracy estimates are due to small variations around the cut-off in both assay methods (SD assay with a mean 0.34 lower, and Hemocue A1c device with a mean 0.43 higher values) as compared to HPLC-based measurements. For a clinician who is assessing poor glycemic control, a Hemocue A1c measurement of 7% or higher, is 81% predictive of a similar result by HPLC in contrast to 92% predictiveness by SD assay (positive predictive value). In contrast, if the aim is to assess optimal glycemic control Hemocue A1c assay is 91% predictive in contrast to 85% by SD assay (negative predictive value).
Overall both assays are similar as their AUCs in ROC analysis are similar. The overall test performances of both index tests in the current study are also comparable to a study by Marley et al. from Australia [21].
There was higher device failure with Hemocue A1c device in our study. Device failure is a condition when either the device does not switch on or is unable to receive a test strip or cartridge. The key reason for high device failure with Hemocue A1c device (11/114) was a power failure, as this equipment operates only on running electricity and does not have a battery backup. The principal reason for SD A1c device to fail was battery run-off. Availability of running electricity is a constraint, and longer battery life is an asset for any point-of-care device in a developing country setting. We encountered 14 test failures with the SD A1c device and 12 test failures with the Hemocue A1c device. Test failure is a condition where the device gives an error message after insertion of the test strip. We could perform the test after using a new test strip in all these 26 situations. The test failures were random and could not be explained by temperature and more parameters like humidity have to be taken into account in future studies. The test failure rate and test-strip/cartridge wastage were similar for both devices. While both the manufacturers suggest performing the test below a room temperature of 28°C, the ambient room temperatures were higher than this benchmark on most occasions. This is also typical of a tropical developing country scenario where POC tests are performed in a non-temperature-controlled environment. The room temperatures are above 28°C in most months of the year in our locality.
The current study was conducted in the same population that would have received the test in actual practice. The performance of the index tests and interpretation of its results was blinded and independent of the reference standard. Both the index tests and reference standard were performed using blood samples that were collected at the same time. We used clinically useful cut-offs for the interpretation of our results. The study was conducted in two different facilities, where the ambient temperature ranged from 25-42°C and the range of HbA1c tested was also wide-ranging from 4.5 to 15.3%. Our study had certain limitations. We did not study the variant hemoglobin and absolute hemoglobin level of the patients, however, this is unlikely to affect the results of our study as the prevalence of hemoglobinopathies is likely to be low. Our sample size was modest, yet this is one of the largest studies conducted. Our testing environment was at variance with what would have been prescribed by the manufacturers, but this limitation is inherent, as actual temperature and humidity levels in developing country facilities are likely to be less than ideal.