Development and Validation of a Comprehensive Questionnaire to Assess Interpersonal Discord (Bullying, Harassment, and Discrimination) at the Workplace in a Healthcare Setting

Objective This study was conducted to develop and validate a comprehensive questionnaire to assess bullying, discrimination, and harassment in healthcare settings. Methodology A mixed-method study design was used to develop and validate the questionnaire. In phase I, qualitative approaches were used for the development, which included literature search, focus group discussions (FGDs), following which the construct was developed. In phase II, face validity and construct validity were established using quantitative approaches. Results The final questionnaire consists of 25 items divided into five sections addressing the burden, impact, reasons for underreporting, risk factors, and mitigation strategies. The questionnaire has very good consistency with a Cronbach’s alpha score of 0.86. Conclusion This is a comprehensive tool with appropriate psychometric properties with potential use for evaluating the problem of interpersonal discord in the form of bullying, harassment, and discrimination in a healthcare setting.


Study design and ethical consideration
The development and validation of the questionnaire were performed using a mixed-method design with standardized techniques (Figure 1) [13,14,15]. The study was approved by the Institute Ethics Committee of the All India Institute of Medical Sciences, New Delhi (IEC-844/06.12.2019, RP-46/2020). The informed written consent was taken from all the participants and confidentiality and anonymity were assured.

Phase 1: Development of the questionnaire
In this phase, a systematic methodology was employed by the incorporation of the following key steps: literature review, focus groups discussions (FGDs), expert evaluation, and pre-testing.
The first step included a comprehensive and exhaustive literature search, which was done using a search string (aggression OR violence OR bullying OR harassment OR abuse OR discrimination) AND (surgeon OR resident OR intern OR physician OR doctor OR "general practitioner" OR "healthcare" OR clinicians) AND (workplace) AND ("risk factor" OR predictor OR determinants) AND (prevent* OR strategy* OR intervention) on PubMed and Wiley. The initial search resulted in 714 articles, from which 18 relevant articles were selected. A total of 32 items were generated in this step.
In the following step, four FGDs were conducted with resident doctors, nurses, and faculty members (n=22), each having a minimum of five and a maximum of seven participants from various departments of the hospital. Seven items were generated in this step. The duplicate items were removed, and the final pool of questionnaires with 30 items was categorized into the following five domains: forms of discord, the impact of discord, reporting of discord, mitigation strategies for discord, and risk factors of discord. Emphasis was laid on keeping the language of the questionnaire simple, clear, and unambiguous.

Phase 2: Validation of the questionnaire
For expert validation, qualitative and quantitative approaches were used. A panel of 12 experts was invited to evaluate the qualitative validity of the questionnaire and comment on the correctness and quality of the items. Items were changed or reworded as per the inputs. For quantitative content validity, the content validity ratio (CVR) and content validity index (CVI) were derived. The experts evaluated the items based on need, clarity, and relevance. The usefulness of items was graded on a 3-point scale, with scores ranging from 1 (not required), 0 (helpful but not essential), and +1. (essential). The CVR formula is CVR=(Ne-N/2)/(N/2), where Ne is the number of participants who marked an item as essential, and N is the total number of participants [16]. The appropriate CVR values were calculated using the Lawshe scores. Each item's relevance and clarity were also determined using a 4-point Likert scale: (1) not relevant/clear, (2) marginally relevant/clear and need revision, (3) relevant/clear and requires minimal revision, and (4) very relevant/clear. The proportion of experts who rated an item as relevant/clear/simple determined its CVI (ratings of 3 or 4) [17]. Items with a CVI value of less than 0.7 were removed, and those with a score between 0.7 and 0.79 were changed based on expert advice [17]. After this, the questionnaire with 25 items was pretested on 20 participants, and further changes were implemented as per their suggestions. The questionnaire was modified by adding (three items), deleting (eight items), and rewording (four items) items based on their suggestions.
Following this, a web-based questionnaire was distributed in August 2021, through a shared web link. The participants were informed about the purpose of the study through a 'Participant Information Sheet', and their responses were recorded by ensuring anonymity and prior consent. A total of 130 resident doctors, faculty members, and other healthcare workers from various departments of the hospital completed the questionnaires. The data was collected through a convenience sampling method, and the principle of maximum diversity was ensured.

Statistical analysis
The analysis of qualitative demographic variables such as gender, education status, and occupation was done using descriptive statistics. For the quantitative variables, mean, median, standard deviation, quartile, and range were calculated. The internal consistency (i.e., the range at which the items on the instrument measure the same thing) of the questionnaire was assessed with Cronbach's alpha. Good internal consistency was indicated by a Cronbach alpha score of 0.7 or higher. Exploratory factor analysis was done to evaluate the subdomain structure. The aim of this technique was the estimation of factors and to reduce the dimensionality of a large number of variables to a fewer number of factors. The sample adequacy was measured by the Kaiser-Meyer-Olkin (KMO) test, and values of more than 0.5 showed that the data were suitable for factor analysis. Bartlett's test of sphericity is a statistical test for the overall significance of all correlations within a correlation matrix. Eigenvalues represent the variance in the variables that is accounted for by a specific factor.

Results
The questionnaire comprises 25 items, which are divided into the following five domains: forms of discord (four items), the impact of discord (five items), reporting of discord (eight items), mitigation strategies for discord (four items), and risk-factors of discord (four items). In the first domain, the questions are specific to the spectrum and prevalence of workplace-based discord; they are designed with caution to capture aspects of bullying, harassment, and discrimination based on race, community, religion, and culture. In the second domain, the questions are focused on understanding the impact of such episodes on personal and social life. Even though the incidences of workplace discord are very high in all parts of the world, many significant findings suggest that the reporting of such incidents is very low [18]. Hence the third domain deals with the reasons related to underreporting of such incidents from individual to organizational levels. In the fourth domain, questions related to mitigation strategies are also added, which focus on complaint redressal, legal rights, and responsibilities, improving management facilities, and strong legislative measures, and in the fifth domain, questions related to risk factors specific to the scenario of the country to address the problem holistically are dealt with.

Sociodemographic profile of the participants
The sociodemographic characteristics of 130 participants included in the validation phase are presented in Table 1. The mean age of the participants was 29.31 ± 4.64 years. The entire sample worked in a government setting and had a higher proportion of males (63%) as compared to females (36%). The majority of them hailed from metropolitan cities (92%). There was a fair representation from all departments of the healthcare setting.

Validity of the questionnaire
To screen out the intercorrelation and singularity between items, the intercorrelation matrix was used. In the screening process, researchers found only three items (6, 7, and 8) showing intercorrelation above 0.70. But these were not excluded from the final draft of the questionnaire as these were found to be important items regarding the impact of bullying at the workplace. Cronbach's alpha is considered one of the most effective reliability tests. It helps in finding out the internal consistency of any questionnaire. In our study, it was found to be good, i.e., α=0.86. Principal component factor analysis along with Varimax rotation was run to establish the sampling adequacy and factorial validity of the questionnaire. After running the principal component factor analysis, the factorial validity was found to be 73.66, which is satisfactory. Additionally, the KMO value (0.766) and Bartlett's test of sphericity (p<0.01) determined the adequacy of the sample. The intercorrelation matrix is presented in Table 2.  Q1   Q2  Q3  Q4  Q5  Q6  Q7  Q8  Q9  Q10  Q11  Q12  Q13  Q14  Q15  Q16  Q17  Q18  Q19  Q20  Q21  Q22  Q23  Q24

Discussion
The healthcare system and its workers are the cornerstones of a country as they contribute significantly to its development and prosperity. Their mental, physical, emotional, and social wellbeing are of utmost importance as it directly affects their work efficiency, motivation, turnover intention, and quality of life [19]. The rising incidents of discord at the workplace have motivated various researchers to develop tools to assess the magnitude of the problem and to devise strategies to address it systematically [20,21,22].
Globally, various studies have been conducted to assess the problem of bullying, harassment, and discrimination in the healthcare sector. But most of them have assessed these problems separately and used semi-structured interview techniques, which makes it difficult to perform a comparative analysis between different studies [12]. Additionally, they either have low psychometric properties or lack comprehensiveness as a single scale to capture various domains. While we constructed a questionnaire on various aspects that affect interpersonal discord in healthcare settings, we extensively reviewed the existing data and based on that, various domains of bullying, harassment, and discrimination were identified. The questionnaire is free to use and is presented in Table 3.

Name:
Age (in years): Gender:  A heterogenous Likert scale was used to scale the responses based on the intensity or severity of the problem in each domain. Section A comprises items related to forms of discord like bullying, harassment, and discrimination based on race/caste/community, as well as the state of origin. Along with this, the issue of sexual harassment has also been given due importance, as these forms of non-physical violence are more common and mostly overlooked [23]. Since such episodes affect all aspects of an individual's life, Section B has focused on its impact on personal, familial, social, and psychological wellbeing as well. Despite a high prevalence, the discord cases are often underreported and normalized [24]; therefore, in Section C, the prime focus was on assessing their reasons for not reporting such events, which included questions related to their psychological conditioning and organizational dynamics redressal system. In Section D, the focus was placed on solving the problem by incorporating changes in the training, legislation, redressal system, etc. Lastly, Section E intends to highlight the various risk factors associated with discord, which includes institutional, interpersonal, social, and legislative attributes. This questionnaire represents a way forward to tackle the problem of bullying, harassment, and discrimination, as it is likely to provide baseline parameters to work upon. It will help identify not just the frequency or severity of the problem but its risk factors, impact, and mitigation strategies that will help in implementing changes from individual to policy levels. This will also aid in conducting large cross-sectional studies for analyzing the contrast in workplace discord among various countries or within a country across various settings.

Strengths and limitations
The questionnaire we devised is easy to administer and addresses the issues of discord in a simple and clear manner. It is scientific and psychometrically evaluated and, to our belief, will be useful in assessing a wide range of issues from the spectrum and prevalence to the mitigation strategies for workplace discord from individual to policy levels. The major limitation is the semi-quantitative nature of the questionnaire and the fact that it lacks the assessment of predictive validity.

Conclusions
Despite increased awareness and having various laws and measures in place, discord in the healthcare setting persists, and there are significant factors that pose as barriers to reporting and eradicating the problem. Gaining awareness and knowledge about the most prevalent behaviors will aid in the development of interventions targeted at the most problematic negative behaviors. The questionnaire we developed is a reliable and valid tool to assess bullying, harassment, and discrimination in the healthcare setting. It is clear, concise, and easy to administer. It has the potential to dismantle the normalization of discord at the workplace and will contribute greatly to the existing literature on the topic.