Reliability and Validity of EBM Resources of Pharmacy Research Questionnaire in Saudi Arabia

Phone no: +966504417712 E-mail: yalomi@gmail.com ABSTRACT Objectives: To declare reliability and validity of EBM Resources Knowledge used for Pharmacy Research Survey in Saudi Arabia. Methods: It is a cross-section survey developed by the authors and the researcher team. It’s based on the updated literature, national and international accreditation standards organizations. The internal consistency reliability through, inter-rater reliability, item-item coloration, item-total coloration, split half reliability (Gutmann’s λ6) and McDonald’s ω, Cronbach alpha. The validity contained of face content validity, construct validity through exploratory factorial analysis and confirmatory factor analysis. All analysis had been done through Statistical Package of Social Sciences (SPSS), Statistical Package of Social Sciences-Analysis of Moment Structures (SPSS-Amos) and Jeffrey’s Amazing Statistics Program (JASP). Results: A total of 209 pharmacists responded. The majority of responders were Saudi 185 (88.52%). The responders were males 108 (61.77%) and females 101 (48.33%). The three tests had been done with reliability of 16 questions. The completed number of responders (200) (mean ± SD) was 3.179 ± 0.436, McDonald’s ω, Cronbach alpha and Gutmann’s λ6 were 0.967, 0.966 and 0.976, respectively, inter-item coloration was 0.719, the item-total coloration >0.7, McDonald’s ω, Cronbach alpha and Gutmann’s λ6 value if deleted was >0.97. By using Exploratory Factor Analysis (EFA), the Kaiser-Meyer-Olkin Measure of sampling adequacy was 0.941 and Bartlett’s test of sphericity with approximate chi-square was <0.001. The commonalities extraction for all questions were >0.58, the related components were one of the rotated component matrix >0.59 of all 16 questions in component 1 as suggested. They were established by confirmatory with statistically significant (p<0.001) of the factor model, by factor analysis, by scree plot, pathway analysis and fit with the original survey. The confirmatory factor index was 0.862, Tucker-Lewis Index (TLI) was 0.841, Goodness of fit index (GFI) was 0.906 and Expected cross validation index (ECVI) was 3.270. The collinearity of 16 questions was the auto-correlation was 0.066 with not statically significant (p=0.317). The majority of 16 questions had Enjuone value close to number 1, while 3 questions only had condition index more than 30. All of the 16 questions had The Variance inflation factor (VIF) less than 10 and had tolerance more than 0.1. Conclusion: The pharmacist survey knowledge of evidence-based medicine resources for pharmacy practice and distributed the Kingdom of Saudi Arabia had high reliability and validity scale level. The pharmacy researcher can utilized surveys with an adequate number of sample sizes with further research in Saudi Arabia.


INTRODUCTION
Over the past years, pharmacy or healthcare resources developed from textbooks and journal paper types to primary or secondary and tertiary references like newly type of resources called evidence-based medicine resources. It's based on the best and high quality of method research design with meta-analysis of randomized controlled studies. The EBM resources one great to validate studies to accept or reject the pharmacy or medical recommendation. As a result, several types of research were used methods to validate their tool of collecting the data. Two concepts the researcher implemented for observations cross-sectional researches with very much used for survey distributed to target sample. The first concept called reliability "which defined as scale or test is reliable to the extent that repeat measurements made by it under constant conditions will give the same result". 1 There were several types of reliability, for instance, test-retest reliability, inter-rater reliability and the internal consistency reliability through by using biostatistical analysis. The reliability text demands to assure the accuracy of the data with got off an excellent survey that can be repeated several times with the same accuracy. The second concept used in the research called validity, "which defined as the degree to which the researcher has measured what he has set out to measure". 1 The validation procedures had various types, for instance, content and construct validity, which commonly used in practice. The construct validity needs particular training in the biostatistical software program to implement the tools. The validation process measured as a quality management tool used in the research and keep in direct specific goal without deviation, the high validity survey correlated with the question and each group of questions targets specific aim of the research. The scientific research contained of several steps that including revision of literature. The literature with substantial evidence like meta-analysis trial and randomized controlled study to low strength evidence with case report or experience. Multiple types of resources started from evidence-based medical (EBM) references and general searching engines. The literature review with an emphasis of EMB resources are very perilous in the pharmacy research and high demand for skills of searching all types of references. 2 Various studies discussed EBM knowledge or perception with validated survey. 3 The authors and his team have done one survey about EBM as general concept or resources of pharmacy with brief reliability and validity. 4 Besides, various publications released for undergraduate or post-graduate research without detail evidence or simple reliability and validity procedures. 5,6 There were some problems with pharmacy or medical researches with an emphasis on reliability and validity sections. 7,8 The author not away from any study about the reliability or validity survey of EBM used in pharmacy or medical research, however, some literature discussed about pharmacist perception of EBM including resources by using used previous validated survey. 6,[9][10][11][12] The aim of the current study is to discover the reliability and validity of the survey for EBM used in pharmaceutical research in the Kingdom of Saudi Arabia.

Survey Development
It is a cross-section survey developed by the authors and hit the researcher team. It's based on the updated literature, national and international accreditation standards organizations. [13][14][15][16] The survey entailed of two parts, the first section about demographic data of responders including genders, nationality with dichotomous data and age with ordinal data. The rest of the data as ordinal information that's including the responder's qualifications, background education, the board of pharmaceutical certificate and the current job and experience. The other section of demographic information was hospital data with ordinal data and included hospital bed capacity based on the ministry of health classification; the university updated hospital accreditation status from national and international accreditation institutions. The second part of the survey about patient satisfaction of pharmacy services. The section divided into several domains and each domain had several questions. The answers of the domains were likely with 1 (I do not need this knowledge), 2 (I do not have knowledge), 3 (Weak knowledge), 4 (Incomplete knowledge) and 5 (Complete knowledge). A pilot study was done through the authors and the team distributes electronically or manually to target responders 20-30 as a pilot. Sometimes they interview patients to assure all the questions clear and understood by the responders. All comments brought for discussion. The correction of the survey was done based on the agreement of most research members. 17 The research team tested the McDonald's ω and Cronbach alpha for internal reliability in the pilot responders by using Statistical Package of Social Sciences (SPSS), Statistical Package of Social Sciences-Analysis of Moment Structures (SPSS-Amos) and Jeffrey's Amazing Statistics Program (JASP). 18

Internal Consistency Reliability Item-item Correlation
The method was used to measure each question to another one, with high correlation results in more than 0.7, that is high internal consistency reliability survey. 1,18

Item-total Correlation
The method was used to measure the total questions allocates with each question alone. High results more than 0.7, the high correlate internal consistency reliability of the survey. 1,18 Split Half Reliability (Gutmann's λ6) The method was used through the SPSS and JASP program with the scale option and reliability section for all the questions with scale or ordinal data need to be measured. The test used a split-half option. The SPSS or JASP will split the question into two half and measure the coloration of the two groups. The high results more than 0.9 of correlation means the high reliability with internal consistency. 17,19,20 McDonald's ω, Cronbach Alpha The research team applied McDonald's ω and Cronbach alpha for internal reliability by using SPSS and JASP. All the questions with scale or ordinal data included in the analysis. The scale more than 0.9; it will be excellent internal consistency, 0.7-0.9 means good reliability, 0.3-0.6 means not acceptable reliability and the score less than 0.3 means weak reliability. 17,21,22 Face Content Validity The principle authors designed the survey and research team revised self-reliantly. Each member revised all the survey content questions based on the updated literature and experience. Any violations had been sent to all research team for further discussion and agreements. The survey had been corrected and agreement from the research team. One of the team members transferred all the surveys to the Arabic language and double-checked by all team members again for content and accurate translation. 1,17

Construct Validity Exploratory Factorial Analysis
The method was used for the construct validity of the survey. The factor was used univariate description and Kaiser-Myer-Olin measure of sampling adequacy and Bartlett's test sphericity. The extraction used principal components analysis, the Eigen values greater than (1) with the maximum iteration of convergence (25) and display through un-rotated faction solution and scree plot. The rotation used Varimax. 15,20

Confirmatory Factor Analysis
The test was done through SPSS-Amos and JASP software programs with factor variances, R-Sequated, fit measurements, factor loading, without emulation, error calculated with CI 95% and robust method, it was with the auto-estimator and without standardization, it was with pathway analysis. 15,20 Collinearity The test was done through JASP with linear regression for collinearity diagnostics including Eigen value and condition index, the coefficient used with CI 95% tolerance and variance inflation factor, the model fit through ANOVA and autocorrelation with Durbin-Watson. 23

Statistical Analysis
Various biostatical analysis was done in the current study like the Mc-Donald's ω, Cronbach alpha and Gutmann's λ6 for calculation reliability. The Kaiser-Meyer-Olkin measure of sampling adequacy and Bartlett's test of sphericity with approximate chi-square for Exploratory Factor Analysis (EFA). The factor variances, R-Sequated, fit measurements, factor loading, without emulation, error calculated with CI 95% and robust method, it was with the auto-estimator and without standardization, it was with scree plot and pathway analysis. Collinearity had been diagnostician through linear regression the variance inflation factor was calculated, the model fit through ANOVA and auto-correlation with Durbin-Watson. All biostatical analysis was done by the Statistical Package of Social Sciences (SPSS), SPSS-AMOS and Jeffrey's Amazing Statistics Program (JASP).

RESULTS
A total of 209 pharmacists responded. The majority of responders were Saudi 185 (88.52%). The among responders were males 108 (61.77%)  (Table 3 and 4). The square, multiple correlations of the questions R2 were from 0.326 to 0.813, while factor loading was all the questions >0.599 and it was a range (0.599-1.057) with p <0.001. In the pathway analysis that's each latent factor and observed coloration with >0.7 with p <0.001 as explored in pathway analysis (Figure 2).

Reliability
The three tests had been done of reliability with 16 questions for the initial (29) responders mean ± SD was 3.254 ± 0.320 and McDonald's ω, Cronbach alpha and Gutmann's λ6 were 0.977, 0.976 and 0.991, respectively, while inter-item coloration was 0.642. After the completed number of responders (200), the tests mean ± SD was 3.179 ± 0.436, McDonald's ω, Cronbach alpha and Gutmann's λ6 were 0.967, 0.966 and 0.976, respectively and inter-item coloration was 0.719 among the 30 responders. All the questions of item-total coloration >0. 6

Validity
By using Exploratory Factor Analysis (EFA), the Kaiser-Meyer-Olkin measure of sampling adequacy was 0.941 and Bartlett's test of sphericity with approximate chi-square was <0.001 ( Table 2). The commonalities extraction for all questions was >0.58, the related components were one with the rotated component matrix >0.59 of all the 16 questions in component 1 as suggested and discovered by scree plot (Figure 1). They were confirmed by confirmatory with statistically significant (p<0.001) of the factor model, by pathway analysis and fit with the original survey. The confirmatory factor index was 0.862, Tucker-Lewis Index (TLI) was 0.841, Goodness of fit index (GFI) was 0.906 and Expected cross valida-     only had condition index more than 30. All of the 16 questions had the Variance inflation factor (VIF) less than 10 and had tolerance more than 0.1 (Table 5).

DISCUSSION
The evidence-based medicine was playing a noteworthy role in pharmacy practice. 2 The pharmacist utilized evidence-based medicine during drug information resources and pharmacy research. 24 The authors and his colleagues conducted the study of pharmacists' knowledge about evidence-based medicine references in pharmacy research. The study was a cross-sectional survey distributed to the pharmacists. The author did some tools of reliability and validity briefly. 4 However, the details of reliability or validity for the survey of the knowledge of EBM resources used in pharmacy research had been done in the current study. The results showed in the pilot study of 30 responders with three different reliability tests was excellent and once repeated with completed responders, was also excellent with exceedingly more 0.97. The inter-item correlation or item-total coloration was right in the primary pilot responders and completed responders means the question of internal consistency was high reliable. The findings of the validity test showed a good sample size for exploratory factor analysis. The EFA recommends one factor with 16 questions with proper extraction and rotated component matrix. It's confirmed by CFA with one factor with statistical significance through a regression model, scree plot and pathway analysis. The findings showed that there is no collinearity with all the 16 questions except three-question might affect the validity of the survey. However, by increasing the number of responders will remove all collinearity related issues. The survey of knowledge about EBM resources used for pharmacy research was high reliability and validity. This is the first investigation that was done in the Kingdom of Saudi Arabia or the Gulf and Middle East countries.

CONCLUSION
The assessment of pharmacist knowledge with evidence-based medicine resources used in pharmacy research with a specific survey distributed in the Kingdom of Saudi Arabia. It was done through the reliability of