Self-rated health (SRH) is a commonly used outcome measure in social epidemiology studies, where it has been found to be inversely associated with various aspects of social position, such as level of education, ethnicity and socioeconomic groups.[1–5] Studies correlated with SRH have found varying results concerning medical health, functional ability, physiological variables, symptoms, sociodemographic variables, social factors, behavioral factors and personality factors, and the clinical correlate of SRH may be less evident than that of other measures such as longstanding illness.[1] Nevertheless, since the first clear demonstration in 1982[6], SRH has repeatedly been found to be a strong predictor of subsequent mortality.[7] Few, however, have investigated whether the strength of that predictive power is the same for different subgroups of the population and in different cultures. The association between SRH and mortality may not be as strong for some groups as for others. This is becoming a more pressing question as interest mounts in investigating social inequalities in health in low-income countries, in which anomalies are surfacing. Studies in India, for instance have shown that poorer groups report less morbidity than would be expected from their mortality profile. [8] This could indicate that self-rated health is context dependent and raises the possibility of cultural and socioeconomic differences in perceptions or reporting of SRH that might change its relationship with subsequent mortality, throwing doubts on the validity of using SRH for certain groups. For example, health problems may be voiced more loudly by wealthier groups, while poorer groups suffer in silence. Most previous studies have been carried out on elderly subjects. However, studies that have also included younger adults [4, 5, 9–11] have found a predictive value of SRH with respect to subsequent mortality. Most studies have found a stronger relation between SRH and mortality among men than among women, [7] but others have reported the opposite. [5] One study concluded that perceived health levels mainly reflect underlying disease burden. [12] However, little is known about how the relation between SRH and mortality is modified by age, sex, social class and concurrent longstanding illness and over time. Self-medication (SM), as one element of selfcare, is the use of medication without the prescription of health care professionals (e.g. resubmitting old prescriptions, sharing medication with relatives/family members or using leftover medications) for the treatment of self-recognized illnesses[13]. SM is a component of self-care and it is considered as a primary public health resource in the health care system. SM also encompasses the use of the medicines by the users for self-perceived health problems or the continuing use of medications formally prescribed earlier. Further broadening of the definition includes treatment of family members especially minors and elderly. Globally, the prevalence of SM is inconsistent ranging from 32.5 to 81.5% [14-16]. In Ethiopia, the prevalence of self-medication practice ranges from 12.8% to 77.1% [17]^. People may practice self-medication for a variety of reasons, like the urge for self-care, sympathy for family members in sickness, lack of health services, poverty, ignorance, misbelief, excessive advertisements of drugs, and availability of drugs in establishments other than pharmacies[18].Questionnaires are commonly used to assess views and beliefs in epidemiological research as well as clinical practice. Compared to alternative methods of assessing them, questionnaires are short, easy to administer, and require minimal resources, making them ideal for administration to large populations or as a quick screening tool. A few questionnaires are available on SM and SRH but none of these questionnaires arefully validated. Accordingly, to fill this gap, we aimed in the present study to design and validate a questionnaire for the assessment of Knowledge and Attitude related to SRH and SM in the general population. We undertook an extensive literature review to design a comprehensive questionnaire concerning both self-rated health and SM in Greece. We described the development process of this questionnaire and reported the assessment of its face, content, and construction validity. We also examine the questionnaire reliability, responsiveness, and acceptability in Greece. The purpose of this study was to develop and assess the validity and reliability of two physical activity surveys. These surveys were selected because they are both short and relatively easy to administer, making them ideal for use as brief physical activity screening tools. These questionnaires have not been previously validated for use in a Greek speaking population.

Patras/Peloponnisos is a region in northwest Greece. Theoretically, 100% of the population is covered by the National Health System (NHS). The NHS affords universal coverage for the population is almost fully funded by taxes and is predominantly within the public sector. Provision of all health services, other than pharmaceutical, is free of charge at the point of delivery. Patras as well as the Greek Primary Care model follows the guidelines laid down in the Alma-Ata Declaration [19]. GP data (name, healthcare centre and postal address) are included in a public domain web site (EOPPY).

Step I: Item Generation and Questionnaire Design

The itemswere created based on the results of our previous literature review.We comprehensively reviewed the literature to identify published SRH and SM questionnairesin the general population. We applied the following search syntaxin Medline from inception until September [20, 21]: ("Self-rated health" OR "Self-medication") AND ("Surveys and Questionnaires") AND ("Attitude to Health" OR"Health Knowledge), (Attitudes, Practice"OR "Knowledge" OR beliefs OR perception OR "HealthBehavior" OR "Awareness") AND (“misuse or overuse or use or abuse”). We also searched conferencepapers in the Conference Proceedings Citation Index-Science (CPCI-S) as well as the reference list of relevantstudies. In addition, we reviewed reports about the determinants of self-rated health or self-medication as wellas aspects of the misuse of medicine. Potentially important candidate knowledge and attitude items were identified, too. By way of instructions on how to completethe questionnaire, participants were provided with a set that had alreadybeen used for similar questionnaires, had been drawn up and used by theresearch group, and had been well understood by populations compared to the oneused for this study20.The questionnaire and a cover letter explaining the study and ensuring the confidentiality of the answers were sent to all participants totheir mail address.

Step II: Content Validation

The flow diagram of the full validation procedure is summarized in Fig.1.The content validity of the questionnaire is an assessment of the adequateness and the comprehensiveness of the items of the questionnaire to measure the target construct and is routinely performed by a multidisciplinary panel of experts [21, 22]with proven experience in this field. During this stage, the inclusion of a section to assess theusefulness of certain sources of knowledge in the subject was considered, and some questions in the general sectionwere reformulated. The overall linguistic aspects of the questionnairewere evaluated duringthis phase. An expert in community medicine was tasked with evaluating and correcting aspectsrelating to the grammar, syntax, and organization of each part of thequestionnaire, to make it more easily comprehensibleand to prevent itfrom giving rise to any misunderstandings. Our panel of experts consisted of 16members with experience in questionnaire design and who were specialized in at least one of the following fields: oncology, respiratory medicine, public health, epidemiology, nursing, pathology. At first, we provided the panel of experts with the 85-item questionnaire and collected their feedback about the items’ clarity, relevance, and ease of understanding as well as the comprehensiveness of the questionnaire. The experts received the Greek version of the questionnaire, as Greek is their native language. They were also requested to identify deficient areas, suggest any additional relevant items and/or possible answers, and make suggestions for improvement. The second type of empirical analysis was CVR, which measures the essentiality of an item[26]. CVR varies between 1 and −1, and a higher score indicates greater agreement among panel members. The formula for the CVR is CVR = (Ne – N/2)/ (N/2), where Ne is the number of panelists indicatingan item as “essential” and N is the total number of panelists. Although I-CVI is extensively used to estimate contentvalidity, [23]. suggested that due to chance agreement this index does not consider the possibility of inflated values, and instead suggested a kappa statistic to be calculated inaddition to CVI be calculated. Kappa provides thedegree of agreement beyond chance, as is calculatedusing the following formula: K= (I-CVI – Pc)/ (1- Pc), where Pc = [N!/A!(N-A)!]*0.5N. In this formula Pc= the probability of chance agreement; N = number ofexperts; and A = number of experts that agree the itemis relevant. Kappa values above 0.74 are considered excellent, between 0.60 to 0.74 good and 0.40 to 0.59 fair. To consider the agreement by chance between experts we estimated a modified Kappa (k*) 27. The probability of agreement by chance (Pc) was calculated using the formula: Pc = (N/A (N −A) Χ 0.5N, where N is the total number of experts, A is the number of experts that rated the item by ≥ 3. k* is calculated as follows:k*= (I-CVI − Pc)/(1 − Pc)[24].

Step III: Face Validity and Pilot Testing

Face validity entails an examination of the questionnaire by the research group in order to determine whether the included items are appropriate and relevant and whether the questionnaire measures what it is intended to measure i.e. Knowledge, Attitudes and Practices towards SRH and SM[22].Therefore, after content validity, two members of the group (G.O and A.V) subjectively checked the face validity of the questionnaire by reviewing the clarity and the completeness of the questions to measure the target outcome. In order to pinpoint and clarify possible problems of comprehensionwith anypart of the questionnaire, a pilot test was conducted on 20adults, though not belonging to the study population but displaying verysimilar characteristics to it (e.g. region bordering on the designated studyarea). We asked the participants to provide feedback about the clarity and the understandingof the questions, the questionnaire design, the ease of answering and finally on the time taken to answer thequestionnaire.

Step IV: test–retest reliability analysis and Questionnaire overall reliability and accessibility

Reliability is concerned with measurement error, and it reflects the stability of the measurement process over time. Stability is routinely evaluated through test–retest procedure. It involves administering the same questionnaire to the same participants on two occasions, provided that the measured characteristic does not change during the testing period [27]. As knowledge and attitudes are characteristics that tend to be stable overtime, a test–retest study can be most useful when it comes to assessingquestionnaire reliability. We therefore re-sent the questionnaire to the first135participants who answered the cohort study.Since Knowledge and attitudes are considered stablecharacteristics over time, we examined the stability of these domains by conducting a test–retest reliabilityassessment in this sample of 135 adults. The participants were randomly recruited from the general adult population in Greece. Participants were unrelated to the health or medical fields. The questionnaireswere delivered to each participant twice, at an interval of 6weeks.People who participated in the analysis of the reliability did not know theywould receive a second questionnaire. This decreases the likelihood of recallbias (i.e. memorizing or recording the responses given the first time).The participantswere informed about the study objective, and they agreed to answer the questionnaire on both occasions. We assessed the reliability by calculating the Intraclass Correlation Coefficients (ICCs) with their 95% ConfidenceInterval (CI) relative to the average measure of the two-way mixed-effects model as recommended fortest–retest settings[24].Items with ICC ≥ 0.4 were considered reliable[25]. McDowell provides a guideline for interpreting ICCs:values above 0.75 indicate excellent inter-rater agreement; values from 0.6 to0.74 indicate good agreement; values from 0.4 to 0.59 indicate fair to moderateagreement; and values below 0.4 indicate poor agreement[29]. We also calculated Cronbach’s coefficient alpha to check the overall reliabilityof the questionnaire using data collected from the 135 adult individuals [30].A reliability index ≥ 0.6 is consideredacceptablefor135 individuals by calculating the response rate, i.e. the percentage of individuals who accepted answering thequestionnaire. We also inspected the item-response rate by computing the percentage of missing data for eachitem [31-37]. Postal-questionnaire acceptability is very important, to minimizeselection bias due to non-response. Questionnaire acceptability wasassessed in a cohort of 650 peopleconsideringresponserates and the percentage of missing data in the completed questionnaires.All statistical analyses were carried out using IBM SPSS [28].1and R statistics.

Ethics

Our study was approved by the ethics committee of the University of Patras (R00002, No. 2021). The study was conducted in compliance with the general requirementsof the ethics committee and with the General Data Protection Regulation (Regulation (EU) 2016/679and Organic Law 3/2018). Written informed consent form was obtained from the participants and the data wereanonymized before analysis.

Figure 1 shows the successive phases and the results of each step ofthe development of the questionnaire. Based on the results of the qualitative study,we generated a pool of 120mixed published and unpublished items (questions or statements) concerning3fields: knowledge, attitudes, or practices towards self-rated health and self-medication and COVID-19. After removing duplicated or very similar questions, we selected 75 items based on their relevancy to the topic, and tailored their wordings as needed. General demographic questions (age, sex, education) were also added.The final version of the questionnaire consists of 6 domains and 85 sectional questions. The 85 questions were categorized in 6 domains: 1) demographic (10 questions, e.g. person’s and family’s profile) 2) perceptions and attitudes for SRH (11-18 question)3) perceptions and attitudes for SM (19-32) 4) quality of life (questions 33-59), impact from the COVID-19 outbreak (questions 60-82) 6) and finally, impression from the interview (questions 83-85). The 85-item questionnaire was originally written in English and then forward and backward translated into theGreek language by bilingual researchers. The translated version of the questionnaire was then reviewed by a native language specialist. The panel of experts redrafted the knowledge and attitudeitems relating to the use of antibiotics, eliminated 3 (because they repeated concepts that were included in other items and added other 3. The questionnaire was then modified by discarding any unnecessary items, rephrasing any ambiguous questions, and making the necessary changes in the format. The modified questionnaire was circulated among the experts for a second evaluation. The Item-Content Validity Index (I-CVI) estimations for the total of items that were retainedafter the initial evaluation of the panelists ranged between 0.50 and 1.00, indicating that these items were consideredclear, comprehensible, and relevant to the questionnaire. From the 85 questions, 45 questions from domains 2, 3 and 4 were used in the next validation steps. Twenty-seven items had an ICVI = 1.00, eight a score of 0.88, four ascore of 0.75, four a score of 0.63, andtwo a score of 0.50. All items were considered relevant. The CVR was generated for each item. Items that weremarked not essential had a CVR < 0.7.Twoitems out of 45 (17.7%) were marked as not essential.Non-essentialitems can be eliminated, but in this case were not.Twenty-six items had a CVR of 1.00, eighta score of 0.9, four ascore of 0.8, four a score of 0.7, and twoa score of 0.5. Theaverage CVR value was 0.9.For all items, modified Kappa (k*) values wereexcellent (> 0.8), revealing that the agreement between experts was not due to chance. At this stage, thesixteen experts, evaluated the content of the questionnaire by rating theclarity and the relevance of each otem.It was calculated on a 3-point LikertScale (1 and 3 represented the lowest and the highest levels of clarity and relevance, respectively).The average clarity scores for individual items ranged from2.7 to 3.00.Twenty-six items had an average clarity score of 3.00, eleven ascore of 2.85, five a score of 2.75, three a score of 2.65. The overall clarity score of the questionnaire was 2.9.

Figure 1: Flow diagram describing the steps followed to develop and validate the SRH and SM questionnaire.

Table1: The characteristics of the participants.

Step III: Face Validity and Pilot Study

The two members of the group (I.C and A.V) checked the face validity of the questionnaire by reviewing the clarity and the completeness of the questions to measure the target outcome. None of the experts participating in this procedure made any comments about the comprehensibility of the questionnaire. All the 20 adults who participated in the pilot testing answered the questionnairein its totality. Two participants reported that they misunderstood the scales 0 and 10 as being the lowest and highestlevels of agreement. We, therefore, added an additional indication using arrows to help the participants rememberthe direction of agreement. We also provided an answered example (“practicing sports benefits health”)to facilitate the understanding of the 0–10 Likert Scale concept. One participant declared that it was not clearwhether the Knowledge and Attitudes items should be answered in case the respondent did not use antibiotics. Therefore, we added this statement “Please evaluate the below statements REGARDLESS of whether you areusing antibiotics OR NOT”. For the question, “The last time you had to take antibiotics, did you complete thecourse of treatment?”, the participants suggested adding one additional answer (“still using them”). The questionnairetook 20 min to be completed and the participants showed satisfaction about the length of the questionnaire length.

Step IV: Test–Retest Reliability Analysis and Questionnaire Overall Reliability and Accessibility

The SRH and SM validation questionnaire was administered to 150 participants and 135 questionnaires (135/150, 90%)test-retest questionnaires were completed. General demographic characteristics of participants are summarized in Table 1. The age of the participants was 25 up to 40. Eighty-one (60%) participants were women.The Intraclass Correlation Coefficient (ICC) assessmentshowed that the reliability was good in average measures (ICC=0.779). Cronbach’s coefficient alpha was used to check the overall reliabilityof the questionnaire using the data collected from the 135 adult individuals30. Our group had already conducted a cohort study using the SRH and SM questionnaire. The questionnaire was sent to the study population (650). After the fifth mailing, 68.0% answered the questionnaire. In the 85total items, the percentage of missing data ranged from 0.03% (2/650) to 2. 1% (14/650).

To-date none of the studies that measured the association of Knowledge and Attitudes with SRH and SM in the general population applied a fully validated questionnaire. Therefore, to fill this gap, we designed and validated the properties of a questionnaire about SRH and SMamongthe adult general population. The content and the scale validity indices confirmed the content validity of our questionnaire.Moreover, the test–retest reliability and the confirmatory factor analysis proved the reliability and the construct validity of the questionnaire. Furthermore, the questionnaire was accepted by the general population as reflected by the high response rate and the low percentage of unanswered questions. The availability of a reliable and construct-valid instrument is fundamental for epidemiological studies that aim to measure associations between Knowledge, Attitudes, and Practices towards SRH and SM, as the use of a non-validated questionnaire may induce measurement error in the exposure and the outcome.In particular, the availability of a validated SRH and SM questionnaire is crucial for the design of interventions aimed at improving the Knowledge and modifying Attitudes and Practices towards health promotion and prevention.Knowledge and Attitudes are considered stable variables that are not likely to be changed rapidly. The test–retest reliability of these questions showed that they generate reproducible results. Our instrument encompasses a multidimensional construct.The Knowledge factor correlated significantly with attitudes towards the personal SRH and SM (Attitude-Personal). Such findings were expected, as personal attitudes toward SRH and SM are in part driven by other factors such as patients’ knowledge, education, profession, and age. On the other side, Knowledge was not correlated with attitudes towards health-care provider (Attitude-Healthcare provider), revealing that the items designed to measure the participants’ Knowledge about SRH and SM are not related to those intended to explore the relation between patients and healthcare providers. This confirms that the questions included in each of these two factors are specific and measure unique and unrelated dimensions.Questionnaires about SRH and SM should not be limited to knowledge and personalattitudes only, but should also examine attitudes towards healthcare-providers as well as the quality of life and of course COVID-19 experience. In addition, a negative and weak correlation existed between attitude-Personal and attitude-Healthcare provider, suggesting that individuals who tend to have higher agreement with Attitude-Healthcare provider statements are also more prone to disagree or to agree to a lesser extent with the items included in attitude-Personal factor. This demonstrates that trust and communication between patients and their healthcare providers have a substantial impact on the patients’ attitudes towardsSRH and SM. In another research, we have tested the same knowledge in students (unpublished data). Testing these associations in other populations would further validate our findings.In general; a trustworthy assessment relies on the memory of the participants. Therefore, to decrease the risk of recall bias we have included a time limit in the design of the “Practice” questions by asking about the use in the past six months.

The questionnaire encompasses 85 questions that are elaborated to determine any aspect of misuse. Moreover, the provided choices of answers were based on an extensive literature review to include any possible answers, and therefore avoid leaving questions unanswered.The questionnaire was designed to measure the participants’ Knowledge and Attitudes regardless of their SRH and SM in the last 6 months. Therefore, our questionnaire could prove useful in obtaining data both from both users and non-users of medicine, which represent a crucial issue in epidemiologic studies involving Knowledge, Attitudes and Practices in SRH and SM.

The high acceptability of the questionnaire by the general population reflects the feasibility of its application in general population settings. In fact, the time taken to answer our questionnaire was within the ideal range (30 min), which therefore aided in increasing the response rate[33].Questionnaires with a long list of questions negatively influence the participation rate and the quality of the data[34]. Another factor that could have enhanced the response rate is the fact that the questionnaire wasissued bya research and academic institution.The ease of understanding the questions enhanced the response rate as well.

Our study has an important limitation. In epidemiologic studies, an important step in the validation processinvolves comparing the results obtained from the questionnaire, which is validated to a superior method, deemed as “gold standard”. However, to the best of our knowledge, to date, a gold standard to assess SRH and SMdoes not exist and therefore, our instrument could not be compared against any previousreference method. Due to this limitation, we consider that the present questionnaire is reliable and has validity, but future research is needed to provide a gold standard for SRH and SM questionnaire.Another limitation of our validation study is that the construct was validated in the Greek population only,therefore our questionnaire needs to be further tested in different settings and populations.

This study presents a step forward towards the validation of a knowledge, attitude, and practice questionnaireabout the Self rated health and self-medication. The SRH and SMquestionnaire is a novel instrument developed to assess the factors associated with SRH and SM, and thus to design interventions to improve the health promotion and prevention.Moreover, taking into consideration the inconsistent reporting of validationmethodologies across studies and the abuse of the term validation, as well as the exhaustive review of themethodology carried out in the current manuscript, we believe that this study would help validating SRH and SMmedical, health as well as social studies. Future work, involving the application ofthe questionnaire to other health professionals and the public,could be envisaged as a preliminary step in designing interventions to improve SRH and SM.

1. Bjorner JB, Kristensen TS, Orth-Gomér K. Self-rated health. A useful concept in research, prevention and clinical medicine. Stockholm: Swedish Council for Planning and Coordination of Research, 1996.
2. Kunst AE, Geurts JJM, van den Berg J. International variations in socioeconomic inequalities in self-reported health. J Epidemiol Community Health 1995;49:117-23.
3. Sundquist J, Johansson SE. Self-reported poor health and low educational level predictors for mortality: a population based follow up study of 39 156 people in Sweden. J Epidemiol Community Health 1997;51: 35-40.
4. McGee DL, Liao Y, Cao G. Self-reported health status and mortality in a multiethnic US cohort. Am J Epidemiol 1999;149:41-46.
5. Kaplan GA, Camacho T. Perceived health and mortality: a nine-year follow-up of the human population laboratory cohort. Am J Epidemiol.1983;117: 293-304.
6. Mossey JM, Shapiro E. Self-rated health: a predictor of mortality among the elderly. Am J Public Health 1982;72:800-8.
7. Idler EI, Benyamini Y. Self-rated health and mortality: a review of twenty-seven community studies. J Health Soc Behaviour. 1997; 38:21-37.
8. Murray CJL, Chen L. Understanding morbidity change. In:Chen L, Kleinman A, Ware NC, eds. Health and social change in international perspective. Boston: Harvard University Press, 1994:87-114.
9. Appels A, Bosma H, Grabauskas V. Self-rated health and mortality in a Lituanian and a Dutch population. Soc Sci Med 1996;42:681-689.
10. Moller L, Kristensen TS, Hollnagel H. Self rated health as a predictor of coronary heart disease in Copenhagen, Denmark. J Epidemiol Community Health 1996;50:423-428.
11. Miilunpalo S, Vuori I, Oja P. Self-rated health status as a health measure: the predictive value of self-reported health status on the use of physician services and on mortality in the working-age population. J Clin Epidemiology 1997;50:517-528.
12. Kaplan GA, Goldberg DE, Everson SA. Perceived health status and morbidity and mortality: evidence from the Kuopio Ischaemic Heart Disease Risk Factor Study. Int J Epidemiol 1996; 25: 259-625
13. World Health Organization. Responsible self-care and self-medication; a worldwide review of consumers’ survey; the world self-medication industry. 2010.
14. Lam CLCM, Munro C, Lauder IJ. Self-medication among Hong Kong Chinese. Soc Sci Med. 1994;39:1641-1647.
15. Sanghani SZH, Patel VJ. Self-medication: prevalence and pattern in urban community. J Pharmacovigilance Drug Safety. 2008;5:95-98.
16. Phalke VDPD, Durgawale PM. Self-medication practices in rural Maharashtra. Indian J Commun Med. 2006;31:34-35.
17. Ayalew MB. Self-medication practice in Ethiopia: a systematic review. Patient preference and adherence. 2017;11:401-413
18. Li LJWP. Li-Jian Li, Pei-Shan Wang, Self-medication with antibiotics: a possible cause of bacterial resistance. Med Hypotheses. 2005;65:1000-1001.
19. World Health Organization. Declaration of Alma-Ata. World Health Organization, Regional Office for Europe, 1978. http://www1.paho.org/English/DD/PIN/alma-ata_declaration.htm.
20. Juan M Vazquez-Lagoa, Paula Lopez-Vazqueza,b, Ana Lo´ pez-Dura´nc, Margarita Taracido-Trunka,b and Adolfo Figueirasa,b,* Attitudes of primary care physicians to the prescribing of antibiotics and antimicrobial resistance: a qualitative study from Spain, Family Practice 2012; 29:352-360 .
21. McDowell I. Measuring Health. A Guide to Rating Scales and Questionnaires. 31-46.
22. Lynn MR. Determination and quantification of content validity. Nurs.Res. 1986; 35: 382-385
23. Polit DF, Beck CT. The content validity index: Are you sure you know what’s being reported? Critique and recommendations. Res. Nurs. Health. 2006; 29:489-497.
24. Polit DF, Beck CT, Owen SV. Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Res. Nurs. Health. 2007; 30: 459-467.
25. Zamanzadeh V, Ghahramanian A, Rassouli M, Abbaszadeh A, Alavi- H. Design and implementation content validity Study: development of an instrument for measuring patient-centered communication. J Caring Sci. 2015;4: 165-78.
26. Shi J, Mo X, Sun Z. Content validity index in scale development. Zhong Na Da XueXue Bao Yi Xue Ban. 2012;37:152-155.
27. DeVellis R.F. Scale Development. Theory and Applications. Applied Social Research Methods Series (ed. Seawell, M.H.) 49-58.
28. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986; 1: 307-310.
29. Shrout PE, Fleiss JL. Intraclass correlations: Uses in assessing rater reliability. Psychol. Bull. 1979; 86: 420-428.
30. Taber KS. The use of Cronbach’s alpha when developing and reporting research instruments in science education. Res. Sci. Educ. 2018; 48: 1273-1296.
31. Ursachi G, Horodnic IA, Zait A. How reliable are measurement scales? External factors with indirect influence on reliability estimators. Proc. Econ. Finance. 2015; 20: 679-686.
32. Galesic M, Bosnjak M. Effects of questionnaire length on participation and indicators of response quality in a web survey.Public Opin. Q. 2009; 73:349-360.
33. Egleston BL, Miller SM, Meropol N. J. The impact of misclassification due to survey response fatigue on estimation and identifiability of treatment effects. Stat. Med. 2011; 30: 3560-3572.
34. Burkell JA. The dilemma of survey nonresponse. Libr. Inf. Sci. Res. 2003; 25:239-263.
35. Rosenberg AR. What’s missing in missing data? Omissions in survey responses among parents of children with advanced cancer. J. Palliat. Med. 2014; 7:953-956.
36. Revilla M, Ochoa C. Ideal and maximum length for a web survey. J. Mark. Res. 2017; 59:557-565
37. Burchell, B. & Marsh, C. The effect of questionnaire length on survey response. Quant. 1992; 26:233-244.