Other Titles
Selection and development of tools [Session]
Abstract
Introduction: The purpose of this presentation is to assist nurse faculty in continuing development of their personal, research skills in selecting and evaluating the psychometric qualities of an instrument for use in their own research program. Many nurse researchers need more confidence with selecting instruments that are psychometrically sound for their education research. In an effort to improve nursing education research, this topic will help inform and improve personal research skills in selecting an instrument and evaluating the psychometric qualities. It is important to do credible and rigorous research. Thus, leading to credible research that can be used as a catalyst to transform our practice. The nurse's ability to select instruments, and judge the quality of instruments, is a necessary skill for building a strong research program. Nurse researchers can only be confident, in their findings, if their selected instruments yield data that are of credible quality. Their research acumen must include the nurse researche's ability to select appropriate instruments and delineate the quality of their instruments, be it a developed instrument or a new instrument. The data obtained, from the instrument, should be psychometrically sound to ensure useful and meaning research results. Therefore, selecting an instrument and evaluating the quality is paramount in nursing education research. This presentation provides an example of an established, validated instrument and the process of what's important in selecting an instrument. The process includes understanding the construct under study, the instrument development, testing, and usage. The instrument example is known as the General Well-Being Schedule (GWB) (Dupuy, 1978). This presentation will delineate a review of the instrument development, use, and the psychometric properties. A description of the GWB instrument, along with what it measures, will be also be provided. The known, published, psychometric properties will be reported by providing several reports, in articles, reviewing the validity and reliability. The GWB has been used, for several decades, in a large body of research and many references will be provided. Conclusions will be drawn and will include nursing research implications concerning discussions to enhance and document instrument selection/quality through acceptable research methods.
Description of the GWB Instrument: The GWB is a self-administered questionnaire and widely used instrument, which was developed for the U.S. Health and Nutrition Examination Survey (HANES I) and was conducted from 1971 through October 1975 as part of the program of the National Center for Health Statistics (Fazio, 1977; Monk, 1981). The HANES I study was a nationwide survey used as part of a national health examination of 6,931 individuals aged 25-74 years (Fazio, 1977). The first version of the GWB contained 68 items, 18 of which were used in the HANES study and continue to be used. The GWB is a highly structured instrument for assessing self-representations of subjective well-being and distress (Fazio, 1977). The GWB represents several psychological distress constructs and is designed to obtain responses to questions concerning the presence, severity, or frequency of common clinical symptoms that are considered significant in making assessments about general well-being (Wan & Livieratos, 1978). The content emphasizes the most prevalent symptoms of psychological distress, anxiety and depression (Veit & Ware, 1983). The items included in the GWB measure were developed based on a straightforward two-process model of effective psychotherapy from a problem-solving framework, one in which patients acquire a positive rapport with a therapist though therapeutic exercises (Andrews & Robinson, 1991).
Instrument Quality - the Psychometric Properties: As nurse researchers, we know that it is important to achieve the highest quality of measurement possible in our research. Reliability and validity are the two important aspects of the quality of quantitative research measures (Bekhet & Zausaniewski, 2014). Reliability: The reliability of instruments/measures is concerned with consistency. You can think of reliability in other terms, like repeatability, reproducibility, stability, consistency, predictability, agreement, and homogeneity. So, reliability is always a concern. If you develop a new instrument/measure, reliability is a primary focus along with validity. If you are using a standard, well-tested instrument/measure, your main concern is whether is continues to evidence reliability in your study. In this case, you want to know if the instrument/measure provides consistent results when used with your sample. One researcher has said that "no study, no matter how sophisticated the design, analyses, theoretical background, or other components, can rise above the threat of non-reliability of the measures used in the study." As nurse researchers are aware, one of the most important characteristics of an instrument is its reliability.
Validity: Validity is simply that you are measuring what is claimed to be measured. The validity of an instrument is always important and critical. More specifically, construct validity involves the measurement of constructs that theoretically should be related to each other are, in fact, observed to be related to each other (i.e., assessment instruments should be able to show a correspondence or convergence between similar constructs). Correlations between theoretically similar measures should be high while correlations between theoretically dissimilar measures should be low. Fazio (1977) found that the GWB total score correlated 0.47 with an interviewer's rating of depression, 0.66 with the Zung Depression Scale, and 0.78 with the Personal Feelings Inventory-Depression. The average correlation between the GWB and the six independent depression scales was 0.69.
Factor Structure and Analysis: Previous factor analysis research was found to be inconsistent and has not substantiated the existence of Dupuy's (1978) six hypothesized dimensions. For the purpose of examining the dimensionality of the GWB items, Wan and Livieratos (1978) used a principal components analysis. The purpose of the study was to determine the feasibility and suitability of using the GWB as a general health measure with special attention given to age differences in ratings of well-being. A sample of 6931 of non-institutionalized adults between 25 and 74 years of age were given the GWB questionnaire. Three common factors were found: Depressive mood, health concern, and life satisfaction and emotional stability. The three common factors accounted for 51% of the total variance in the GWB items and were moderately correlated with the total GWB score (Wan & Livieratos, 1978). However, Veit and Ware (1983) examined the factor structure of a 22-item revision of the GWB. The hypothesis of 6 underlying factors was supported.
Conclusions and Recommendations about this Instrument: The GWB is well designed, easy to comprehend, and its content covers the appropriate constructs (Egan, Davis, Dubouloz, Kessler & Kubina, 2014; Hildebrandt, 2006; Lorenz & Anders, 2014; Yuen, Mueller, Mayor & Azuero, 2011). The GWB has emerged as a useful instrument in measuring depression. The GWB has been used in a variety of research and applied settings. The GWB has potential research value because of the strong psychometric properties (Egan, Davis, Dubouloz, Kessler & Kubina, 2014; Kobau, Sniezek, Zack, Lucas & Burns, 2010; Fazio, 1977). Because of the large database for the GWB, from the NCHS National Health Examination Survey of Adults from 1971-1975, sample comparisons can be made by researchers in numerous settings (Fazio, 1977). The validity and reliability of this instrument has been established through extensive testing and has been found to be an extremely useful instrument in research.
Nurse Researcher Implications: Measurement, through the use of reliable and valid instruments, is necessary for nurse researchers due to the objectivity and production of credible research. It is important to remember that it is best to never switch to a less valid instrument because it is more reliable. Instead, you should improve the reliability of the more valid instrument as much as possible.
Improving Reliability: Be sure that the raters/examiners that collect the data are responsible, competent, and trained to use the instrument. Supervise the collections of the data and periodically check the raters/examiners, Standardize the conditions while testing/using the instrument and minimize distractions. Ensure that directions and the wording of the questions leave nothing to interpretation. Standardize the data collection.
Validity: Simply stated: when you select an instrument to use in your study you must be sure that its validity has been thoroughly evaluated. The selection of instruments requires the consistent use of available research rules and the psychometric assessment discussed in this presentation. Novice nurse researchers must be encouraged to learn to use previously tested instruments to enhance their research and at some future point in time, develop a new instrument. Also, it is important for nurse researchers, as nursing faculty, to take appropriate steps to enhance and speak to instrument quality, in their research reporting, by summarizing the psychometric qualities using these appropriate areas. Thus, the use of the GWB has been an example of instrument selection, development and its use in research. These important steps can be used as an example of the use of an instrument and what is important in selection of instruments in nursing research. Nurse researchers need to continue to look for resources that will improve their research skills and selection of instruments for their continued research endeavors. Onward . . . with building credible, nursing education research programs.
Sigma Membership
Beta Chi
Type
Presentation
Format Type
Text-based Document
Research Approach
N/A
Keywords:
Instrument, General Well-Being Schedule, Nurse Faculty Researcher
Recommended Citation
Landreneau, Kandace, "Selecting an instrument: Important psychometric properties using the general well-being schedule as the example" (2016). NERC (Nursing Education Research Conference). 82.
https://www.sigmarepository.org/nerc/2016/presentations_2016/82
Conference Name
Nursing Education Research Conference 2016
Conference Host
Sigma Theta Tau International,National League for Nursing
Conference Location
Washington, DC, USA
Conference Year
2016
Rights Holder
All rights reserved by the author(s) and/or publisher(s) listed in this item record unless relinquished in whole or part by a rights notation or a Creative Commons License present in this item record.
All permission requests should be directed accordingly and not to the Sigma Repository.
All submitting authors or publishers have affirmed that when using material in their work where they do not own copyright, they have obtained permission of the copyright holder prior to submission and the rights holder has been acknowledged as necessary.
Review Type
Abstract Review Only: Reviewed by Event Host
Acquisition
Proxy-submission
Selecting an instrument: Important psychometric properties using the general well-being schedule as the example
Washington, DC, USA
Introduction: The purpose of this presentation is to assist nurse faculty in continuing development of their personal, research skills in selecting and evaluating the psychometric qualities of an instrument for use in their own research program. Many nurse researchers need more confidence with selecting instruments that are psychometrically sound for their education research. In an effort to improve nursing education research, this topic will help inform and improve personal research skills in selecting an instrument and evaluating the psychometric qualities. It is important to do credible and rigorous research. Thus, leading to credible research that can be used as a catalyst to transform our practice. The nurse's ability to select instruments, and judge the quality of instruments, is a necessary skill for building a strong research program. Nurse researchers can only be confident, in their findings, if their selected instruments yield data that are of credible quality. Their research acumen must include the nurse researche's ability to select appropriate instruments and delineate the quality of their instruments, be it a developed instrument or a new instrument. The data obtained, from the instrument, should be psychometrically sound to ensure useful and meaning research results. Therefore, selecting an instrument and evaluating the quality is paramount in nursing education research. This presentation provides an example of an established, validated instrument and the process of what's important in selecting an instrument. The process includes understanding the construct under study, the instrument development, testing, and usage. The instrument example is known as the General Well-Being Schedule (GWB) (Dupuy, 1978). This presentation will delineate a review of the instrument development, use, and the psychometric properties. A description of the GWB instrument, along with what it measures, will be also be provided. The known, published, psychometric properties will be reported by providing several reports, in articles, reviewing the validity and reliability. The GWB has been used, for several decades, in a large body of research and many references will be provided. Conclusions will be drawn and will include nursing research implications concerning discussions to enhance and document instrument selection/quality through acceptable research methods.
Description of the GWB Instrument: The GWB is a self-administered questionnaire and widely used instrument, which was developed for the U.S. Health and Nutrition Examination Survey (HANES I) and was conducted from 1971 through October 1975 as part of the program of the National Center for Health Statistics (Fazio, 1977; Monk, 1981). The HANES I study was a nationwide survey used as part of a national health examination of 6,931 individuals aged 25-74 years (Fazio, 1977). The first version of the GWB contained 68 items, 18 of which were used in the HANES study and continue to be used. The GWB is a highly structured instrument for assessing self-representations of subjective well-being and distress (Fazio, 1977). The GWB represents several psychological distress constructs and is designed to obtain responses to questions concerning the presence, severity, or frequency of common clinical symptoms that are considered significant in making assessments about general well-being (Wan & Livieratos, 1978). The content emphasizes the most prevalent symptoms of psychological distress, anxiety and depression (Veit & Ware, 1983). The items included in the GWB measure were developed based on a straightforward two-process model of effective psychotherapy from a problem-solving framework, one in which patients acquire a positive rapport with a therapist though therapeutic exercises (Andrews & Robinson, 1991).
Instrument Quality - the Psychometric Properties: As nurse researchers, we know that it is important to achieve the highest quality of measurement possible in our research. Reliability and validity are the two important aspects of the quality of quantitative research measures (Bekhet & Zausaniewski, 2014). Reliability: The reliability of instruments/measures is concerned with consistency. You can think of reliability in other terms, like repeatability, reproducibility, stability, consistency, predictability, agreement, and homogeneity. So, reliability is always a concern. If you develop a new instrument/measure, reliability is a primary focus along with validity. If you are using a standard, well-tested instrument/measure, your main concern is whether is continues to evidence reliability in your study. In this case, you want to know if the instrument/measure provides consistent results when used with your sample. One researcher has said that "no study, no matter how sophisticated the design, analyses, theoretical background, or other components, can rise above the threat of non-reliability of the measures used in the study." As nurse researchers are aware, one of the most important characteristics of an instrument is its reliability.
Validity: Validity is simply that you are measuring what is claimed to be measured. The validity of an instrument is always important and critical. More specifically, construct validity involves the measurement of constructs that theoretically should be related to each other are, in fact, observed to be related to each other (i.e., assessment instruments should be able to show a correspondence or convergence between similar constructs). Correlations between theoretically similar measures should be high while correlations between theoretically dissimilar measures should be low. Fazio (1977) found that the GWB total score correlated 0.47 with an interviewer's rating of depression, 0.66 with the Zung Depression Scale, and 0.78 with the Personal Feelings Inventory-Depression. The average correlation between the GWB and the six independent depression scales was 0.69.
Factor Structure and Analysis: Previous factor analysis research was found to be inconsistent and has not substantiated the existence of Dupuy's (1978) six hypothesized dimensions. For the purpose of examining the dimensionality of the GWB items, Wan and Livieratos (1978) used a principal components analysis. The purpose of the study was to determine the feasibility and suitability of using the GWB as a general health measure with special attention given to age differences in ratings of well-being. A sample of 6931 of non-institutionalized adults between 25 and 74 years of age were given the GWB questionnaire. Three common factors were found: Depressive mood, health concern, and life satisfaction and emotional stability. The three common factors accounted for 51% of the total variance in the GWB items and were moderately correlated with the total GWB score (Wan & Livieratos, 1978). However, Veit and Ware (1983) examined the factor structure of a 22-item revision of the GWB. The hypothesis of 6 underlying factors was supported.
Conclusions and Recommendations about this Instrument: The GWB is well designed, easy to comprehend, and its content covers the appropriate constructs (Egan, Davis, Dubouloz, Kessler & Kubina, 2014; Hildebrandt, 2006; Lorenz & Anders, 2014; Yuen, Mueller, Mayor & Azuero, 2011). The GWB has emerged as a useful instrument in measuring depression. The GWB has been used in a variety of research and applied settings. The GWB has potential research value because of the strong psychometric properties (Egan, Davis, Dubouloz, Kessler & Kubina, 2014; Kobau, Sniezek, Zack, Lucas & Burns, 2010; Fazio, 1977). Because of the large database for the GWB, from the NCHS National Health Examination Survey of Adults from 1971-1975, sample comparisons can be made by researchers in numerous settings (Fazio, 1977). The validity and reliability of this instrument has been established through extensive testing and has been found to be an extremely useful instrument in research.
Nurse Researcher Implications: Measurement, through the use of reliable and valid instruments, is necessary for nurse researchers due to the objectivity and production of credible research. It is important to remember that it is best to never switch to a less valid instrument because it is more reliable. Instead, you should improve the reliability of the more valid instrument as much as possible.
Improving Reliability: Be sure that the raters/examiners that collect the data are responsible, competent, and trained to use the instrument. Supervise the collections of the data and periodically check the raters/examiners, Standardize the conditions while testing/using the instrument and minimize distractions. Ensure that directions and the wording of the questions leave nothing to interpretation. Standardize the data collection.
Validity: Simply stated: when you select an instrument to use in your study you must be sure that its validity has been thoroughly evaluated. The selection of instruments requires the consistent use of available research rules and the psychometric assessment discussed in this presentation. Novice nurse researchers must be encouraged to learn to use previously tested instruments to enhance their research and at some future point in time, develop a new instrument. Also, it is important for nurse researchers, as nursing faculty, to take appropriate steps to enhance and speak to instrument quality, in their research reporting, by summarizing the psychometric qualities using these appropriate areas. Thus, the use of the GWB has been an example of instrument selection, development and its use in research. These important steps can be used as an example of the use of an instrument and what is important in selection of instruments in nursing research. Nurse researchers need to continue to look for resources that will improve their research skills and selection of instruments for their continued research endeavors. Onward . . . with building credible, nursing education research programs.