9. Module: SCALE DEVELOPMENT

Preface

The development of psychological scales is essential for assessing complex constructs in research and practice. This module provides a detailed guide for creating and validating scales, emphasizing theoretical foundations, item development, and psychometric evaluation. By mastering these principles, participants will be equipped to design effective tools that ensure reliability, validity, and applicability across diverse domains.

Part 1: Foundations of Psychological Scale Development

This section introduces the theoretical and practical basis for psychological scale development:

• Introduction to Psychological Scales: Definition, significance, and applications of scales in psychological research and assessment.
• Types of Scales: Overview of Likert, Thurstone, Guttman, and semantic differential scales, among others.
• Construct Definition: Importance of selecting and defining psychological constructs clearly and precisely.

Part 2: Methods for Developing and Refining Scale Items

This part delves into the process of creating and refining scale items:

• Generating Items: Techniques for drafting items that align with the defined construct, avoiding double-barreled questions, and balancing positively and negatively worded items.
• Expert Review and Pilot Testing: Engaging subject-matter experts for feedback and conducting pilot tests to refine items and enhance clarity.
• Item Wording, Formatting, and Response Options: Guidelines for creating concise, comprehensible items with consistent formatting and appropriate response scales.

Part 3: Evaluating Scale Validity and Reliability

This segment addresses the critical steps in ensuring scale quality:

• Assessing Validity: Types of validity (content, construct, and criterion) and methods for validation using statistical and theoretical approaches.
• Measuring Reliability: Techniques such as Cronbach's alpha, test-retest reliability, and internal consistency.
• Factor Analysis: Exploratory and confirmatory factor analysis for evaluating the structural validity of the scale.

Part 4: Applications and Emerging Trends in Scale Development

The final section explores practical uses and future directions:

• Practical Applications: Case studies and examples illustrating scale use in education, clinical psychology, and organizational settings.
• Ethical Considerations: Discussing fairness, accessibility, and implications of using scales across diverse populations.
• Emerging Trends: Introduction to computerized adaptive testing, online survey platforms, and integrating AI in scale development.

Conclusion

By completing this module, participants will gain comprehensive skills to design, validate, and apply psychological scales effectively. These tools will support robust data collection and decision-making in research and practice, fostering advancements in understanding complex psychological constructs.

Introduction

Explanation of the significance of psychological scale development in research and assessment. Highlighting the relevance of competence-oriented methodologies in scale development. Outline of the training material's structure and objectives.

Module 1: Understanding Psychological Scales

Definition of psychological scales and their role in psychological research. Distinction between various types of psychological scales, including Likert scales, Thurstone scales, and others. Discussion of key characteristics of psychological scales, including reliability and validity.

Module 2: Identifying the Construct of Interest

Guidance on selecting and defining the psychological construct to be measured. Emphasis on the importance of a well-defined and clear construct. Examples of psychological constructs in various domains, such as personality traits, attitudes, and mental health indicators.

Module 3: Generating and Refining Scale Items

Explanation of the process of generating potential scale items related to the chosen construct. Guidance on refining and enhancing these items through expert reviews and pilot testing. Guidelines for item wording, formatting, and response options.

Module 4: Content Validity and Item Selection

Introduction to content validity and its role in ensuring that scale items adequately represent the construct. Explanation of the process for assessing content validity, including expert judgment and content validity ratio (CVR). Discussion of item selection and strategies to reduce redundancy, resulting in a concise and focused scale.

Module 5: Validity Assessment

Exploration of different types of validity, including content, criterion, and construct validity. Discussion of techniques and methods for establishing each type of validity. Illustration of validity assessment through examples and case studies.

Module 6: Reliability Assessment

Definition of reliability within the context of psychological scale development. Explanation of methods to measure reliability, such as Cronbach's alpha and test-retest reliability. Emphasis on the importance of internal consistency and scale stability.

Module 7: Data Collection and Analysis

Detailed guidance on planning and conducting data collection for scale validation. Introduction to exploratory and confirmatory factor analysis for assessing construct validity. Discussion of item analysis and statistical techniques for refining the scale based on data.

Module 8: Pilot Testing and Feedback Integration

Explanation of the role of pilot testing in the scale development process. Description of the process to collect feedback from pilot participants and integrate it into scale refinement. Emphasis on the iterative nature of scale development and the value of feedback loops.

Module 9: Practical Applications and Case Studies

Presentation of real-world applications of psychological scale development in research and assessment. Sharing of case studies and examples in various psychological domains. Discussion of ethical considerations in scale development and usage.

Module 10: Future Directions and Emerging Trends

Insights into the evolving field of psychological scale development. Discussion of emerging trends and technologies, such as computerized adaptive testing and online survey platforms. Encouragement for participants to stay updated with the latest advancements in the field.

Conclusion

Summary of key takeaways from the training material. Encouragement for participants to apply their newfound knowledge in research or professional practice. Suggested additional resources, references, and further reading materials.

Introduction:

Explanation of the significance of psychological scale development in research and assessment. Highlighting the relevance of competence-oriented methodologies in scale development. Outline of the training material's structure and objectives.

Module 1: Understanding Psychological Scales

Definition of psychological scales and their role in psychological research. Distinction between various types of psychological scales, including Likert scales, Thurstone scales, and others. Discussion of key characteristics of psychological scales, including reliability and validity.

Module 2: Identifying the Construct of Interest

Guidance on selecting and defining the psychological construct to be measured. Emphasis on the importance of a well-defined and clear construct. Examples of psychological constructs in various domains, such as personality traits, attitudes, and mental health indicators.

Module 3: Generating and Refining Scale Items

Explanation of the process of generating potential scale items related to the chosen construct. Guidance on refining and enhancing these items through expert reviews and pilot testing. Guidelines for item wording, formatting, and response options.

Module 4: Content Validity and Item Selection

Introduction to content validity and its role in ensuring that scale items adequately represent the construct. Explanation of the process for assessing content validity, including expert judgment and content validity ratio (CVR). Discussion of item selection and strategies to reduce redundancy, resulting in a concise and focused scale.

Module 5: Validity Assessment

Exploration of different types of validity, including content, criterion, and construct validity. Discussion of techniques and methods for establishing each type of validity. Illustration of validity assessment through examples and case studies.

Module 6: Reliability Assessment

Definition of reliability within the context of psychological scale development. Explanation of methods to measure reliability, such as Cronbach's alpha and test-retest reliability. Emphasis on the importance of internal consistency and scale stability.

Module 7: Data Collection and Analysis

Detailed guidance on planning and conducting data collection for scale validation. Introduction to exploratory and confirmatory factor analysis for assessing construct validity. Discussion of item analysis and statistical techniques for refining the scale based on data.

Module 8: Pilot Testing and Feedback Integration

Explanation of the role of pilot testing in the scale development process. Description of the process to collect feedback from pilot participants and integrate it into scale refinement. Emphasis on the iterative nature of scale development and the value of feedback loops.

Module 9: Practical Applications and Case Studies

Presentation of real-world applications of psychological scale development in research and assessment. Sharing of case studies and examples in various psychological domains. Discussion of ethical considerations in scale development and usage.

Module 10: Future Directions and Emerging Trends

Insights into the evolving field of psychological scale development. Discussion of emerging trends and technologies, such as computerized adaptive testing and online survey platforms. Encouragement for participants to stay updated with the latest advancements in the field.

Conclusion

Summary of key takeaways from the training material. Encouragement for participants to apply their newfound knowledge in research or professional practice. Suggested additional resources, references, and further reading materials.

Assessment

Inclusion of quizzes or self-assessment exercises after each module to reinforce learning. Suggestion of a final project where participants can develop and validate a psychological scale of their own.

Assoc. Prof. PhD Dana RAD

Aurel Vlaicu University of Arad, Center of Research Development, and Innovation in Psychology

Ajzen, I., & Fishbein, M. (2005). The influence of attitudes on behavior. In D. Albarracín, B. T. Johnson, & M. P. Zanna (Eds.), The handbook of attitudes (pp. 173-221). Psychology Press.

American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. American Educational Research Association.

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). American Psychiatric Publishing.

American Psychological Association. (2020). Publication manual of the American Psychological Association (7th ed.). American Psychological Association.

Anastasi, A., & Urbina, S. (1997). Psychological testing (7th ed.). Prentice Hall.

Baron-Cohen, S., Richler, J., Bisarya, D., Gurunathan, N., & Wheelwright, S. (2003). The systemizing quotient: An investigation of adults with asperger syndrome or high-functioning autism, and normal sex differences. Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences, 358(1430), 361-374.

Baron-Cohen, S., Wheelwright, S., Skinner, R., Martin, J., & Clubley, E. (2001). The autism-spectrum quotient (AQ): Evidence from asperger syndrome/high-functioning autism, males and females, scientists and mathematicians. Journal of Autism and Developmental Disorders, 31(1), 5-17.

Beck, A. T., Steer, R. A., & Brown, G. K. (1996). Manual for the Beck Depression Inventory-II. Psychological Corporation.

Beck, A. T., Ward, C. H., Mendelson, M., Mock, J., & Erbaugh, J. (1961). An inventory for measuring depression. Archives of General Psychiatry, 4(6), 561-571.

Blei, D. M., & Lafferty, J. D. (2006). Dynamic topic models. Journal of Machine Learning Research, 3, 993-1022.

Boerma, T., AbouZahr, C., Evans, D., & Evans, T. (2014). Monitoring intervention coverage in the context of universal health coverage. PLoS medicine, 11(9), e1001728.

Brown, T. A. (2006). Confirmatory factor analysis for applied research. Guilford.

Brown, T. A., & Barlow, D. H. (2002). A proposal for a dimensional classification system based on the shared features of the DSM-IV anxiety and mood disorders: Implications for assessment and treatment. Psychological Assessment, 14(3), 288-299.

Cacioppo, J. T., Tassinary, L. G., & Berntson, G. G. (2016). Handbook of psychophysiology. Cambridge University Press.

Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56(2), 81-105.

Carmines, E. G., & Zeller, R. A. (1979). Reliability and validity assessment. Sage.

Clark, L. A., & Watson, D. (2015). Constructing validity: Basic issues in objective scale development. Psychological Assessment, 7(3), 309-319.

Cohen, R. J., & Swerdlik, M. E. (2017). Psychological testing and assessment: An introduction to tests and measurement (9th ed.). McGraw-Hill Education.

Crocker, L., & Algina, J. (1986). Introduction to classical and modern test theory. Wadsworth Belmont.

Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297-334.

Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52(4), 281-302.

DeVellis, R. F. (2003). Scale development: Theory and applications. Sage.

DeVellis, R. F. (2016). Scale development: Theory and applications. Sage.

DeVellis, R. F. (2017). Scale development: Theory and applications. Sage.

Dillman, D. A., Smyth, J. D., & Christian, L. M. (2014). Internet, phone, mail, and mixed-mode surveys: The tailored design method (4th ed.). Wiley.

Dubois, D., & Prade, H. (1991). Fuzzy sets and systems: Theory and applications. Academic Press.

Dufau, S., Duñabeitia, J. A., Moret-Tatay, C., McGonigal, A., Peeters, D., Alario, F. X., ... & Grainger, J. (2011). Smart phone, smart science: How the use of smartphones can revolutionize research in cognitive science. PloS One, 6(9), e24974.

Eid, M., & Diener, E. (2006). Handbook of multimethod measurement in psychology. American Psychological Association.

Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18(1), 39-50.

Guttman, L. (1950). The basis for scalogram analysis. In S. A. Stouffer, L. Guttman, E. A. Suchman, P. F. Lazarsfeld, S. A. Star, & J. A. Clausen (Eds.), Studies in social psychology in World War II: Measurement and prediction (Vol. 4, pp. 60-90). Princeton University Press.

Haladyna, T. M., Downing, S. M., & Rodriguez, M. C. (2002). A review of multiple-choice item-writing guidelines for classroom assessment. Applied Measurement in Education, 15(3), 309-334.

Hallgren, K. A. (2012). Computing inter-rater reliability for observational data: An overview and tutorial. Tutorials in Quantitative Methods for Psychology, 8(1), 23-34.

Haynes, S. N., Richard, D., & Kubany, E. S. (1995). Content validity in psychological assessment: A functional approach to concepts and methods. Psychological Assessment, 7(3), 238-247.

Hays, R. D., & Revicki, D. (2016). Reliability and validity (including responsiveness). In F. G. Fayers & D. Machin (Eds.), Quality of life: The assessment, analysis, and interpretation of patient-reported outcomes (2nd ed., pp. 25-42). Wiley.

Huskisson, E. C. (1974). Visual analogue scales. In J. Melzack (Ed.), Pain measurement and assessment (pp. 33-37). Raven Press.

John, O. P., & Srivastava, S. (1999). The Big Five trait taxonomy: History, measurement, and theoretical perspectives. In L. A. Pervin & O. P. John (Eds.), Handbook of personality: Theory and research (2nd ed., pp. 102-138). Guilford Press.

Kline, R. B. (2015). Principles and practice of structural equation modeling. The Guilford Press.

Klir, G. J., & Yuan, B. (1996). Fuzzy sets and fuzzy logic: theory and applications. Possibility theory versus Probab. Theory, 32(2), 207-208.

Kroenke, K., & Spitzer, R. L. (2002). The PHQ-9: A new depression diagnostic and severity measure. Psychiatric Annals, 32(9), 509-515.

Krosnick, J. A., & Presser, S. (2010). Question and questionnaire design. In P. V. Marsden & J. D. Wright (Eds.), Handbook of survey research (2nd ed., pp. 263-314). Emerald.

La Greca, A. M., & Lopez, N. (1998). Social anxiety among adolescents: Linkages with peer relations and friendships. Journal of Abnormal Child Psychology, 26(2), 83-94.

Lawshe, C. H. (1975). A quantitative approach to content validity. Personnel Psychology, 28(4), 563-575.

Likert, R. (1932). A technique for the measurement of attitudes. Archives of Psychology, 22(140), 1-55.

Lohr, S. (2018). Big data analytics and the promise of big data. Big Data & Society, 5(1), 2053951718756680.

Lord, F. M. (1952). A theory of test scores. Psychometric Monographs, 7(3), 1-72.

Mayer, R. E., & Wittrock, M. C. (2015). Problem solving. In J. Hattie & E. M. Anderman (Eds.), International guide to student achievement (pp. 224-227). Routledge.

Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons' responses and performances as scientific inquiry into score meaning. American Psychologist, 50(9), 741-749.

Meyer, T. J., Miller, M. L., Metzger, R. L., & Borkovec, T. D. (1990). Development and validation of the Penn State Worry Questionnaire. Behaviour Research and Therapy, 28(6), 487-495.

Murphy, K. R., & Davidshofer, C. O. (2005). Psychological testing: Principles and applications (6th ed.). Pearson.

Netemeyer, R. G., Bearden, W. O., & Sharma, S. (2003). Scaling procedures: Issues and applications. Sage.

Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). McGraw-Hill.

O'Neill, T. A., Goffin, R. D., & Tett, R. P. (2009). Content validation is fundamental for optimizing the criterion validity of personality tests. Industrial and Organizational Psychology, 2(4), 509-513.

Osgood, C. E., Suci, G. J., & Tannenbaum, P. H. (1957). The Measurement of meaning. University of Illinois Press.

Polit, D. F., & Beck, C. T. (2006). The content validity index: Are you sure you know what's being reported? Critique and recommendations. Research in Nursing & Health, 29(5), 489-497.

Polit, D. F., Beck, C. T., & Owen, S. V. (2007). Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Research in Nursing & Health, 30(4), 459-467.

Reis, H. T., & Judd, C. M. (Eds.). (2000). Handbook of research methods in social and personality psychology. Cambridge University Press.

Revelle, W. (2016). Psych: Procedures for personality and psychological research. Northwestern University.

Revelle, W. (2020). Psych: Procedures for personality and psychological research. Northwestern University. https://CRAN.R-project.org/package=psych

Robins, R. W., & Trzesniewski, K. H. (2005). Self-esteem development across the lifespan. Current Directions in Psychological Science, 14(3), 158-162.

Rosenberg, M. (1965). Society and the adolescent self-image. Princeton University Press.

Sarstedt, M., Ringle, C. M., & Hair, J. F. (2021). Partial least squares structural equation modeling. In Handbook of market research (pp. 587-632). Springer.

Schmitt, N. (1996). Uses and abuses of coefficient alpha. Psychological Assessment, 8(4), 350-353.

Smith, G. T., & Glass, G. V. (1977). Meta-analysis of psychotherapy outcome studies. American Psychologist, 32(9), 752-760.

Smith, P. C., Kendall, L. M., & Hulin, C. L. (1969). The measurement of satisfaction in work and retirement. Rand McNally.

Smith, R. H. (2016). The joy of pain: Schadenfreude and the dark side of human nature. Oxford University Press

Spearman, C. (1910). Correlation calculated from faulty data. British Journal of Psychology, 3(3), 271-295.

Spence, J. T., & Helmreich, R. L. (1972). The attitudes toward women scale: An objective instrument to measure attitudes toward the rights and roles of women in contemporary society. JSAS Catalog of Selected Documents in Psychology, 2, 66-67.

Spitzer, R. L., Kroenke, K., Williams, J. B., & Löwe, B. (2006). A brief measure for assessing generalized anxiety disorder: The GAD-7. Archives of Internal Medicine, 166(10), 1092-1097.

Streiner, D. L., & Norman, G. R. (2008). Health measurement scales: A practical guide to their development and use. Oxford University Press.

Streiner, D. L., Norman, G. R., & Cairney, J. (2015). Health measurement scales: a practical guide to their development and use. Oxford University Press.

Tabachnick, B. G., & Fidell, L. S. (2019). Using multivariate statistics (7th ed.). Pearson.

Thurstone, L. L. (1928). Attitudes can be measured. The American Journal of Sociology, 33(4), 529-554.

Wainer, H. (2000). Computerized adaptive testing: A primer (2nd ed.). Lawrence Erlbaum.

Wechsler, D. (1949). The Wechsler intelligence scale for children. The Psychological Corporation.

Willis, G. B. (2005). Cognitive interviewing: A tool for improving questionnaire design. Sage.

Zadeh, L. A. (1973). Outline of a new approach to the analysis of complex systems and decision processes. IEEE Transactions on Systems, Man, and Cybernetics, (1), 28-44.