Challenges and recommendations of exploratory and confirmatory factor analysis: A narrative review from a nursing perspective

Article Type : Reviews

Author

1 Department of Psychiatric Nursing, School of Nursing and Midwifery, Tehran University of Medical Sciences, Tehran, Iran

2 Student Research Committee, Mazandaran University of Medical Sciences, Sari, Iran

Abstract

Factor analysis is a statistical method used to explore the underlying structure of a set of variables. Exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) are two widely used methods in this domain. In recent years, new findings have emerged that shed light on the strengths and limitations of both methods. This paper discussed about new findings of EFA and CFA, challenges ahead and some necessary recommendations.

Keywords

1 Introduction

Factor analysis is a statistical technique that has a rich history, dating back to the early 20th century. The technique was first developed in the field of psychology, but it has since been applied to a wide range of disciplines, including social sciences, economics, and engineering. The earliest known use of factor analysis can be traced back to the British statistician, Charles Spearman, who introduced the concept of "general intelligence" in 1904. Spearman used factor analysis to show that intelligence was not a single, unitary construct, but rather a collection of distinct abilities that were correlated with each other. Spearman's work laid the foundation for modern psychometric theory, which explores the underlying structure of psychological constructs [1].

In the 1920s and 1930s, a number of prominent psychologists, including Louis Thurstone and L.L. Thurstone, began to refine and expand on Spearman's work. They developed new factor analysis techniques, such as multiple factor analysis and the method of successive approximations, which allowed for more complex and nuanced analyses of psychological constructs. During the 1950s and 1960s, factor analysis became increasingly popular in the social sciences, particularly in the fields of sociology and political science. This was in part due to the growing availability of computers, which made it easier to perform complex analyses on large datasets. During this time, researchers developed new factor analysis techniques, such as confirmatory factor analysis (CFA), which allowed for more rigorous testing of theoretical models. In the 1970s and 1980s, factor analysis became a standard tool in the field of marketing research. Researchers used factor analysis to identify the underlying factors that influenced consumer behavior and to develop more effective marketing strategies. Today, factor analysis is widely used in a variety of fields, including psychology, sociology, economics, and engineering. It is used to explore the underlying structure of complex phenomena, to develop new theories and models, and to validate the measurement tools used in research [2].

 

2 Novel findings and advancements of exploratory factor analysis (EFA) and CFA

EFA and CFA are widely used statistical methods for analyzing data in social sciences, psychology, education, and other fields. EFA is an exploratory technique used to uncover the underlying structure of a set of variables, whereas CFA is a confirmatory technique used to test a pre-specified theoretical model. Both techniques are used to identify the relationships between observed variables and their underlying latent constructs.

Recently, there have been several new findings about performing EFA and CFA that have the potential to improve the accuracy and reliability of the results obtained from these techniques. One new finding concerns the choice of the extraction method in EFA. Traditionally, researchers have used Principal Component Analysis (PCA) or Principal Axis Factoring (PAF) as the extraction method in EFA. However, recent research has shown that these methods can lead to biased results, especially when the data is non-normal or when the number of variables is large. Therefore, alternative extraction methods, such as Maximum Likelihood (ML), Unweighted Least Squares (ULS), and Robust Maximum Likelihood (RML), have been proposed as better alternatives [3].

Another important finding concerns the evaluation of the goodness-of-fit in CFA. The traditional method of evaluating the fit of a CFA model has been to use the chi-square test. However, this method has several limitations, such as its sensitivity to sample size, its tendency to reject models with large sample sizes, and its inability to provide information about the degree and direction of misfit. Therefore, alternative fit indices, such as the Comparative Fit Index (CFI), the Tucker-Lewis Index (TLI), the Root Mean Square Error of Approximation (RMSEA), and the Standardized Root Mean Square Residual (SRMR), have been proposed as better alternatives. Another new finding concerns the issue of model complexity in CFA. Researchers have traditionally used a rule of thumb of having at least three observed variables per latent factor to ensure model identification. However, recent research has shown that this rule of thumb can lead to overparameterization and model misspecification, especially when the sample size is small. Therefore, alternative methods, such as the Bayesian Information Criterion (BIC) and the Sample-Size Adjusted BIC (SABIC), have been proposed as better alternatives for determining model complexity [4].

One area of research that has gained attention in recent years concerns the issue of non-normality in EFA and CFA. Traditionally, researchers have assumed that the data used in these techniques are normally distributed. However, this assumption is often violated in practice, especially when dealing with ordinal or skewed data. Therefore, alternative methods, such as the Weighted Least Squares (WLS) and the Robust Maximum Likelihood (RML), have been proposed as better alternatives for handling non-normal data. These methods are more robust to non-normality and can produce more accurate and reliable results. Sample size in EFA and CFA is another main issue. Traditionally, researchers have used a rule of thumb of having at least five observations per variable to ensure adequate power and reliability. However, recent research has shown that this rule of thumb can lead to underpowered and unreliable results, especially when dealing with complex models or small effect sizes. Therefore, alternative methods, such as Monte Carlo simulations and bootstrapping, have been proposed as better alternatives for determining the appropriate sample size for EFA and CFA [4, 5].

A related issue concerns the issue of missing data in EFA and CFA. Traditionally, researchers have used ad hoc methods, such as listwise deletion or mean imputation, to handle missing data. However, these methods can lead to biased and unreliable results. Therefore, alternative methods, such as Full Information Maximum Likelihood (FIML) and Multiple Imputation (MI), have been proposed as better alternatives for handling missing data. These methods can produce more accurate and reliable results and can help to reduce bias and increase power. Another important finding concerns the issue of model identification in EFA and CFA. Traditionally, researchers have assumed that the latent factors are uncorrelated with each other. However, recent research has shown that this assumption can lead to model misspecification and biased results, especially when the factors are conceptually related or when the data is multidimensional. Therefore, alternative models, such as the oblique factor model and the correlated factor model, have been proposed as better alternatives for modeling the interrelationships between latent factors. These models can produce more accurate and reliable results and can help to improve the validity and reliability of the measurement instruments. Finally, another important area of research concerns the issue of measurement invariance in CFA. Traditionally, researchers have assumed that the underlying structure of the data is invariant across different groups, such as genders or cultures. However, recent research has shown that this assumption can be violated in practice, especially when dealing with complex models or diverse populations. Therefore, alternative methods, such as multigroup CFA and multiple indicators multiple causes (MIMIC) models, have been proposed as better alternatives for testing measurement invariance across different groups. These methods can help to identify potential sources of bias and can help to improve the validity and reliability of the measurement instruments [5].

Determining the appropriate number of factors to extract is a crucial step in EFA is another important subject. Traditionally, researchers have relied on the Kaiser criterion, which suggests retaining all factors with eigenvalues greater than one. However, recent research has shown that this method can lead to overextraction of factors and that alternative methods, such as parallel analysis and minimum average partial (MAP) criterion, can be more accurate in determining the number of factors to retain. Similarly, in CFA, researchers have traditionally relied on fit indices such as the chi-square test, which can be affected by sample size and model complexity. Recent research has shown that alternative fit indices, such as the root mean square error of approximation (RMSEA) and the comparative fit index (CFI), can provide more accurate and reliable estimates of model fit. Factor analysis is often used to explore the underlying structure of data and to identify the latent constructs that influence the observed variables. However, traditional factor analysis methods are data-driven and do not incorporate theoretical knowledge or prior expectations into the analysis. Recent research has shown that incorporating theory into factor analysis, such as using Bayesian methods or hierarchical models, can improve the accuracy and validity of the results and can lead to more meaningful and interpretable factor structures [4].

In conclusion, recent findings about EFA and CFA have highlighted the importance of using alternative methods for handling non-normal data, determining sample size, handling missing data, modeling the interrelationships between latent factors, and testing measurement invariance across different groups. By taking these new findings into account, researchers can improve the accuracy and reliability of the results obtained from these techniques, which can have important implications for theory development, measurement, and intervention in the social sciences, psychology, education, and other fields.

 

3 Limitations of EFA and CFA

While EFA and CFA have numerous advantages, there are also several limitations that should be considered when interpreting the results.

EFA and CFA assume that the data are normally distributed. If the data are not normally distributed, it can lead to biased and inaccurate estimates of the factor loadings and model parameters. While there are methods for dealing with non-normal data, such as robust estimation methods, these methods may not always be appropriate or reliable. The accuracy and reliability of EFA and CFA results depend on the sample size. In general, larger sample sizes are better because they provide more reliable estimates of the factor loadings and model parameters. However, if the sample size is too small, it can lead to unstable or unreliable results. The sample used in EFA and CFA should be representative of the population of interest. If the sample is not representative, it can lead to biased and inaccurate results. For example, if the sample only includes individuals from a certain age group or socioeconomic status, the factor structure may not be generalizable to the larger population.

CFA assumes that the factor structure is known and specified in advance. If the factor structure is misspecified, it can lead to biased and inaccurate estimates of the factor loadings and model parameters. While there are methods for dealing with model misspecification, such as modification indices, these methods should be used with caution and should be interpreted carefully. The interpretation of factors in EFA and CFA can be subjective and open to different interpretations. Factors are often named based on the items that load most strongly on them, but different researchers may interpret the same factor structure differently. Furthermore, the interpretation of factors may be influenced by the researcher's prior assumptions or theoretical perspectives. EFA and CFA are correlational techniques and do not establish causal relationships between the latent constructs and observed variables. While EFA and CFA can identify the underlying structure of data, they cannot determine whether the latent constructs cause the observed variables or vice versa [6].

In summary, EFA and CFA have several limitations that should be considered when interpreting the results. These limitations include assumptions of normality, sample size and representativeness, sensitivity to model misspecification, subjective interpretation of factors, and lack of causality. While these limitations do not invalidate the usefulness of EFA and CFA, they should be taken into account when interpreting the results and drawing conclusions from them.

 

4 Challenges and recommendations of EFA and CFA

4.1 Model Misspecification

Model misspecification is a common problem in both EFA and CFA. This occurs when the proposed model does not fit the data well, leading to inaccurate and unreliable results. In EFA, model misspecification can occur when the number of factors selected is incorrect, or when the assumption of uncorrelated factors is not met. In CFA, model misspecification can occur when the measurement model does not match the theoretical model, or when the assumption of measurement invariance across different groups is not met. To address model misspecification, researchers can use several techniques, such as goodness-of-fit indices, modification indices, and exploratory factor analysis of residuals. These techniques can help to identify potential sources of model misspecification and provide guidance for modifying the model to better fit the data [7].

 

4.2 Non-Normal Data

Another challenge with EFA and CFA is the assumption of normality. These techniques assume that the data are normally distributed, which may not always be the case in practice. Non-normal data can lead to biased and unreliable results and can also affect the accuracy of the goodness-of-fit indices used to evaluate model fit. To address non-normal data, researchers can use alternative methods, such as robust estimation, weighted least squares, or maximum likelihood estimation with non-normal distributions. These techniques can help to account for non-normality and produce more accurate and reliable results [8].

 

4.3 Sample Size

Sample size is another important challenge in EFA and CFA. Traditionally, researchers have used a rule of thumb of having at least five observations per variable to ensure adequate power and reliability. However, recent research has shown that this rule of thumb can lead to underpowered and unreliable results, especially when dealing with complex models or small effect sizes. To address sample size, researchers can use power analysis techniques, such as Monte Carlo simulations and bootstrapping, to determine the appropriate sample size for their model. These techniques can help to ensure that the sample size is large enough to detect small effect sizes and produce reliable results [8].

 

4.4 Missing Data

Missing data is a common challenge in EFA and CFA. Traditional methods, such as listwise deletion or mean imputation, can lead to biased and unreliable results. Furthermore, missing data can affect the accuracy of the goodness-of-fit indices used to evaluate model fit. To address missing data, researchers can use alternative methods, such as full information maximum likelihood (FIML) or multiple imputation (MI). These techniques can help to account for missing data and produce more accurate and reliable results [9].

 

4.5 Multiple Comparisons

Multiple comparisons are another challenge in EFA and CFA. When conducting multiple tests, the probability of obtaining a false positive result increase. This can lead to the identification of spurious factors or correlations, which can negatively affect the validity and reliability of the results. To address multiple comparisons, researchers can use techniques such as Bonferroni correction or false discovery rate (FDR) correction to adjust the significance level of their tests. These techniques can help to reduce the likelihood of false positive results and improve the validity and reliability of the results [10].

 

4.6 Multicollinearity and Outliers

Multicollinearity occurs when two or more variables in the analysis are highly correlated with each other. This can lead to instability in the factor structure and affect the accuracy of the factor loadings. Multicollinearity can also lead to inflated standard errors and biased estimates. To address multicollinearity, researchers can use techniques such as principal axis factoring, which allows for the correlation between factors, or orthogonal rotation methods, which force the factors to be uncorrelated. Outliers are data points that are significantly different from the rest of the data. Outliers can have a strong influence on the factor structure and affect the accuracy of the factor loadings. They can also affect the accuracy of the goodness-of-fit indices used to evaluate model fit. To address outliers, researchers can use techniques such as robust estimation, which downweights the influence of outliers, or Winsorization, which replaces extreme values with less extreme values [11].

 

4.7 Non-Linear Relationships

EFA and CFA assume linear relationships between the variables and the latent constructs. However, in practice, these relationships may be non-linear. Non-linear relationships can lead to biased and unreliable results and affect the accuracy of the goodness-of-fit indices used to evaluate model fit. To address non-linear relationships, researchers can use techniques such as polynomial regression, spline regression, or non-linear factor analysis.

 

4.8 Model Selection

Finally, model selection is an important challenge in both EFA and CFA. Researchers must choose the appropriate number of factors or latent constructs to include in the analysis. Choosing too few factors can lead to underfitting, while choosing too many factors can lead to overfitting. To address model selection, researchers can use techniques such as scree plots, parallel analysis, or Bayesian information criteria (BIC) to determine the appropriate number of factors to include in the analysis [12].

Moreover, it is important to note that EFA and CFA should not be used in isolation, but rather in conjunction with other techniques, such as qualitative analysis, theory building, and replication studies. These techniques can help to ensure that the results are valid, reliable, and robust. In addition, researchers should also pay close attention to the assumptions and limitations of EFA and CFA. For example, they should be aware that these techniques assume linearity, additivity, and homoscedasticity. They should also be aware that these techniques are limited to the variables and samples used in the analysis and may not generalize to other populations or contexts. Overall, while EFA and CFA have their challenges, they are valuable tools for identifying latent constructs and validating measurement instruments. By using appropriate techniques and strategies, researchers can overcome these challenges and produce reliable and valid results.

 

5 Conclusions

In conclusion, while exploratory and confirmatory factor analysis offer valuable insights into the underlying structures of observed variables, they come with their fair share of challenges. By implementing the recommended strategies outlined above, researchers can enhance the reliability and validity of their factor analyses, leading to more robust and trustworthy conclusions. Vigilance in model specification, factor extraction, and interpretation is key to harnessing the full potential of EFA and CFA in advancing our understanding of complex phenomena.

 

Acknowledgements

Many thanks to student research committee of Mazandaran university of medical sciences (Sari, Iran) for financial support. The proposal code of this article is 4724 and this study was proved in ethics committee of Mazandaran university of medical sciences (ethics code: IR.MAZUMS.REC.1398.610).

 

Authors’ contributions

Substantial contributions to the conception or design of the work; or the acquisition, analysis, or interpretation of data for the work: AHG; Drafting the work or revising it critically for important intellectual content: AHG; Final approval of the version to be published: AHG; Agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved: AHG.

 

Funding

Self-funded.

 

Ethics approval and consent to participate

Not applicable.

 

Competing interests

We do not have potential conflicts of interest with respect to the research, authorship, and publication of this article.

 

Availability of data and materials

The datasets used during the current study are available from the corresponding author on request.

 

This is an open access article under the terms of the Creative Commons Attribution-NonCommercial 4.0 License (CC BY-NC 4.0).

© 2023 The Author(s).

  1. Kim J-O, Ahtola O, Spector PE, Kim J-O, Mueller CW. Introduction to factor analysis: What it is and how to do it: Sage: Thousand Oaks, CA, USA, 1978.
  2. Gorsuch RL. Factor analysis: Psychology press; 2013.
  3. Taherdoost H, Sahibuddin S, Jalaliyoon N. Exploratory factor analysis; concepts and theory. Adv Pure Appl Math. 2022;27:375-382.
  4. Koran J. Indicators per factor in confirmatory factor analysis: More is not always better. Struct Equ Modeling. 2020;27(5):765-772.
  5. Marsh HW, Guo J, Dicke T, Parker PD, Craven RG. Confirmatory Factor Analysis (CFA), Exploratory Structural Equation Modeling (ESEM), and Set-ESEM: Optimal Balance Between Goodness of Fit and Parsimony. Multivariate Behav Res. 2020;55(1):102-119.
  6. Steenkamp J-BE, Maydeu-Olivares A. Unrestricted factor analysis: A powerful alternative to confirmatory factor analysis. J Acad Mark Sci. 2022;51(1):86-113.
  7. Savalei V. The relationship between root mean square error of approximation and model misspecification in confirmatory factor analysis models. Educ Psychol Meas. 2012;72(6):910-932.
  8. Brauer K, Ranger J, Ziegler M. Confirmatory factor analyses in psychological test adaptation and development: a nontechnical discussion of the WLSMV estimator. Psychol Test Adapt Dev. 2023;4(1):4–12.
  9. Lei P-W, Shiverdecker LK. Performance of estimators for confirmatory factor analysis of ordinal variables with missing data. Struct Equ Modeling. 2020;27(4):584-601.
  10. Glickman ME, Rao SR, Schultz MR. False discovery rate control is a recommended alternative to Bonferroni-type adjustments in health studies. J Clin Epidemiol. 2014;67(8):850-857.
  11. Sürücü L, Yikilmaz İ, Maslakçi A. Exploratory Factor Analysis (EFA) in Quantitative Researches and Practical Considerations. Center for Open Science; 2022.
  12. McNeish D, Wolf MG. Dynamic fit index cutoffs for confirmatory factor analysis models. Psychol Methods. 2023;28(1):61-88.
Volume 1, Issue 3
October 2023
Pages 133-137
  • Receive Date: 25 June 2023
  • Revise Date: 20 July 2023
  • Accept Date: 23 July 2023
  • First Publish Date: 01 October 2023
  • Publish Date: 01 October 2023