Factor analysis (FA) is a method for identifying a structure (factors, dimensions) that underlies the relationship among a set of observed variables. Factor analysis is a technique that transforms the correlations among a set of observed variables into a smaller number of underlying factors that capture all the essential information about the linear relationships among the original test scores.
Table of Contents
Factor analysis is a statistical procedure that examines the relationships between observed variables (measurements) and underlying latent factors.
Application of Factor Analysis
Explore Data for Pattern
Factor analysis can be done in an explanatory fashion to reveal patterns among the interrelations of the items
Data Reduction
Factor analysis can be used to reduce a large number of variables into a smaller and more manageable number of factors. Factor analysis can be used to reduce a large number of variables into a parsimonious set of few factors that account better for the underlying variance (causal impact) in the measured phenomenon.
Confirm Hypothesis of Factor Structure
Factor analysis can be used to test whether a set of items designed to measure a certain variable(s) does infact reveal the hypothesized factor structure (that is, whether the underlying latent factor truly “Causes” the variance in the observed variables and how “Certain” we can be about it).
In measurement research, when a researcher wishes to validate a scale with a given or hypothesized factor structure, conformatory factor analysis is used. Let the theorized model
Each variable contains a bunch of items (questions), and the relationship looks like we have new constructs or highly bound constructs.
For exploratory factor analysis, one should have a latent and reflective construction.
Basic Assumptions
Kaiser-Meyer-Olkin (KMO)
Kaiser-Mayer-Olkin (KMO) is a measure of sampling adequacy. It generally indicates whether or not the variables are able to be grouped into a smaller set of underlying factors. That is, will data factor well? It varies from 0 to 1 and should be 0.6 or higher to proceed. If the value is less than 0.5, the results of the factor analysis probably would not be very useful.
Bartlett’s Test of Sphericity
In Factor Analysis, Bartlett’s Test of Sphericity is a statistical test used to determine whether your dataset is suitable for structure detection. It checks if the variables in your sample are related at all.
The test compares your observed correlation matrix (the actual relationships between your variables) against an identity matrix (a theoretical model where all variables are perfectly independent and have zero correlation with each other).
Extraction Methods
In Factor Analysis, the “extraction method” is the mathematical process used to uncover the underlying factors (latent variables) from your set of observed variables. The goal is to find a small number of factors that explain as much of the variance in your data as possible.
Here are the most common extraction methods, broken down by when and why you would use them:
Principal Component Analysis (PCA)
Consider all of the available variance (common + unique) (places 1’s on the diagonal of the correlation matrix). It seeks a linear combination of variables, such that maximum variance is extracted. Use PCA if you are having trouble. It is best to use it when your primary goal is to reduce a large number of variables into a smaller set of components while retaining as much information as possible.
Principal Axis Factoring (PAF)
Considers only common variance (places communality estimates on the diagonal of the correlation matrix). It seeks the least number of factors that can account for the common variance (correlation) of a set of variables. Principal Axis Factoring (PAF) is preferred in SEM because it accounts for covariation, whereas PCA accounts for total variance. Try PAF before PCA.
Its best use is when your data violates the assumption of multivariate normality, or when you want to identify latent constructs rather than just reducing data. It ignores the unique variance (error) of each variable and only analyzes the shared variance (commonalities).
Maximum Likelihood (ML)
The Maximum Likelihood (ML) method maximizes differences between factors, Provides model fit estimate. This is the approach used in AMOS, so if you are going to use AMOS for CFA and a structural model, you should use this one during EFA. It estimates factor loadings that are most likely to have produced the observed correlation matrix, assuming the data follow a multivariate normal distribution.
It is best to use it when your data is normally distributed. A major advantage is that it provides “goodness-of-fit” indexes (like $\chi^2$) to tell you how well your model fits the data.
Alpha Factoring
This method treats the variables as a sample from a universe of possible variables. It aims to maximize the reliability (alpha coefficient) of the factors. It is best to use it when you want to ensure the factors you find are internally consistent and would likely appear if you used a different set of similar variables.
Image Factoring
Based on the concept of “image analysis,” it uses multiple regression to predict each variable from all other variables. It is used less frequently today, but helpful when you have a very large number of variables and want to focus strictly on the predictable portion of the data.
