Factor Analysis - Components, PCA, Comparison of Factor Analysis and PCA

HELLO FRIENDS !!!

Hope you all are enjoying reading the blog, and the data that is updated is valuable to all of you. Let us today get more familiar with a new concept called Factor Analysis, PCA.

Factor Analysis

Factor analysis is a collection of methods used to examine how underlying constructs influence the responses on a number of measured variables.

There are basically two types of factor analysis: exploratory and confirmatory.

1. Exploratory factor analysis (EFA) attempts to discover the nature of the constructs influencing a set of responses.

2. Confirmatory factor analysis (CFA) tests whether a specified set of constructs is influencing responses in a predicted way.

Both types of factor analyses are based on the Common Factor Model, illustrated in figure 1.1. This model proposes that each observed response (measure 1 through measure 5) is influenced partially by underlying common factors (factor 1 and factor 2) and partially by underlying unique factors (E1 through E5). The strength of the link between each factor and each measure varies, such that a given factor influences some measures more than others. This is the same basic model as is used for LISREL analyses.

Factor analyses are performed by examining the pattern of correlations (or covariances) between the observed measures. Measures that are highly correlated (either positively or negatively) are likely influenced by the same factors, while those that are relatively uncorrelated are likely influenced by different factors.

Exploratory Factor Analysis

Objectives:

The primary objectives of an EFA are to determine:

· The number of common factors influencing a set of measures.

· The strength of the relationship between each factor and each observed measure.

Some common uses of EFA are to:

· Identify the nature of the constructs underlying responses in a specific content area.

· Determine what sets of items “hang together” in a questionnaire.

· Demonstrate the dimensionality of a measurement scale. Researchers often wish to develop scales that respond to a single characteristic.

· Determine what features are most important when classifying a group of items.

· Generate “factor scores" representing values of the underlying constructs for use in other analyses.

Confirmatory Factor Analysis

Objectives

The primary objective of a CFA is to determine the ability of a predefined factor model to fit an observed set of data.

Some common uses of CFA are to:

· Establish the validity of a single factor model.

· Compare the ability of two different models to account for the same set of data.

· Test the significance of a specific factor loading.

· Test the relationship between two or more factor loadings.

· Test whether a set of factors are correlated or uncorrelated.

· Assess the convergent and discriminant validity of a set of measures.

Factor Analysis vs. Principal Component Analysis

· Exploratory factor analysis is often confused with principal component analysis (PCA), a similar statistical procedure. However, there are significant differences between the two: EFA and PCA will provide somewhat different results when applied to the same data.

· The purpose of PCA is to derive a relatively small number of components that can account for the variability found in a relatively large number of measures. This procedure, called data reduction, is typically performed when a researcher does not want to include all of the original measures in analyses but still wants to work with the information that they contain.

· Differences between EFA and PCA arise from the fact that the two are based on different models. An illustration of the PCA model is provided in figure 2.1. The first difference is that the direction of influence is reversed: EFA assumes that the measured responses are based on the underlying factors while in PCA the principal components are based on the measured responses. The second difference is that EFA assumes that the variance in the measured variables can be decomposed into that accounted for by common factors and that accounted for by unique factors. The principal components are defined simply as linear combinations of the measurements, and so will contain both common and unique variance.

In summary, you should use EFA when you are interested in making statements about the factors that are responsible for a set of observed responses, and you should use PCA when you are simply interested in performing data reduction.