How to find out which survey questions belong together
Exploratory factor analysis checks whether the questions in a survey fall into groups, such as questions about the work itself and questions about the manager. This guide explains the idea in plain terms and shows how to run it and read the results in AnalyZ Solutions.
12 min read Scale & Reliability Intermediate
01What is exploratory factor analysis?
When you write a questionnaire you usually have themes in mind, such as questions about the work itself and questions about the manager. Exploratory factor analysis, or EFA, checks from the answers people actually gave whether the questions fall into those groups. Questions that people answer in a similar way, so that those who agree with one tend to agree with another, are treated as measuring the same theme. Each theme is called a factor. You never see a factor directly. You infer it from the pattern in the answers.
The short version. Select the questions, enter how many themes you expect, and click run. The defaults work well for most surveys, and you can switch to Oblimin if you think the themes are related. Check that Bartlett's test is significant and KMO is at least 0.60. Then read the loadings table: each question should be strongly tied to one factor, with a loading of 0.40 or more.
An example
You survey 300 health workers with ten statements about their jobs, each rated from 1 to 5. You wrote them with three ideas in mind: how meaningful the work feels, how good the supervision is, and how manageable the workload is. Before you add the items into three subscale scores, you want evidence that the answers really fall into three groups, and that each statement sits in the group you intended. EFA gives you that evidence from the data.
Key terms
Factor. A theme behind a group of questions. It is not measured directly. It shows up in how people answer several questions in the same way.
Loading. How closely a question is tied to a factor. Ignoring the sign, a bigger number means a closer tie.
Communality. How much of the variation in a question's answers the factors explain together.
Uniqueness. What the factors do not explain. It is the part that belongs to the question alone, plus measurement error.
EFA compared with PCA
EFA
Explain and name
Models only the variation items share. Assumes hidden factors. Normally rotated so that each item loads mainly on one factor.
PCA
Describe and reduce
Models all the variation. Gives mixes of the variables with no hidden causes assumed. See the PCA guide.
What you get out of it
Suitability checks KMO and Bartlett's test, which show whether the items are correlated enough for a factor analysis.
Factor loadings A table and heatmap of how strongly each item loads on each factor.
Variance and communalities How much each factor explains, and how well the factors account for each item.
Factor correlations For an oblique rotation, how strongly the factors are related to each other.
02When to use it
Use EFA when you do not yet know the structure of a set of items, or when you want to check the structure before building scales. Typical cases are developing a new questionnaire, adapting one to a new language or population, and finding items that fit badly and could be removed.
If you already have a firm theory about which item belongs to which factor, and want to test how well that structure fits, the right tool is confirmatory factor analysis, which AnalyZ Solutions does not offer. EFA is also not the choice when you only want to condense variables. For that, use principal component analysis. Once you know which items form a factor, check each group with Cronbach's alpha.
How much data you need
A common minimum is five respondents for each item, so ten items need at least 50 people. Results are steadier with 100 or more, and with 200 or more when the loadings are modest. The items should be numeric ratings with at least five response options. Yes or no items distort the correlations that EFA relies on.
03The factor model
EFA describes each standardised item as a weighted sum of the factors plus a part that belongs to the item alone.
For the item I plan to stay here in the worked example below, the loadings are 0.463, 0.258 and 0.073, so the communality is 0.463² + 0.258² + 0.073² = 0.286. The factors explain 28.6% of the variation in that item, and the remaining 71.4% is unique to it.
The variance explained by a factor is the sum of the squared loadings of all items on it, called the sum of squared loadings, divided by the number of items. It shows how much of the total variation among the items that factor accounts for.
Are the items suited to factor analysis?
The module reports two checks before you read any loadings.
Bartlett's test of sphericity asks whether the correlation matrix differs from a matrix in which no items are correlated. If it does not, there is nothing to factor. The test uses the determinant of the correlation matrix, written |R|, with p items and N people.
chi-square = -( N - 1 - (2p + 5) / 6 ) × ln |R| degrees of freedom = p × (p - 1) / 2
With a large sample it is almost always significant, so passing it is a low bar. The module shows it in a green banner when p is below 0.05, and in grey otherwise.
The KMO measure compares ordinary correlations with partial correlations, and runs from 0 to 1. Values above about 0.60 are usually accepted, and the module labels 0.70 to 0.79 as middling, 0.80 to 0.89 as meritorious and 0.90 and above as marvellous. Below 0.50 is unacceptable.
04Extraction, number of factors and rotation
Extraction method
The module offers two ways to extract the first solution.
Principal axis factoring (PAF) is the default. It starts with an estimate of each item's communality, puts those estimates on the diagonal of the correlation matrix, extracts the factors, and then re-estimates the communalities from the result. It repeats up to 25 times until the estimates settle. Because it uses only shared variation, it follows the logic of factor analysis.
Principal components uses all the variation in each item, as PCA does. Communalities come out higher, and the result is closer to a PCA than to a true factor model.
For most questionnaire work, PAF is the better choice.
How many factors
You choose the number, and it must be smaller than the number of items. The usual guides are the same as for PCA: how many eigenvalues of the correlation matrix are above 1, where the scree plot bends, and whether the factors make sense in words. The EFA module does not draw a scree plot. To see one, run a PCA on the same items. Theory counts too. If you designed the questionnaire around three ideas, a three-factor solution is the first one to test.
Rotation
An unrotated solution puts most of the weight on the first factor, so many items load on it and the picture is blurred. Rotation turns the factors to make each item load strongly on one and weakly on the others, which is called simple structure. It changes how the variation is shared out among the factors. It does not change the communalities.
Varimax (orthogonal) keeps the factors uncorrelated. It is simple and gives clear results, and it is the module's default.
Oblimin (oblique) lets the factors correlate. Many scales in the social sciences measure related ideas, so this is often the more realistic choice. The module adds a Factor correlations tab for this option.
None (unrotated) shows the raw solution.
A practical rule is to try Oblimin first. If the factor correlations are all low, Varimax would give much the same picture. The module comments on this: a correlation above 0.32 between two factors suggests that an oblique rotation is appropriate.
05Reading the output
The top of the results shows the Bartlett banner, then the number of factors extracted, the percentage of variance explained and the KMO value with its label. Up to four tabs follow.
Factor loadings
A heatmap shows the loading of every item on every factor, shaded by size. The same display options as in PCA apply. Dominant factor lists each item under the factor where it loads most strongly, and By threshold lists it under every factor where its loading reaches the minimum, which is 0.35 to begin with.
Loading sizes
Absolute loading
How the module describes it
0.60 or more
Strong
0.40 to 0.59
Moderate
0.30 to 0.39
Weak
Below 0.30
Negligible
A well-behaved item loads 0.40 or more on one factor and below 0.30 on all the others. An item that loads 0.30 or more on two factors is a cross-loader, and the interpretation text names it as ambiguous.
Variance explained
The table gives the sum of squared loadings, the percentage of variance and the cumulative percentage for each factor, with a bar chart. These percentages are lower than the ones PCA gives for the same items, because only shared variation is counted. The module calls 60% or more a reasonable result, and 40% to 60% a moderate one.
Communalities
Communality and uniqueness are listed for each item. Values below 0.30 are highlighted. A low communality means that the factors explain little of the item.
Factor correlations
This tab appears only with Oblimin. It lists the correlation between each pair of factors.
Next step: check each factor with alpha. Once the structure looks sensible, calculate Cronbach's alpha separately for the items on each factor, and report these values alongside the loadings.
06How to run it in AnalyZ Solutions
Open Scale & Reliability, then choose Exploratory Factor Analysis.
Select the variables Choose three or more numeric items, such as rating items from one questionnaire.
Set the number of factors Enter how many factors to extract. It must be smaller than the number of items. The starting value is 2.
Choose the rotation Varimax (orthogonal), Oblimin (oblique) or None (unrotated).
Choose the extraction method Principal Axis Factoring (PAF) or Principal Components (PC-based).
Apply a subsample filter if needed Use it to run the analysis for one group, for example to see whether the structure holds for women and men separately.
Click Run EFA Check the Bartlett banner and KMO first, then read the loadings, the variance, the communalities and, for Oblimin, the factor correlations. Export the tables as CSV, Excel or Word, or add the result to AnalyZense or to a Data Canvas story.
The module extracts exactly the number of factors you ask for. It does not choose the number for you. Compare two or three plausible solutions, for example two, three and four factors, and keep the one that is the clearest and makes the most sense. If you remove items after a first run, remove them one at a time, starting with the worst, because the loadings of the remaining items change each time.
When your data come from a survey with sampling weights. With a survey weight set, extraction and rotation work on the survey-weighted correlation matrix, so the loadings, communalities, variance explained and KMO reflect the weights. Bartlett's test uses the effective sample size implied by the weights, which is smaller than N, so it is more cautious. Weighted N appears next to N.
07Interpreting the result
The figures below are invented to show the steps, and they follow the results tables in AnalyZ Solutions, with a few columns left out to save space. They come from the health worker survey described earlier: 300 people, ten items rated from 1 to 5, three factors, principal axis factoring and Varimax rotation.
Bartlett's test gives chi-square(45) = 742.5, p < 0.001, and the KMO value is 0.769, labelled middling.
Factor loadings, communalities and uniqueness (Varimax rotation)
Variable
F1
F2
F3
Communality
Uniqueness
I enjoy my work
0.781
-0.035
0.197
0.650
0.350
My work feels meaningful
0.700
-0.000
0.199
0.529
0.471
I am proud of my team
0.617
-0.050
0.226
0.434
0.566
I plan to stay here
0.463
0.258
0.073
0.286
0.714
Clear instructions
0.221
0.082
0.655
0.485
0.515
Supervisor is available
0.227
0.081
0.686
0.528
0.472
I get regular feedback
0.130
0.019
0.687
0.489
0.511
Enough staff on shift
0.088
0.630
0.068
0.409
0.591
Enough time per patient
-0.091
0.663
0.088
0.455
0.545
Manageable caseload
0.061
0.666
-0.012
0.448
0.552
Variance explained
Factor
SS loadings
% Variance
Cumulative %
F1
1.832
18.32
18.32
F2
1.364
13.64
31.96
F3
1.519
15.19
47.15
Figure 2. Eigenvalues of the correlation matrix for the health worker survey. Three are above 1 and the fourth drops to 0.71
Reading it step by step
Check the suitability Bartlett's test is significant and KMO is 0.769. The items are correlated enough for a factor analysis.
Decide on the number of factors Three eigenvalues are above 1 (3.09, 1.88 and 1.31), and the fourth drops to 0.71. That agrees with the three ideas the questionnaire was written around, so a three-factor solution is a sensible choice.
Read the loadings Factor 1 collects the four items about meaning (loadings 0.46 to 0.78). Factor 3 collects the three supervision items (0.66 to 0.69). Factor 2 collects the three workload items (0.63 to 0.67). Every item loads at 0.40 or more on its own factor and below 0.30 on the others, so there are no cross-loaders. The factors can be named Meaning in work, Supervision and Workload.
Look at the communalities Nine items are between 0.41 and 0.65. I plan to stay here has 0.286, below 0.30, and is highlighted. Its loading on Factor 1 is 0.46, the weakest of the four. Intention to stay probably depends on much more than how meaningful the work feels, such as pay and family circumstances.
Look at the variance The three factors explain 47.1% of the variation in the items, which the module calls moderate. The rest is variation specific to each item and measurement error. A figure of this size is common for short rating scales.
Check each group with alpha Alpha is 0.75 for the four meaning items, 0.75 for supervision and 0.69 for workload. Workload falls in the questionable band, and has only three items. Without I plan to stay here, the three meaning items give 0.77, so that item may be worth dropping or analysing on its own.
Figure 1. The structure found in the worked example. Each line shows the largest loading of an item, and a thicker line means a stronger loading
In plain words
A factor analysis of ten items from 300 health workers found three factors, which together explain 47% of the variation in the answers: meaning in work, supervision and workload. All items loaded clearly on the factor they were written for. One item, I plan to stay here, was only weakly explained by the factors. This analysis describes the structure in this sample. A confirmatory analysis in a new sample would be needed to confirm it.
08Common mistakes
Treating the result as proof. EFA is exploratory. A structure found in one sample should be confirmed in another.
Choosing the number of factors by the Kaiser rule alone. Look at the scree plot, compare solutions and consider the meaning of each factor.
Using Varimax when the factors are clearly related. The forced independence distorts the loadings. Check the factor correlations from an Oblimin run.
Dropping many items at once. Each removal changes the loadings of the others. Remove one at a time and rerun.
Ignoring cross-loaders and low communalities. They signal items that are ambiguous or poorly explained, and they weaken the subscales built from them.
Forgetting wording effects. Negatively worded items sometimes form a factor of their own that reflects the wording and not the content. Check how reverse-worded items behave.
Naming factors too early. Read the content of every item on a factor before you choose a label.
Too little data. Small samples give unstable loadings. Aim for at least five respondents per item, and more when you can.
Frequently Asked Questions
What is the difference between EFA and PCA?
EFA models the variation shared between items and assumes hidden factors cause it. PCA models all the variation and just reshuffles it into components. Use EFA to find and name dimensions, and PCA to reduce the number of variables.
How many factors should I extract?
Use several guides together: the number of eigenvalues above 1, the bend in the scree plot, your theory and how readable each solution is. Try neighbouring numbers and keep the clearest.
Should I use Varimax or Oblimin?
If you expect the factors to be related, as with most attitude and wellbeing scales, use Oblimin and read the factor correlations. If those are all low, Varimax gives about the same result and is easier to report.
What loading counts as high enough?
The module treats 0.40 as the point where a loading becomes moderate and 0.60 as strong. A common aim is 0.40 or more on one factor and below 0.30 on the others. The cut-offs are conventions and can be stricter when the sample is small.
What should I do with a cross-loading item?
Look at its wording first. If it is ambiguous, rewriting or removing it usually helps. Remove only one item at a time and rerun, since the other loadings change.
Can I use yes or no items?
Not well. The analysis relies on Pearson correlations, which are distorted for items with only two answers. Ratings with five or more options are much better.
Can I use EFA and then confirm the structure?
Yes, and that is the usual route. Explore in one sample and confirm in a different one with a confirmatory factor analysis in specialist software.
Find the structure behind your items
Open Scale & Reliability in AnalyZ Solutions. Free to sign up, free to analyze.