What is internal consistency and how can you use it to improve your skills assessments?

What is internal consistency?

Internal consistency is a term that describes how well the different parts of something align or fit together. The term is often used in the context of organizations and testing. 

Internal consistency within an organization could refer to how well its code of conduct, corporate policies, and leaders’ behaviors line up and reinforce one another. 

In the context of testing and assessments, internal consistency refers to how well test items or sets of test items measure what they’re supposed to. For instance, questions in a communication skills test intended to measure active listening skills should all reliably measure active listening skills. 

Tests with weak internal consistency have questions that don’t correlate well and may measure different aspects. The result? Test-taker's scores may vary significantly across these questions, even though they’re meant to measure the same skill.  

Meanwhile, tests with strong internal consistency have strong correlations between items – and these usually give more consistent results. In classical test theory, this consistency signals reliability; the theory states that in reliable tests, a candidate’s “true score” for a skill (like reading comprehension) shows up consistently across similar questions.

Examples of internal consistency in assessments

To see internal consistency in action, check out the examples table below.

The importance of internal consistency

Internal consistency is important because it helps ensure all parts of a process or system work harmoniously toward the same goal. In assessments, when all test items measure the same skill, the results are more reliable and useful.

How internal consistency leads to better hiring and decision-making

Internal consistency does three critical things for your company’s hiring process:

1. Reduces hiring errors

Inconsistent test results lead to confusion during hiring and hiring errors. Weak internal consistency could show a candidate is a strong fit based on one set of questions while flagging concerns in another. 

For example, if one section of a leadership skills test indicates leadership potential, but another section contradicts it, the inconsistency can confuse you. You could dismiss a strong candidate who didn’t perform well because the results obscure their fit for the role. Alternatively, you could hire someone who doesn’t have what it takes to succeed in the role. 

Reliable testing creates confidence in recruiting and minimizes poor hiring decisions.

2. Supports talent development

Internal consistency is also vital when using assessments for talent management and development – whether you're evaluating new hires or tracking current employees’ growth and progress. 

This is because test discrepancies and unreliable tests can cloud results. Say you’re testing your employees’ skills with a specific software before adopting the software across the company. If your tests aren’t reliable, you won’t know the best way to train employees to use the software. 

In contrast, high internal consistency reliability in assessments allows talent teams to gain meaningful insights, identify areas for improvement, create effective learning programs, and ensure people are developing the right skills.

3. Gives you trust in test results

Whether you’re hiring for technical skills, assessing cultural fit, or evaluating employee growth, you need test results you can count on. Internal consistency is one psychometric property that helps guarantee reliable test results. 

The bonus? Candidates can also rest assured that you’re testing them fairly and accurately. 

Ways to measure internal consistency

There are several ways to evaluate internal consistency reliability, and using them together is a good idea for creating and maintaining internal consistency.

1. Cronbach’s alpha

Described by Lee Cronbach in 1951 as the ”coefficient alpha,” Cronbach’s alpha is a formula used in statistical tests that helps pin down how closely related different questions are as a group. This formula can look at all the items in the test or specific “subscales” or sets of test items. Scores run from 0 to 1. 

As a rule of thumb, you want your tests to have a high Cronbach’s alpha of around 0.70. This indicates strong internal consistency. A weak Cronbach’s alpha would be 0.50 or less. 

However, you don’t want too high internal consistency, such as a Cronbach’s alpha of 0.90. That means some questions might be redundant and will offer little additional insight.

2. Split-half reliability

Split-half reliability divides a test or set of similar test items into two halves and compares the scores from each. If the scores are similar, the test might have strong internal consistency. 

This method helps ensure that different test parts contribute equally to the overall result and there’s no data skew from each half.

3. Item response theory (IRT)

Item response theory (IRT) is a way to evaluate the performance of individual questions on a test. Instead of assuming each question is equally valuable, IRT looks at questions’ difficulty, discrimination (how well the question separates high and low performers), and pseudo-guessing parameter (how likely it is that a test-taker will answer correctly by guessing).

IRT is particularly useful for tests with varying difficulty levels, like skills testing. If you use IRT in a structured way along with the two methods above, you can improve internal consistency in your assessments.

To learn more about how TestGorilla uses internal consistency, visit our science page. You can also visit our blog to learn about various assessment-related topics. 

FAQs

How do I know if a test has strong internal consistency?

You can evaluate a test’s internal consistency by looking at its Cronbach's alpha score, which should be above 0.70 for reliable assessments.

What does a low internal consistency mean?

If an assessment has low internal consistency, it contains some questions that don’t measure what the overall test – or set of related test items – is designed to measure. For instance, if a test is supposed to measure business ethics but some of its questions focus on unrelated topics (like marketing strategy), the test will have a lower internal consistency.