Calibrating a Numerical Ability Selection Test for Job Candidates with Item Response Theory

Item Response Theory (mirt) for Human Resource Management

Field: Human Resource ManagementLevel: Doctoral dissertation (S3)Method: Item Response Theory (mirt)

0 views

The goal is to calibrate selection test items and estimate applicants' latent ability; the data are synthetic and for illustration only. The two-parameter (2PL) Item Response Theory model calibrates each test item by its discrimination and difficulty, then estimates each examinee's latent ability (theta) and its precision. It suits theses in education, psychometrics or certification that analyse the quality of multiple-choice or right-wrong test items.

Calibrating a Numerical Ability Selection Test for Job Candidates with Item Response Theory: result sheet

Project info

Field of studyHuman Resource Management
LevelDoctoral dissertation (S3)
Method typeSEM and Psychometrics
Estimated build time± 7 working days
Toolsmirt::mirt, mirt::coef, mirt::M2, mirt::itemfit, mirt::fscores, mirt::empirical_rxx, mirt::testinfo, ggplot2
PriceUS$31 - US$194

Research goal

The goal is to calibrate selection test items and estimate applicants' latent ability; the data are synthetic and for illustration only.

Variables

Role	Variable	Unit
Main	Applicant numerical ability	theta
Supporting 1	Grade point average	score
Supporting 2	Applicant age	years

Example result

0.377M2 p-value
0.011RMSEA (M2)
0.798Empirical reliability
1.563Mean discrimination a
0Misfit items (S-X2)
0.304SD-effect Grade point

The 2PL model for 14 items fits the data (M2 RMSEA = 0.011) with empirical reliability 0.798. Discrimination ranges 0.76 to 2.74 and difficulty -1.95 to 1.96; 0 item(s) flagged by S-X2. Applicant numerical ability theta is significantly related to Grade point average (0.304 per 1 SD, p < 0.001).

Computed from illustrative synthetic data, not from a real study.

About Item Response Theory (mirt)

The two-parameter (2PL) Item Response Theory model calibrates each test item by its discrimination and difficulty, then estimates each examinee's latent ability (theta) and its precision. It suits theses in education, psychometrics or certification that analyse the quality of multiple-choice or right-wrong test items.

Steps

1Prepare dichotomous response data (1 = correct, 0 = wrong), one column per item
2Calibrate a unidimensional 2PL model with mirt::mirt
3Extract discrimination (a) and difficulty (b) with coef(IRTpars = TRUE)
4Assess model fit with the M2 statistic and RMSEA, and item fit with S-X2 (itemfit)
5Estimate examinee theta (EAP) with fscores and compute the empirical reliability
6Plot item characteristic curves and the test information function, then relate theta to examinee characteristics

Assumptions to check

Unidimensionality: items measure one main latent ability
Local independence: responses are independent given theta
The probability of a correct answer rises monotonically along a logistic curve
The sample is large enough for calibration (usually several hundred examinees)

Reading and reporting the results

Discrimination a above about 0.65 means an item separates examinees well, and b is the theta at which the chance of a correct answer is 50% (negative means easy). Report M2, RMSEA, misfitting items, the empirical reliability and the theta range with the highest test information, where the test is most precise.

Example R code

A short starting script with this project's variable names. The delivered version is adapted to your data and fully commented.

Show code

# Contoh kode R: Item Response Theory (mirt)
suppressPackageStartupMessages({
  library(ggplot2)
  library(patchwork)
  library(mirt)
})
set.seed(42)
# ---- Gaya grafik ----
pal <- c("#1f6feb", "#e8590c", "#2f9e44", "#ae3ec9", "#f59f00")
warna <- list(bg = "#f4f6fa", panel = "#ffffff", teks = "#1f2933", redup = "#5f6b78", grid = "#e3e8ee", bar = "#1d3557")
tata <- 1
theme_set(theme_minimal(base_size = 11) + theme(legend.position = "bottom"))

# ---- Data sintetis: 14 butir dikotomus (benar = 1, salah = 0) dari model 2PL ----
n <- 200 + 200                                                # IRT butuh sampel cukup besar
nb <- 14
z1 <- rnorm(n); z2 <- rnorm(n)
g1 <- runif(1, 0.25, 0.45); g2 <- runif(1, 0.1, 0.3) * sample(c(-1, 1), 1)
th <- g1 * z1 + g2 * z2 + rnorm(n, 0, sqrt(1 - g1^2 - g2^2))   # kemampuan laten (theta), varians 1
a_sb <- runif(nb, 0.7, 2.2)                                     # daya beda butir
b_sb <- sort(runif(nb, -2, 2))                                  # tingkat kesulitan butir
pr <- plogis(outer(th, b_sb, "-") * rep(a_sb, each = n))        # peluang menjawab benar
d <- data.frame(grade_point_average = round(50 + 10 * z1, 2), applicant_age = round(30 + 6 * z2, 2))
jw <- matrix(rbinom(n * nb, 1, pr), n)
colnames(jw) <- sprintf("item%02d", 1:nb)
d <- cbind(d, jw)
write.csv(d, "data_contoh.csv", row.names = FALSE)

# ---- Kalibrasi model 2PL ----
butir <- d[, sprintf("item%02d", 1:nb)]
fit <- mirt(butir, 1, itemtype = "2PL", verbose = FALSE)
par <- coef(fit, IRTpars = TRUE, simplify = TRUE)$items         # a = daya beda, b = kesulitan
print(round(par[, c("a", "b")], 3))

# ---- Kecocokan model dan butir ----
m2 <- M2(fit)                                                   # statistik M2 dan RMSEA
print(m2)
itf <- itemfit(fit, fit_stats = "S_X2")                         # S-X2 per butir
print(itf)
misfit <- sum(itf$p.S_X2 < 0.05, na.rm = TRUE)

# ---- Skor kemampuan (EAP) dan reliabilitas empiris ----
fs <- fscores(fit, method = "EAP", full.scores.SE = TRUE)
rel <- unname(empirical_rxx(fs))
d$applicant_numerical_ability <- fs[, 1]                                              # theta hasil estimasi
reg <- lm(applicant_numerical_ability ~ grade_point_average + applicant_age, data = d)                    # kaitan theta dengan karakteristik peserta (b_x1 = per 1 SD X1)
print(summary(reg))
b_x1 <- coef(reg)[["grade_point_average"]] * sd(d$grade_point_average); p_x1 <- summary(reg)$coefficients["grade_point_average", 4]


# ---- Grafik ----
tg <- seq(-4, 4, length.out = 121)
pb <- probtrace(fit, matrix(tg))                                # kolom: itemXX.P.0, itemXX.P.1
pk <- pb[, grep("P.1$", colnames(pb))]
dc <- data.frame(theta = rep(tg, nb), p = as.vector(pk), butir = rep(sprintf("item%02d", 1:nb), each = length(tg)),
                 kesulitan = rep(par[, "b"], each = length(tg)))
p1 <- ggplot(dc, aes(theta, p, group = butir, colour = kesulitan)) +
  geom_hline(yintercept = 0.5, linetype = "dashed", colour = warna$redup) +
  geom_line(linewidth = 0.75, alpha = 0.9) +
  scale_colour_gradient(low = pal[1], high = pal[2], name = "Difficulty b") +
  labs(title = "Item characteristic curves (2PL)", x = "applicant_numerical_ability", y = "P(correct)")
ti <- seq(-3, 3, length.out = 91)
info <- testinfo(fit, matrix(ti))
di <- data.frame(theta = ti, info = info, se = 1 / sqrt(info))
k_se <- max(di$info) / max(di$se)                               # skala sumbu kedua untuk SE
p2 <- ggplot(di, aes(theta)) +
  geom_area(aes(y = info), fill = pal[3], alpha = 0.3) +
  geom_line(aes(y = info, colour = "Information"), linewidth = 1) +
  geom_line(aes(y = se * k_se, colour = "Standard error"), linewidth = 0.9, linetype = "dashed") +
  scale_colour_manual(values = c(Information = pal[3], `Standard error` = pal[4]), name = NULL) +
  scale_y_continuous(sec.axis = sec_axis(~ . / k_se, name = "SE")) +
  labs(title = "Test information and SE", x = "applicant_numerical_ability", y = "Information")
p3 <- ggplot(d, aes(grade_point_average, applicant_numerical_ability)) +
  geom_point(colour = pal[1], alpha = 0.45, size = 1.4) +
  geom_smooth(method = "lm", formula = y ~ x, colour = pal[2], fill = pal[2], alpha = 0.18) +
  labs(title = "Estimated theta (EAP) vs covariate", x = "grade_point_average", y = "Estimated theta")
p_hasil <- if (tata == 1) p1 | (p2 / p3) else p1 / (p2 | p3)

print(p_hasil)

The project file you can download

The original project file is available. It is not just the picture above: you get the working R project. It can be downloaded after payment, and we send it by WhatsApp or email.

ZIP: analisis.R + data_contoh.csv + BACA-DULU.txt

What is inside the ZIP

The complete analysis script (analisis.R), commented step by step, ready to run in RStudio.
A sample dataset (data_contoh.csv) so the script runs straight away.
A short guide (BACA-DULU.txt): which packages to install, how to run it and how to swap in your own data.

Packages and prices

Choose how much help you need. Each higher package includes everything in the lower ones. Prices exclude 11% VAT.

Package 1

Code and Explanation

US$31

excl. 11% VAT

Duration: 7 working days

Output: R files + result figures

Deliverables

Fully commented R code (.R) for Item Response Theory (mirt)
Example data and step-by-step run instructions
Ready-to-use result figures and metrics table
Short explanation of the method, its assumptions and how to read the results

Choose Package 1

Package 2 · most chosen

Your Data Analysis + Interpretation

US$88

excl. 11% VAT

Duration: 13 working days

Output: Code + analysis + interpretation notes

Includes all of Package 1, plus

Adapting the code to your own research dataset (CSV or Excel)
Assumption checks, result validation and data cleaning
Tables and figures formatted for your manuscript
Interpretation notes for the methods and discussion sections, for you to write up in your own words

Choose Package 2

Package 3

Full Guidance

US$194

excl. 11% VAT

Duration: 18-21 working days

Output: Code + analysis + guidance until ready to present

Includes all of Packages 1 & 2, plus

Comparison against one alternative method with the reasoning for the choice
Two 60-minute online coaching sessions so you fully understand the code and results
Revisions following your supervisor feedback (up to 3 rounds)
Practice Q&A to prepare for your seminar and defence
Full documentation (README, R package list (sessionInfo)) so the results can be reproduced

Choose Package 3

Payment: 50% down payment before work starts, balance after the deliverables are handed over
Tax: Prices exclude 11% VAT
Data and academic integrity: Results are computed from your own research data; you write and stand behind your manuscript, we help with the analysis and your understanding of it
Revisions: Up to 2 revisions are included in the price (Package 3: 3 rounds)

Frequently asked questions

Do I get the real project file?

Yes. The original project file is a ZIP containing the complete analisis.R script, a data_contoh.csv sample dataset and a BACA-DULU.txt guide. It can be downloaded after payment, and we send it by WhatsApp or email. For your own data (Package 2 and 3) the script is adapted to your dataset.

Is this suitable for a Doctoral dissertation (S3)?

Yes. Item Response Theory (mirt) is commonly used at this level in Human Resource Management. We adapt the analysis to your research question, your own data and your supervisor's requirements.

Is the result sheet real data?

No. The sheet is computed from illustrative synthetic data so you can see the kind of output R Studio gives. For your order we run the same method on your own data.

Will I understand the code?

That is the point of the packages. Every script is commented, and Package 3 includes two coaching sessions plus practice questions so you can explain the method and the results at your seminar and defence.

How is the price decided?

Each project has three packages: Package 1 (code and explanation) US$31, Package 2 (your data analysis and interpretation) US$88 and Package 3 (full guidance) US$194, excluding 11% VAT. Larger datasets, extra methods or tight deadlines adjust the quote after a short consultation.

Similar projects

See all

PreviousMeteorological Factors of Whirlwind Occurrences per Regency in Central Java

NextImpact of a Mentoring Programme on Young Employees' Salary Growth: Propensity Score Matching

The result sheet uses illustrative synthetic data. R is a programming language of the R Foundation project; RStudio and Posit are trademarks of Posit PBC.

Back to R Studio portfolio

Memuat