Multiple keys (Advanced) - In preparation

Source: vignettes/multiple_keys.Rmd

Motivation

A key is a set of difficulty estimates linked to items from one or more instruments. As more data become available, the key may be updated periodically to incorporate the additional information. This results in multiple versions of the key. Although keys are designed to produce D-scores on the same general scale, each key defines a slightly different scale. As a result, the same set of child responses may yield different D-scores depending on which key is used.

For new data, the most recent default key is usually recommended. However, if strict comparability with earlier analyses is important, it may be preferable to use an older key.

This vignette explains the policy for setting default keys and demonstrates how to compare D-scores across different keys. Because this is an advanced topic, it assumes a basic understanding of the D-score calculation process. If you are new to the D-score methodology, we recommend reviewing the introductory vignettes before proceeding.

Relation to the D-score

  • The D-score depends on the difficulty parameters defined by a key.
  • Updating a key updates those parameters and can slightly shift scores.
  • Differences are typically small but may matter for strict comparability.
# Example (pseudo-code): compute a D-score given a key
# library(dscore)
# d1 <- dscore(items = x, key = "gsed2406")
# d2 <- dscore(items = x, key = "gsed2510")
# cbind(d1, d2, diff = d2 - d1)

Default vs. Alternative Keys

Policy for default key selection

Describe how the default key is chosen (e.g., most recent stable release) and how often it is updated. State where users can find the current default in the package documentation.

When to use an older key

  • Ensure comparability with past analyses or publications.
  • Reproduce earlier results precisely.
  • Regulatory or reporting requirements that fix a specific key.
# Check the package default key (example; replace with your function)
# dscore::get_default_key()

Working with Keys in Practice

List available keys

Show users how to enumerate supported keys.

# Example (replace with the actual function name)
# keys <- dscore::list_keys()
# keys

Set the key

Demonstrate how to select a specific key in your workflow.

# Example usage
# ds <- dscore(data, key = "gsed2406")

Change the default (session or project)

Explain global vs. local configuration, and how to set a default key for a session or project.

# Session-level default (illustrative; adapt to your API)
# options(dscore.default_key = "gsed2510")

# Confirm
# getOption("dscore.default_key")

Comparing Keys

Impact on D-scores

Show the same dataset scored under two keys and compare.

# Example skeleton
# ds1 <- dscore(dat, key = "gsed2406")
# ds2 <- dscore(dat, key = "gsed2510")
# comp <- transform(dat, d1 = ds1$D, d2 = ds2$D, diff = ds2$D - ds1$D)
# head(comp)

Diagnostics

Provide simple diagnostics to understand differences (plots/tables).

# Histogram of differences
# hist(comp$diff, main = "D-score differences (new - old)", xlab = "Difference")

# Correlation and summary
# cor(comp$d1, comp$d2, use = "complete.obs")
# summary(comp$diff)

Stability over time

Discuss empirical evidence (e.g., median absolute difference, 95% intervals) showing that differences across keys are generally small.

# Example summary
# mad <- median(abs(comp$diff), na.rm = TRUE)
# quant <- quantile(comp$diff, probs = c(0.025, 0.5, 0.975), na.rm = TRUE)
# list(median_abs_diff = mad, quantiles = quant)

Caveats and Limitations

  • Instrument versions/translations: published item orders may differ, which can limit the feasibility of a single “native” ordering across locales.
  • Mixing keys: avoid scoring datasets with different keys when analyses are compared directly across groups or time.
  • Reporting: always record and report the key identifier used.

Recommendations for Users

  • Prefer the most recent default key for new studies.
  • Use an older key when exact comparability is required.
  • In publications and reports, cite the key (name and version/date).

Worked Examples

Example 1: Score with the default key 

# dat <- read.csv("your_data.csv")
# ds_default <- dscore(dat) # implicit default key
# head(ds_default)

Example 2: Score with a specific older key and compare 

# ds_old <- dscore(dat, key = "gsed2406")
# ds_new <- dscore(dat, key = "gsed2510")
# delta <- ds_new$D - ds_old$D
# summary(delta)

Example 3: Visualize differences 

# plot(ds_old$D, ds_new$D,
#      xlab = "Old key D-score",
#      ylab = "New key D-score",
#      main = "D-scores under two keys")
# abline(0, 1, lty = 2)

Reproducibility Notes

  • Always record: key, package version, date, random seed (if any).
  • Provide session info in appendices.
## R version 4.5.1 (2025-06-13)
## Platform: aarch64-apple-darwin20
## Running under: macOS Tahoe 26.0.1
## 
## Matrix products: default
## BLAS:   /Library/Frameworks/R.framework/Versions/4.5-arm64/Resources/lib/libRblas.0.dylib 
## LAPACK: /Library/Frameworks/R.framework/Versions/4.5-arm64/Resources/lib/libRlapack.dylib;  LAPACK version 3.12.1
## 
## locale:
## [1] C.UTF-8/C.UTF-8/C.UTF-8/C/C.UTF-8/C.UTF-8
## 
## time zone: Europe/Amsterdam
## tzcode source: internal
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## loaded via a namespace (and not attached):
##  [1] digest_0.6.37     desc_1.4.3        R6_2.6.1          fastmap_1.2.0    
##  [5] xfun_0.53         cachem_1.1.0      knitr_1.50        htmltools_0.5.8.1
##  [9] rmarkdown_2.29    lifecycle_1.0.4   cli_3.6.5         sass_0.4.10      
## [13] pkgdown_2.1.3     textshaping_1.0.3 jquerylib_0.1.4   systemfonts_1.2.3
## [17] compiler_4.5.1    tools_4.5.1       ragg_1.5.0        evaluate_1.0.5   
## [21] bslib_0.9.0       yaml_2.3.10       jsonlite_2.0.0    rlang_1.1.6      
## [25] fs_1.6.6          htmlwidgets_1.6.4