5.2 Modelling strategies

The analytic challenge is twofold:

• to find a subset of items that form a scale;
• to find a subset of equate groups with items similar enough to bridge instruments.

Note that both subsets are related, i.e., changing one affects the other. Thus, we cannot first identify items and then equate groups, or first identify equate groups followed by the items. Rather we need to find the two subsets in an iterative fashion, primarily by hand. This section describes some of the modelling issues the analyst needs to confront.

In general, we look for a final model that

• preserves the items that best fit the Rasch model;
• uses active equate groups with items that behave the same across many cohorts and instruments;
• displays reasonable age-conditional distributions of the D-scores;
• has difficulty estimates that are similar to previous estimates.

The modelling strategy is a delicate balancing act to achieve all of the above objectives. Particular actions that we could take to improve a given model are:

• remove bad items;
• inactivate bad equate groups;
• break up bad equate groups;
• move items from one equate group to another;
• create new equate groups;
• remove entire instruments;
• remove persons;
• remove studies.

In order to steer our actions, we look at the following diagnostics (in order of importance):

• quality of equate groups (both visually and through infit);
• plausibility of the distribution of the D-score by age per study;
• correspondence of difficulty estimates from published (single study) Dutch data and the new model;
• infit of the items remaining in the model.

Various routes are possible and may result in different final models. The strategy adopted here is to thicken active equate groups by covering as many studies as possible, in the hope of minimizing the number of active equates needed.