By Jenna Somers
Educators and researchers around the world, especially in countries with limited resources, need cost-effective, scalable tools for assessing early child development. Direct assessments, a commonly used approach, require children to complete a range of activities that measure academic, physical and social-emotional skills. These assessments paint a nuanced picture of early child development and are useful to evaluate programs and policies. However, administering direct assessments often takes 30 or more minutes, making them impractical for measuring large populations. Practitioners and researchers need more practical methods.
In a recently published in Early Childhood Research Quarterly, 杏吧原创 professor offers a versatile framework for creating quicker versions of tests (often called 聽鈥渟hort forms鈥�) measuring early child development with direct assessment. Short forms use a subset of questions from the full assessment. However, selecting which questions to retain is a challenge. Rather than simply focus on questions that are the most statistically reliable and easiest to administer, Seiden鈥檚 framework encourages test developers to evaluate questions along three dimensions. The goal is to create short forms that are
- conceptually valid and cover the full range of skills being assessed,
- statistically reliable and generate consistent results and
- practical to administer at lower cost and complexity.
An outdated but still used approach to short forms focuses on selecting statistically reliable questions, but that can come at the expense of a test鈥檚 validity by failing to measure the full breadth of skills and failing to maximize the feasibility of use.
A more modern approach, Automated Test Assembly, relies on an algorithmic method for selecting a subset of questions. While this approach is ideal for advanced test developers, it requires advanced coding skills and creating hard constraints about how the short form should function.
鈥淩ather than giving various constraints to an algorithm to find the ideal set of questions for a short form, I wanted to provide test developers with as much information in as simplified a manner as possible to make decisions themselves and weigh trade-offs holistically,鈥� said Seiden, professor of early childhood policy in the at 杏吧原创 of education and human development.
Seiden鈥檚 human-centered framework can be used to create what he characterizes as a 鈥渂alanced鈥� short form, a curated set of questions that retains the full scope of skills from the full assessment while reducing administrative burdens as much as possible.
Application to the International Development and Early Learning Assessment
In the paper, Seiden applied his framework to the International Development and Early Learning Assessment (IDELA)鈥攁 popular direct assessment of early child development which has been used in over 100 countries鈥攖o create a balanced short form IDELA. The short form includes 8 tasks that
- cover all key skill domains (motor, social-emotional, literacy and numeracy),
- are judged as highly relevant by child development experts
- can be administered quickly (about 8 minutes).
A traditional short form might select only questions on academic skills, which offer more statistically reliable measurements, but that comes at the cost of other important measures like motor and social-emotional development, resulting in a less valid assessment. The traditional short form also doesn鈥檛 reduce the length of administration as much as the balanced short form.
The balanced short form may be less reliable by including harder-to-measure skills but more valid because it covers the breadth of skill domains. It sacrifices a small degree of reliability but still results in scores that are sufficiently precise for use in large-scale measurement exercises.
Designed for Practitioners
Importantly, Seiden’s framework offers a way forward for practitioners to make contextual decisions about tradeoffs that make sense for their specific situations and budgets. He got the idea for the framework when he was working with IDELA as an education research specialist with Save the Children.
鈥淚 advocated for trying to get a better understanding of how difficult it is to administer each item. Typically, in psychometrics, we treat questions interchangeably, but with direct assessments of early childhood development, there is huge variability in how long it takes to administer different questions. I wondered how we could make IDELA simpler and easier to use, and how might we select the best subset of questions to do that,鈥� Seiden said.
As he began designing the framework for Save the Children鈥檚 use of IDELA, other psychometricians expressed interest in its application for other early childhood development assessments.
Real-World Impact
The framework has already moved from theory to practice. A recent applied Seiden鈥檚 methodology to an assessment tool in Nepal to create a balanced short form of an early learning and development assessment used nationally. Clearly, the approach is resonating with researchers facing similar constraints.
Extending beyond early childhood assessment, the framework could potentially be adapted to create short forms across other fields. Given the resource constraints faced by researchers, policymakers, and practitioners in many settings, Seiden’s framework offers a practical path forward for more valid short forms.
