Sistema de ayuda para la calibración de ítems por el procedimiento basado en el juicio de expertos

  1. Concepción Presedo 1
  2. Anaje Armendariz 1
  3. Javier López-Cuadrado 1
  4. Tomás A. Pérez 1
  1. 1 Universidad del País Vasco/Euskal Herriko Unibertsitatea

    Universidad del País Vasco/Euskal Herriko Unibertsitatea

    Lejona, España


Revista Internacional de Tecnologías en la Educación

ISSN: 2386-8384

Year of publication: 2015

Volume: 2

Issue: 1

Pages: 1-15

Type: Article

More publications in: Revista Internacional de Tecnologías en la Educación


Evaluation is a key issue in learning and item banks are very common resources used to perform activities related to it. Professional evaluation deals with banks that are calibrated, which means that the parameters of the items (for example, their difficulty) have previously been estimated. One option for item calibration consists in gathering the assessments of a panel of experts and using them to determine the values of the item parameters. This paper describes CALLIE-PRO: a system that customizes and automates the process of item calibration based on expert judgment. It guides the user by recommending a choice for every decision to be made, adapting to changes in the specification, and warning of non advisable or incorrect choices. It also creates an e-learning content package that depends on the process specification and can be included in a learning management system such as, for example, Moodle. This package is responsible for gathering the assessments of the experts via the internet and generating an executable workflow that will include the activities to be performed, which are mainly analyses and calculi, by using all gathered data. Finally, the system provides estimates for the parameters (difficulty) of the items.