Expertum - EXperimental Project on Educational Reckoning and Transition from University to job Market

Project Description

This website includes the information on the research outcomes of a group of scholars who conducted a set of experiments to test the possibility of using computer-assisted techniques for the collection of data on course quality evaluation by students and the early career of graduates. For practical reasons, the project was partitioned into two main lines, one concerned with teaching evaluation and the other with graduates’ career follow up.

The experiments have been held independently both at Padua University and Florence University and were concerned with the utilization of CASI (Computer Assisted Self-administered Interviewing), CAWI (Computer-Assisted Web-based Interviewing), CATI (Computer Assisted Telephone Interviewing) and IDTV (Interactive Data Telephone and Video) techniques, instead of paper and pencil data collection mode.
 
The project was commissioned by the Ministry of University and Research – Observatory for the evaluation of the University System in 2000.

Survey methodology

  1. Project on teaching evaluation. The data collection of students’ evaluations on teaching quality is a self-administered type of data collection mode. The techniques that have been tested used the CAWI and CASI systems. Both systems were based on the spontaneous participation of students to the experiment after the same students already delivered to the academic organs charged of students’ opinions collection their paper and pencil evaluation. This way, the experiment did not interact with the normal activities within the involved universities.
  2. Project on graduates’ follow up. The experiment consisted in testing the validity of a CATI – Computer Assisted Telephone Interviewing system for graduates’ follow up six months after graduation. The data could be linked to the so-called AlmaLaurea survey held by the Universities of Padua and Florence, among the others, on their own graduates. The IDTV technique, based on the interaction between telephone and video, was considered cumbersome to conduct and eventually selective of respondents and was not experienced. The project wanted to test both the viability of the CATI surveys on graduates, test some new software for computer-assisted interviewing and also evaluate the possibility to obtain data from experts on employment and jobs as an alternative source of graduates’ competence.

Scientific results

The Research group presented its results at the Ministry of University and Research in Rome.
The research produced:
  1. Models and a suggestion of a new questionnaire for the collection of data on university teaching quality to be used in surveys with attending and non-attending students,
  2. Various versions of the teaching quality questionnaire distinct according to ways of putting questions and scales for answer collection,
  3.  Ways of analyzing the obtained responses in a longitudinal fashion,
  4. Models and a suggestion of a new questionnaire for the collection of data on graduates’ employment and career, in particular for collecting data conditioned by the responses given in previous occasions on the same people.
  5. Targeted proposals for data collection mode, timing and contents to be elicited.
PeriodThe project started in October 2000 and ended in June 2002. The results of the analyses and the suggestions are described in the final report.
Participants

The following scholars were appointed by the Observatory for the Evaluation of the University System at the Italian Ministry of University and Research:

Luigi Fabbris (coordinator)
Antonio Giusti, University of Florence, local coordinator
 
Matilde Bini, University of Florence (today Università Europea of Rome)
Claudio Capiluppi, University of Padua
Bruno Chiandotto, University of Florence
Cristina Martelli, University of Florence
Bruno Martinazzo, University of Padua
Maria Cristiana Martini, University of Padua (today University of Modena and Reggio Emilia)
Alessandra Petrucci, University of Florence
Gilda Rota, University of Padua
Publications

The outcomes of the experiment are described in a paper presented to members of the Observatory for the Evaluation of the University System at the Ministry. Parts of the results have been presented in several scientific meetings and published in specialized journals and volumes.

The main report was titled: “Experiments with computer-assisted systems for data collection on university teaching and employment and career of graduates (in Italian)” (L. Fabbris editor).
Several related papers were included in the following volumes:
  1. “Il questionario elettronico” (Luigi Fabbris, editor, Cleup ed, Padova, 2000)
  2. “Ingegnerizzazione del processo di produzione dei dati statistici” (Antonio Giusti, editor, Ceup, Padova)
  3. “Valutazione della didattica con sistemi computer-assisted” (Marisa Civardi and Luigi Fabbris, editors, Cleup ed. Padova)
  4. “CAPTOR 2000: Qualità della didattica e sistemi computer-assisted” (Luigi Fabbris editor, Cleup, Padova)