The purpose of the Educational Evaluation Research Consortium (EERC in English, CEIE in Spanish) is to conduct a study of the educational opportunities provided in primary education in the Dominican Republic and to understand the impact of those opportunities on learning Mathematics and Reading Comprehension from 4th to 7th grade. This is done to understand educational opportunities and learning throughout the educational system, to evaluate and monitor the impact of educational projects supported by the United States Agency for International Development (USAID) in the Dominican Republic, as well as to evaluate the progress of different innovations that diverse partners in the Dominican educational system are carrying out.

The EERC is made up of three universities: two from the Dominican Republic (Instituto Tecnológico de Santo Domingo and Pontificia Universidad Católica Madre y Maestra de Santiago) and one from the United States (University at Albany / State University of New York). The Consortium is conducting a longitudinal study of educational opportunities and learning, using vertically scaled tests (in which the scales represent levels of progressive mastery of the curriculum) in conjunction with instruments designed for measuring educational opportunities. These instruments represent the latest technical advancements in these types of measures, developed and validated in reference to the appropriate schooling contexts in the Dominican Republic.

Goals

The principal goals of the Educational Evaluation Research Consortium are:

a) Measurement of learning based on an empirical study of the intended and implemented curriculum in the Dominican Republic;
b) Development and validation of every element of the measurement instruments in reference to the curriculum and conditions of schools, classrooms and communities in the country;
c) Characterization of the progress that schoolchildren show in the learning of the reading and mathematics curriculum from 4th to 7th grade;
d) Measurement and analysis of the opportunities to learn the mathematics and reading curriculum that students are provided in their classrooms, schools, and communities;
e) Comparison of learning and educational opportunities offered in communities and schools that receive support from the evaluated projects, with nationally representative samples of rural public schools, urban public schools, and autonomous private schools;
f) Use of sampling and analysis methods that permit the investigation of different sources of variation in learning
g) Generation of knowledge regarding promising technical alternatives for evaluation in the Dominican Republic and other developing countries;
h) Promotion and stimulation of a scientific community devoted to the debate of topics in evaluation, the investigation of appropriate methods of measurement and analysis, and the use of empirical evidence in the consideration of policy alternatives in education.

Progress

Since its formation in November of 2003, the Educational Evaluation Research Consortium has completed the first analyses of an inventory of the intended curriculum in Mathematics, based on the coding to 1367 pages of textbooks and 413 pages of the National Curriculum – creating a data base for secondary analyses and generating a first draft of the learning test blueprints. The EERC has also assembled a team to design an instrument to perform equivalent analyses of the intended curriculum in Reading, with the purpose of generating a first draft blueprint for the Reading assessment instruments. The EERC has established contacts and engaged the cooperation of relevant technical offices of the Dominican Secretariat of State for Education.

The EERC has integrated a variety of Internet based communication tools, such as Microsoft Messenger and WebCT, which have made it possible to share documents, conduct meetings, share archival material, post agendas and minutes, and share other material amongst staff members. Currently, the Consortium is in the final stages of developing a web site that will incorporate all of the previously mentioned functions, in addition to the possibility of conducting synchronous and asynchronous discussions, performing web-based data entry and database management activities.

The EERC has prepared observation protocols to serve as empirical supports for the development of instruments to measure educational opportunities. It has also concluded a first comprehensive literature review and created a conceptual matrix to be used in the analysis of classroom observations, student notebook materials, videotapes of lessons, and other pieces of evidence related to the intended and implemented curriculum.

With respect to tests and survey development, the EERC has:

a) Used the conceptual matrix to create an bank of items under consideration for the measurement of educational opportunities;
b) Drawn a nationally representative sample of urban and rural public schools, and autonomous private schools, and designed two test and survey administration models to be used in evaluated schools and those in the national comparison sample. These models were designed to insure optimal conditions for the analysis of learning and the educational opportunities that might explain differences in learning. The first model (termed ‘Model A’ for experimental schools) will be for schools supported through specific USAID programs. This model entails the administration of tests and surveys in two contiguous grades, with tracking of the same students, twice per year, from 4th to 7th grade. The second model (‘Model B’ for comparison schools), developed for the national comparison sample, entails the application of instruments once a year, to three contiguous grades, where the same students are not tracked, but in which data collection takes place in the same schools for four years.

Finally, the EERC has begun the work required to verify the information in the data base concerning the 200 schools in the national comparison sample, and the identification of the schools and students that will be included in the Model A administration scheme.

Significance

Because the EERC is stimulating interest in the use of evidence in the consideration of educational policy options, we are working with various actors in the Dominican educational system. During this process it is important to uncover and consider the types of information and data resources that the most important actors in the educational system consider relevant. Hence, the Secretariat of State for Education is a principal partner in the Consortium’s work, as are the non-governmental organizations that are active in education, including research institutions, labor organizations, parents, community organizations, etc.

The EERC purpose is to increase the relevance and validity of evaluation by

a) including the perspectives of these actors in terms of including the priorities and specific initiatives taken or under consideration in different projects in the design of Consortium instruments;
b) considering programmatic alternatives and policy alternatives within the choice set of various relevant actors;
c) the participation of key EERC partners in decisions regarding procedures to be followed in the study;
d) making available bibliographic material and other information resources generated by the study for others to use in their own evaluation and monitoring endeavors; and
e) the opportunity for various relevant actors to participate in the research and development, as well as implementation processes of the study, as an opportunity for professional development in the areas of evaluation, research, and educational policy analysis.

Gilbert A. Valverde, Ph.D. - Project Director and Principal Investigator
J. Leonardo Valeiron, Ed.D. - Principal Advisor for Educational Policy
Sarah Gonzalez de Lora, M. Sc. - Coordinator for the PUCMM/Santiago, Senior Researcher
Sandra Gonzalez, Ed.D. - Coordinator for the INTEC