Taking the measure of human learning - Locally constructed tests

Ed 510 Applications of Educational Research
 

Introduction
 

Collecting the information to answer the research question (s) is often accomplished through the use of tests. This section explores the purposes of local tests (designed by teachers and other educators) and the strategies used to create them.

Other kinds of tools are also used to collect research data. These include checklists, surveys and questionnaires, observation formats and protocols, structured interviews. All of these techniques are designed for local educational purposes as well as for the purpose of answering research questions   Even these approaches to data collection follow design strategies that are similar to the principles used to design tests.

Tests are used for a variety of purposes, mostly related to making decisions. There are several types of decisions made by tests:

• Instructional - to decide whether the goals and objectives of instruction have been met.
• Curricular - to decide whether the goals and objectives of a program or curriculum have been effective.
• Selection - to identify candidates for programs (for example, law school admissions tests).
• Placement- to identify candidates suited for placement in special programs (for example, gifted education).
• Screening - to distinguish among examinees for special attributes or traits (for example, engineering aptitude).
• Program comparison - to compare two or more programs according to one or more important criteria (for example, school districts performance on standardized tests of reading).

 

• Identify and define the attribute to be measured - What is the construct that represents the knowledge, behavior, skill, attitude or opinion that will be assessed?
• Decide on procedures that will be used to demonstrate the attribute - What will you have the examinee do in order to demonstrate the construct objectively using a test?
• Quantify the attribute - How will you assign values to the performance of examinees on tests?

 

Planning questions

 
The first concern of the researcher is to determine the purpose of a test .  Then the researcher must address the following questions.

 
• What are the objectives of instruction are investigated by the instrument?

• When testing attitudes and opinions this questions doesn't seem to apply. But there is a parallel idea that needs to be part of the planning process. E.G., what cognitive processes or psychological attributes are to be demonstrated?

 
• What is the content that is to be covered by the test?

 
• What type of exercise (test performance) will indicate that content and objectives can be objectively demonstrated?

Pulling purposes, objectives, content and test procedures together
 

A well constructed local test is usually developed from a test blueprint. This is a diagram or chart that first of all integrates objectives and content. After these two components of the plan are pulled together, then it is possible to decide on procedure by deciding on the types of items that will be used.

Creating a test blueprint
 

In the following example a researcher wishes to construct a local test that can be used to evaluate a leadership training course. The course was delivered workshop style. Thus candidates for the principals certificate were exposed to theoretical knowledge about leadership, management, human motivation, social learning and learning in organizations. The researcher determines that her purpose is to see whether the outcomes of instruction have been met by the candidates. She begins by deciding how workshop objectives (that represent the outcomes of instruction) can be matched to the content covered in the workshop itself.

A table is created that lists objectives vertically and content horizontally, like so:

In this table the researcher has integrated content (knowledge about leadership) with objectives that were part of a leadership training course. The next step will be to decide on item formats that will assess knowledge strategically.
 

Deciding on procedures

The types of items that are used in a test must be matched to the knowledge that is going to be assessed. Thus items used to assess whether an examinee can identify key terminology may look different from items used to determine whether the examinee can transfer skills to novel problems.

Summarizing questions
 

Why do you think the multiple choice item format is repeated so frequently in the table above?
 

Why is the true/false item format listed only once?
 

What trade off's based on the efficiency of testing might be present. That is, why are some items listed less than others? Does it have something to do with the amount of information that can be gained from an item in relation to the number of items that can be included on an examination?