Making Success and Making Improvements

Combining and analyzing data to evaluate course achievement.

On this page:

To improve your course, you need to review what happened, determine what to change and decide how to improve it. This process requires not just collecting the right data but knowing how to make sense of it, especially when different types of information might lead to contradictory conclusions. For example, what should you do if some students loved an assignment while other students disliked it? How can you improve instruction if many students fail an exam? This page will focus on what to consider when using multiple types of data to make sense of course achievement, as well as how to use these analyses to make improvements.

Considerations for data

Using a variety of types of data together is known as a “mixed-methods” design (Creswell, 2014). Mixed methods is an entire field of study, with many unresolved conceptual and design issues. Therefore, we provide a brief and simple overview here of what to be aware of when planning data collection and analyses for your class.

Types of data

Broadly speaking it may be helpful to consider data types as either quantitative data (numeric) or qualitative data (descriptive). While these are not strict categories, they may help you think about the variety of data you might consider collecting.

Timing

You may choose to collect different types of data at the same time (concurrent design) or in phases (sequential design). There are advantages to each of these and a variety within each type.

Weighting

If you gather more than one type of data, you will need to determine the time or resources for each data collection (e.g., how many students will you have in a focus group? How many in the survey?). You may also care about some findings more than others, and some types of data will only be supporting the other data.

Mixing

You need to determine when and how mixing your different types of data will occur.

Making your analysis plan and course changes

Now that you have considered how data may be combined, the next step is to plan and conduct analyses and determine improvements for your course.

  • Step 1: Using your data collection plan, determine what areas you will be evaluating using your data. For example, are you focusing on student achievement, course design, or instructional effectiveness?
  • Step 2: Combine and analyze data using the above recommendations.
  • Step 3: Once you have completed your analyses, look at the individual results as a whole to understand how certain parts may have interacted.
  • Step 4: Make choices about what you might change the next time the course is taught.

Make important or impactful changes first. Try to limit the number of changes to create fewer variables for future comparisons. If you make too many changes at once, it may be difficult to identify what caused new effects.