Selecting the ‘right’ maths assessment

In this blog post I wanted to share some thoughts about the importance of Numeracy Leaders and teachers developing assessment literacy skills.

The guru is assessment literacy is James W Popham and he has written numerous papers and books on this topic.

He describes assessment literacy as the single most important thing teachers can develop in order to make positive change in education.

Basically, assessment literacy is having the ability to correctly select, administer and analyse the assessments we use with our students.

In this blog I am going to address the first issue: selecting a quality assessment.

This is a topic I am pretty passionate about because my PhD was all about designing a valid and reliable assessment for teachers in place value.

I wanted to do this because I knew that there was a missing piece in the teaching of place value.

When I was a Numeracy Leader, we would use the Early Years Numeracy Interview with Junior students, but I found there was no quality place value assessments for Year 3-6 students.

This left both myself and my teachers with a ‘blind spot’, where we did not have access to accurate information on student understanding. This in turn, led to issues with the quality of place value teaching.

It is safe to say I know how much thinking, research, effort and planning is required to design quality assessment- I dedicated 4 years to it!

One of the things that is super important is ensuring teachers are able to discern the quality and appropriateness of tests.

Not all assessment are created equal. 

Below are some of the things to consider  when selecting maths assessments:

1.Who designed the items and constructed the assessment? 

Was there input from teachers? Was there input from experts in the field? Was there a psychometrician or expert in assessment measurement consulted? This isn’t always necessary, we create assessments in our class all the time and most of us are not psychometricians…BUT if we are purchasing assessment platforms I think it is important they come from a place of research, because our teachers often (rightly so) take these results as ‘gospel’, and don’t question the results…so before it gets to that point, we need to make sure we feel confident in the testing platform! 

2.How was the assessment designed? 

Were the items trialled? Were they validated using psychometric measures such as Rasch analysis or Item Response Theory (this gives a lot more weight to the results, as you know they are painting an accurate picture of student achievement). Assessments like the PAT, Scaffolding Numeracy in the Middle Years (SNMY), NAPLAN (and my test the PVAT) use these analysis methods. Before I completed my PhD I thought a rash was thing that appeared on your skin… now the word ‘Rasch’ means something very different to me!

3. What is the purpose of the assessment?

Is it a general assessment designed to give you a broad indication of a range of maths content? Or is it drilling down into just one content area? For example, my test  was designed to give in depth data on student understanding of place value, a Progress Achievement Test-Maths (PAT-M) assessment from ACER is not designed to give this level of detail, so should not be used to determine the achievement of students in a specific content area like place value. PAT-M provides other important data, but was not designed for small grain size analysis. This is important for teachers to know.

4. Who is analysing the results? 

Does the assessment allow teachers to easily see the errors students are making. Or does it just present a summary of the results at the end. Although time consuming, the process to mark, see student misconceptions and make a plan to remedy their problems, for me is one of the most critical parts of developing teachers’ assessment literacy and practice. I don’t think we should be outsourcing this part of the process!

So, there you have it, my take on assessment literacy and how important it is for us to be fully informed about the assessments we are choosing.

I hope this has raised a few questions in your mind. If it has I encourage you to ring the test designers of the assessment platforms you use and pose the above questions to them. The first critical step in the assessment process is ensuring you are choosing the right assessment for your students and you don’t know that unless you ask!

Want to learn more from Dr Ange? Click here to find out more about her “Assessing Place Value in Years 3-6” Mini Course.