When an assessment tells you nothing

In Victoria, Year 12 is the last year of formal schooling and for those students who are completing the Victorian Certificate of Education (VCE), the exams at the end of the school year form a very important summative measure of their understanding in the subjects they have chosen to study.

Even though it was nearly 30 years ago, I remember very clearly sitting those exams and the pressure I put on myself during that exam period!🤦‍♀️At the time it seemed like the results of the exams would decide the path I would take for the rest of my life, in reality this was certainly not the case! 

If you live in Victoria, you may be aware of the issues that plagued the Year 12 exams last year (and in previous years). If you are not from Victoria or missed the controversy, this article will get you up to speed. 

In 2024, most Victorian Year 12 exams were affected by the 'cheat sheet fiasco' (65 of the 74 'common' exams). This is something that caused stress (at an already very stressful time) for many students, teachers and parents involved with Year 12 studies.

And while the 'cheat sheet fiasco' certainly challenged the integrity of the examination process, it was an article about a question in the Economics exam that really caught my eye. 

As you can see below, the newspaper reported that there was a "dud" question in the exam.

The article went on to explain that the first multiple choice question left students feeling ‘confused’!

Let’s face it, there is a lot going on in Year 12 exams, and a feeling of ‘confusion’ is certainly not something we want to add into the mix!

Reading this article made me think of the assessment we use in our maths classrooms.

How often do we ask 'dud' questions?

Assessments essentially provide a window into each student's mind. 

Although it would make things a lot easier, unfortunately, we can't physically 'see' if a student has acquired knowledge, or is fluent in a skill by simply looking at their head! The only information we can accurately gather looking directly at a child's head is if head lice are present!  😯

Assessment provides us an opportunity to observe something tangible and use that observation (formal or informal) to make a comparative judgement. 

However, if the questions we ask are 'duds', the output we receive is compromised.

I find we expect too much of assessments. A few questions loosely linked to a content area on an online assessment will more than likely not accurately tell us if a student is 'at standard'.

The reality is many Mathematics curriculum descriptions are incredibly dense (remember it is the Achievement Standards we assess against and the descriptions we teach). They are multi-layered, often involve multiple skills, and embed multiple proficiencies.

ACARA reports that Version 9.0 of the Australian Curriculum has 21% less content descriptions compared to Version 8.4. This is a positive change as it has seen the culling of some less important descriptions- for example I was never a fan of the Year 5 description:

'Apply the enlargement transformation to familiar two dimensional shapes and explore the properties of the resulting image compared with the original' (ACMMG115)

and was happy to see it had been removed in Version 9.0.

Similarly, I appreciate the decision to the move 'volume' into the Secondary school curriculum.

As ACARA correctly states, less descriptions allow us to teach with 'depth and rigour', something we should be always seeking in our maths teaching.

However, another reason we have less descriptions is because in Version 9.0 some descriptions have been combined.

Case in point:

In Year 4, description AC9M4A02 states:

recall and demonstrate proficiency with multiplication facts up to 10 × 10 and related division facts, and explain the patterns in these; extend and apply facts to develop efficient mental and written strategies for computation with larger numbers without a calculator.

this new description is a combination and 'rejigging' of these Version 8.4 descriptions:

Investigate number sequences involving multiples of 3, 4, 6, 7, 8, and 9 (ACMNA074)

Recall multiplication facts up to 10 × 10 and related division facts (ACMNA075)

For us, as teachers, there is a great deal to unpack here.

Firstly, if we think about the proficiency embedded in the first part of the description 'recall and demonstrate proficiency with multiplication facts up to 10 × 10 and related division facts'  for me 'recall and demonstrate' points directly to the embedded proficiency of fluency.

The next part of the description 'and explain the patterns in these'  points to the embedded proficiency of reasoning.

The final part of the description...extend and apply facts to develop efficient mental and written strategies for computation with larger numbers without a calculator points to the embedded proficiency of problem solving.

For me, the proficiencies embedded in this description need to be taught and assessed in very different ways.

There is no use assessing our students on just their accuracy with the multiplication facts up to 10 x 10, when the description points to fluency. Administering an assessment that requires students to answer a sheet of multiplication facts does NOT provide us with reliable data on their fluency. There needs to be a timed element to measure fluency.

Similarly, if are looking to see if students can 'explain the patterns', an assessment where they are timed answering their multiplication facts, will not provide the data we require. 

It is critical we understand the purpose behind our assessments- what exactly is it that we trying to 'see' in the mind of the student?

In order to gather data that will allow us to accurately assess the curriculum, we need to think deeply about the type of questions we are asking, and the format and purpose of the assessments we are using.

Now, you might be looking at this description and thinking, I don't agree with you Ange... I see xyz proficiency embedded in this description, and herein lies a challenge. The curriculum leaves us to make judgments and interpret the descriptions. We know that whenever we leave something open for interpretation, there are...surprise, surprise... different interpretations. These interpretations are heavily influenced by our experience and our level of knowledge and understanding of mathematics teaching and learning.

This is where I believe taking time to unpack descriptions and break them down into smaller more manageable, more assessable pieces is key.

When, as a teaching team, we all know what (multiplication fact fluency for up to 10x10) how (formal timed assessment), why (because timing allows us to assess 2 important parts of fluency- accuracy and efficiency- this assessment does not allow us to measure flexibility- this is something we will need to measure in a different way) and when (during Week 5 lesson 3) we are gathering data...

and

if we all know the expected standard we are judging against (we want our Year 4s to correctly answer each fact within 3 seconds), assessment is no longer a guessing game. We can make consistent, informed judgments about what our students know and don't (yet) know.

If we don't think carefully about the assessments practices we use, we can end up asking 'dud' questions that fail to provide us the insights we require to accurately teach and assess our students.

This week I encourage you to think about one description in your Year Level. Take a few minutes to unpack it (bonus points if it is with a colleague or your teaching team!). What content is it asking you to teach? what proficiencies are embedded? what would you expect to see from students who are 'at standard' ,what types of assessment (informal and formal) might you use to gather quality data around the skills and knowledge.

The more deeply we think about our teaching and assessment, the better practitioners we become for our students (who, fingers crossed, we don't assess as having head lice!😔).

Have a great week!

Ange🎓🎲

P.S. You can download the PDF version of this blog to print or share with colleagues here.