Accurately assessing number sense in elementary age students is essential for future mathematical success. Without strong number sense students will be unable to participate in rich mathematical problem solving and discussion. The development and implementation of a British Columbia curriculum based number sense assessment referenced to the First Steps in Mathematics program and the Learning Trajectories by Clements and Samara will allow teachers to identify student strengths and provide additional instruction in areas for growth. Focused teaching of number sense should increase student’s numeracy skills.
Assessing student number sense will require a mix of quantitative and qualitative approaches to data collection and analysis. Quantitative data will be valuable for its objectivity and reliability, our two external measures of numeracy the Foundation Skills Assessment (FSA) in the fall of grade 4 and the District Numeracy Assessment (DNA) in the spring of grade three provide insight into how many students are meeting grade level expectations and how many need additional supports as well as confirming that our focus on number sense is supporting overall development on . By having these two measures as a staff we will be able to determine if our efforts to improve number sense are having the desired effect on student achievement. Quantitative data is seen as reliable and holds a great deal of accountability for school staff as it is a direct measure of student success. We will also need to rely on quantitative data when preparing our school’s Action Plan for Learning and presenting our ideas to the school board office for funding. Being able to use statistics to show where our gaps are and how much growth could happen by focusing on number sense and assessing it specifically to inform teaching and determine a starting point with the First Steps in Mathematics program. Our number sense assessment will also generate quantitative data in our pilot classrooms for determining how many students are currently meeting grade level expectations as well as how many require additional support.
While the quantitative data gives a snapshot of what is happening while the qualitative data dives deeper into the why. Quantitative data allows for a deeper understanding of the context in which students are participating in classroom activities and assessments. Meeting with students, parents and staff and conducting interviews or surveys allows for qualitative data to be gathered capturing their attitudes towards number sense and numeracy as well as where they are looking to see growth. Qualitative data could also include observations of student work and collected work samples.
Looking at creating and implementing a number sense assessment we will be utilizing a mixture of methods for creating groups as well as determining how a focus on number sense effects student learning. We will be using different classes as our groupings in a quasi-experimental design and having our pilot class at each grade level using a within subjects’ design. To increase validity, it might be useful to pre and post test one or more of the other classes at a grade level who are not participating in explicit daily number sense instruction.
Our data will need to be generated from a variety of sources. The quantitative data from the FSA and DNA results as well as from common classroom assessments such as the district math screening tool for each grade. Using these tools we can validate our measures from the school created number sense assessment. A great deal of data regarding student progress will come from qualitative measures in teacher’s classroom observations of daily work and the collection of work samples showing how students are able to apply number sense concepts. The ANIE may be a useful tool for regular weekly or biweekly progress monitoring as it uses one question or problem to capture a snapshot of student learning and has students demonstrate their learning in a variety of ways. Using the ANIE allows teachers to gather and quickly code regular work samples showing student progress. Student portfolios will also be a source of qualitative data and can include both work samples as well as students’ reflections on the work. Another tool that will allow teachers to make observations about student’s understanding is having students create videos showing others how to solve a specific problem and explaining their process, this not only captures student’s answers but allows teachers to hear student’s tone of voice and pacing in explaining their process which could give an indication to student’s confidence in applying the current concepts. Additional quantitative data can be collected through documenting conversations had during small group instruction and interviews with students.
Classroom data from teachers and students can not be the only data collected, to gather the complete picture of what is happening with number sense non-enrolling teachers, special education assistants and parents will also need to be included in the conversation. Including other groups in sharing observations of student learning could involve surveys and interviews or focus groups. Inviting parents from the pilot classes to participate in focus groups and transcribing the sessions for analysis to identify key themes and recuring ideas. While students are the ones in the classroom their families impact their perceptions of the tasks that are being completed, it will be essential to survey and interview family members to understand parent’s views of number sense and numeracy instruction, they can also provide insight into what is occurring at home and their views of their student’s number sense and attitude towards mathematics.
When looking at ensuring validity with our data and having an accurate sample of the students in our school in our pilot classes, relying on the class building process should allow for a breadth in student abilities, backgrounds and gender balance. Our school strives to have an equal number of students with IEPs in each classroom and balance the needs where possible, however students are individuals, and no class composition will ever be perfectly balanced. Cultural diversity is a challenge in my school as our students and staff are primarily of European descent. While we have one or two students in each grade with indigenous heritage, I am the only indigenous staff member permanently placed at the school as we have gone three years with out an Aboriginal Support Worker. It would be supportive to meet with the district principal for aboriginal education as well as aboriginal teacher leaders to ensure that the assessment and activities are respectful, and any use of aboriginal content is meaningful and not tokenistic. Bringing in our English Language Learning (ELL) teacher would help develop the cross-cultural understandings as would working with leaders in our international department and the Settlement Workers in Schools (SWIS) team. The SWIS team is able to provide translation services to our families who speak Mandarin, Korean, Arabic and Hindi as well as support school staff in creating culturally responsive and respectful resources and learning environments. The learning support and resource team will be a valuable resource in adapting tasks to meet the needs of students with learning challenges. One of the things we will have to ensure in developing a number sense assessment is that the language is clear and simple so that it does not become a reading comprehension task which may hinder many learners.
When implementing the assessment first in pilot classes then on a broader scale we are going to have to establish timelines for how often the assessment should be given and when the data will be reported. My aim is to complete an initial assessment within the first two weeks of the school year to establish a base line then do repeat assessments at the end of each reporting period. By doing repeat assessments in November, February/ March and May/June we will be able to track student growth throughout the year and adapt teaching strategies or provide additional supports. The end of year reassessment also serves as a summative assessment. If our assessment is valid, we should have similar numbers of students who are meeting or exceeding expectations to the FSA and DNA assessments. Quantitative data about student scores on each section of the numeracy assessment can be input through Microsoft Forms and exported to an Excel sheet or entered directly into the Excel sheet. Microsoft Forms will also be used to collect survey data from all groups involved, staff, students and families. Forms has some built in tools to do quick data analysis showing the mean and the mode for numerical data as well as showing the stretch of data on Likert scales. Data can also be exported to Excel and coded numerically for further statistical analysis with descriptive statistics. Bar and circle graphs can be used to examine the change in student scores year after year allowing teachers to visualize the trends and over all progress. We will also be able to look at differences between the classes receiving focused number sense instruction and those who are not using comparative analysis. While it is possible to compare different cultural or socio-economic groups as well as the differences between genders due to the homogeneity of the school, I work in I do not see how working with comparative data between those groups would be helpful. The qualitative data will be more challenging to analyze. Analysis of this data will focus on the content of interviews and any emerging themes. I am most interested in the themes that come up in interviews with all three groups involved, are there common ideas about number sense or wide-ranging misconceptions to be addressed. The surveys will also revel common trends and share valuable insights by comparing the survey data with interview transcripts it will give a more rounded picture of the qualitative measures.
Taking on a project at this scale will bring about many challenges. It asks a lot of stakeholders to participate in the pilot groups and to take on something new of this scale. Teachers already have limited time and need to be intentional about how they use it. It will be important to schedule interviews at times when teachers have less on their plates but would be helpful to schedule assessment dates for students near the end of each term so the assessment can also be used as evidence of student progress in report cards and student inclusive conferences. Consistent quality within the data could pose another challenge, the solution would be to train teachers to implement the assessments the same way with students through a standardized protocol, giving specific directions and possibly a script. A third challenge is analyzing all the data, there will be hundreds of data points from individual student assessments to be recorded and analyzed as well as the interviews with stakeholders to be coded and understood. Having common documents and folders possibly through Teams we can share data and post the coding of interviews to common documents and folders; by utilizing existing district systems we can ensure compliance with local and provincial privacy protection laws and practices.
Before embarking on this project, we will need to meet with instructional services and district staff that can give approval and guidance surrounding ethics. This will include developing literature that explains to parents what the class will be participating in as part of their educational program and getting informed consent to interview and survey all involved. Data protection should follow the Freedom of Information and Protection of Privacy Act (FOIPPA) that governs the storage and usage of all student data and that all teachers are trained on annually. Part of ensuring FOIPPA compliance is that project related data is stored only on district devices and servers which are known to follow FOIPPA and like with all other student data strongly discouraging members of the project team from storing project related data on personal devices. In terms of data transparency aggregate data may be used to share global results while respecting student and staff privacy. One issue I can see with this is that it will likely be known which divisions are the pilot classes in each grade which narrows down which students, staff and families are part of the initial data set. To preserve anonymity with the surveys they will be set to anonymous mode before distribution so that all information is collected without requiring identifying information. To ensure that there is no bias present in the assessments, survey or interview questions they will need to be looked at by multiple people in different roles and from different perspectives.
In closing the collection and analysis of number sense data will be an enormous task and will only be possible with a dedicated team and significant staff buy in. To have a complete picture of the data we will need to utilize a mixed methods approach to using qualitative and quantitative data. As we will start with a pilot project, we need to be prepared to adapt our approaches throughout to best serve the needs of our students. Overall, the hope is that we will be able to gain important data and insights that will support a school wide approach to improving numeracy through focused number sense instruction.
References
Aidley, D. (2019). Chapter 6: Basic concepts of data description and analysis. In D. Aidley, Introducing quantitative methods : a practical guide (pp. 155-190). London: Red Globe Press.
Bird, K., & Savage, K. (n.d.). The ANIE. 2014: Pembroke Publishers.
Carlyle, M., & Paton, D. (2023, September). Langley Fundamental Elementary School School Improvement Plan 2023-2024. Langley, BC: Langley School District (SD35).
Clements, D. H., & Sarama, J. (2014). Learning and Teaching Early Math The LEarning Trajectories Approach (Second ed.). New York: Routledge.
Sarama, J., & Clements, D. H. (2023). Learn Trajectories. Retrieved from LearningTrajectories.org: https://www.learningtrajectories.org/documents/1691785025839.pdf
Services, W. O. (Director). (2015). Overview of Qualitative Research Methods [Motion Picture]. Retrieved from https://www.youtube.com/watch?v=IsAUNs-IoSQ
Services, W. O. (Director). (2015). Overview of Quantitative Research Methods [Motion Picture]. Retrieved from https://www.youtube.com/watch?v=cwU8as9ZNlA&t=52s
The Government of Western Australia Department of Education. (2013). First Steps in Mathematics Number Book 1. Perth: Department of Education WA.
