In the following weeks, I will be publishing a series of posts of Provincial Achievement Test (PAT) data. Before that, I wanted to provide some definitions and notes around the information that I will be presenting.
Provincial Achievement Test (PAT): The development and assessment of these tests are done at the provincial level. They are administered on an annual basis to all students in Grades 6 and 9 in the subject areas of English Language Arts, Mathematics, Social Studies and Science. Students in French Immersion and Francophone school districts will also write a French Language Arts PAT and may also write the Mathematics, Social Studies and Science PAT in French. In Grade 9, students who are enrolled in Knowledge and Employability (KAE) programs of study will take PATs developed for those courses. These test are, for the most part, administered in mid-June for Grade 6 and the late June for Grade 9. Having Gr. 9 students write a test in the last week of school has caused some people to question the validity of the results given the lack of motivation they might feel to do well on a test that “doesn’t count”. Further information on the PATs can be found on the Alberta Education website.
Participation Rate: Some students may be away on the day of the test, or some parents may have requested that their child not participate in the test. These students are marked as absent. Other students may be excused from writing the test if students are incapable of writing the test (even with provided accommodations) or if writing the test would be harmful to the student. This requires the approval of the school district’s superintendent. The province provides data on the total number of students enrolled in a particular grade, the number of students absent and the number of students excused. The number of students writing the test divided by the total number of students enrolled is the participation rate.
Acceptable Standard: Most people would simply term this as “passing” the test. Since it is almost impossible to develop tests of the same difficulty level from year to year, the province will determine a “cut score” for each test to determine what mark will constitute meeting the acceptable standard in any given year. Thus, you may see on your child’s PAT result that while they answered less than half of the questions correctly, they still met the acceptable standard.
Standard of Excellence: If the acceptable standard was a score of 50%, then the standard of excellence would be 80%. However, as described above the cut-score may vary from year to year. All students who meet the standard of excellence are also part of the group of students who meet the acceptable standard.
Percentage of all students enrolled: When releasing PAT results, the province will release data on how many students met the acceptable standard and the standard of excellence. They will then divide this by the total number of students enrolled to get the percentage of students enrolled that meet each standard. The province uses the data of all students enrolled when determining how well a school authority is doing on the provincial Accountability Pillar.
Percentage of students writing: The province also divides the number of students who met each standard by the number of students who actually wrote the test.
Province: The provincial results based on all the students in Alberta. Note that not all students are enrolled in a school authority, which is why I do not think that comparing school authority results to the province is helpful.
All school authorities: All public, separate (eg. Catholic) and Francophone school districts, as well as all private and charter schools. There are 175 school authorities in Alberta, which includes the 62 school districts. The four largest school districts are the Calgary Board of Education (CBE), Edmonton Public School District (EPSD), the Calgary Separate School District (CSSD or Calgary Catholic), and Edmonton Catholic School District (ECSD) which together enrol 44% of all the students in the province.
Mean: Otherwise known as the “average” score of all the students who wrote the test. This is not the actual average of any given test, but one that is statistically calculated to take into account the different cut scores from year to year.
Notes on examining PAT data
- While the province seems to encourage the comparison of school district data with provincial data, a comparison with all school authorities is probably better as the participation rate of the province is significantly lower than that of all school authorities. Students cannot pass the test if they do not write the test.
- It is important to look at data in a variety of ways. Results can be interpreted differently depending on which data set you are looking at. Each has a different perspective and each should be considered when trying to determine how well a school authority is doing on PATs.
- When looking at district wide performance, the percentage of students enrolled achieving at the acceptable standard is important so that we know how many students are meeting learning expectations.
- However, just because a certain percentage of students met the acceptable standard based on all students enrolled, doesn’t mean that all the other students are not meeting expectations. Participation rates need to be taken into consideration.
- It is imperative that we remember that the PATs do not test every learning outcome in the curriculum and therefor should not be used as the sole measure of success.
- Research has overwhelmingly shown that the strongest predictor of standardized test scores is socioeconomic status. While quality teaching and supports can ensure that every child is successful, we must recognize that there are many factors beyond the control of the education system that can impact test scores.
- Thoughts on comparing PAT results to the “province”
- Grade 6 English Language Arts PAT data