My struggles with the FSAs

Earlier this week I was sitting in the computer lab watching the students complete the grade 4 numeracy portion of the FSA  (Foundation Skills Assessment - British Columbia's standardized tests for grade 4 and 7). There were many thoughts going through my head during this time.  I had looked at the rankings that the Fraser Institute published and saw that my school had jumped significantly in the standings.  I knew that we would be moving up from where we were last year, our focus has been on writing for the past few years.  We have enjoyed some rich professional development, had great discussions, found different ways to emphasize and promote writing in the school and continued to work with different writing programs.  These have all lead to improvements.  I do believe that the students have improved in writing because of these changes, but that does not tell the whole story.

Our results are directly impacted by the number of eligible students who write the different tests and the number of students who have been excused from writing it for a variety of reasons including parental request that their child not write the tests.  If a number of very strong students are excused from writing the test, our results will drop, conversely if the number of students excused from writing are struggling learners our results will rise.  If there is not a flu bug running through the school around that time then the results go up, if there are a few kids who families all go on vacation at that time our results could drop.  There are so many external factors that impact the results of the test beyond the teaching that if they fall in your favour as they mostly did last year, the result is a rise in the rankings, if they do not, as was the case two years ago and this year, they will drop.

To give you some insight as to how it works I will include a brief description.  We received the package a few weeks earlier and I decided to wait to start.  Over the past few years the system has had problems where students will be kicked out of the electronic portion and have to log back in.  For some students this would happen 3-4 times over the course of the time they were working on one portion of the test.  This caused a lot of stress and caused some to panic and become very anxious, worried that they would have to start again.  (I cannot even begin to imagine what would be going through a high school student's mind when writing a provincial test and this happens!).  I hoped that by waiting there would be fewer students using it at the same time, thereby reducing the possibilities of crashes.

During the couple of weeks leading up to us starting I had some conversations with other schools, people on my soccer team and elsewhere discussing the FSAs.  A couple of friends with students at schools outside of our district were telling me that the kids had been practicing the tests for a couple of weeks and were a bundle of nerves because they were told how important these tests are and that they had to do well to represent the school.  One of my colleagues was telling me that she had 3 different parents come to her and ask whether or not their child should be writing the test because they were worried that their kids would not be successful because of their IEPs (individual education plans) and would therefore bring the school results down.  She told the parents that it was their choice, but that they should not be excluding their children for that reason.  Would all principals have said the same thing?  At our school all students write the test unless we are concerned about the impact that this could have on the student, and we do not spend any time practicing.

I had a few parents come to me a little before we were going to start wondering why we had not started our tests when when some of the neighbouring schools had.  I explained my rationale and they felt that it made sense.  One then asked me if we all received the tests at the same time and I let her know that I believed that to be the case.  The next questions caught me off guard as it was not something that had entered my mind.  If you held onto the tests for a couple of weeks, is there anything stopping schools from practicing with the actual tests before starting?  Could they all do a rough draft and then copy the good draft into the booklet?  If they were not all doing the electronic portion at the same time, would it be possible for a class to begin the reading portion and then copy the stories, print them and be able to go over them with the other students to prepare them for the questions?  When I answered that hypothetically speaking this was possible she was stunned.  How was this standardized?  In order for it to be standardized should all the kids in the province be doing the tests at the same time, or at the very least, all of the kids in the same school at the same time?

There are other factors that concern me as to how the data is used.  Two years ago our school received additional staffing partly due to the fact that we had a larger number of students with learning challenges than we normally had.  That year, when the students were writing the FSAs, the flu went through the school and some got sick, 1 missed a lot of time from school so I decided to not have that student finish the FSA. Result, not meeting expectations in all areas.  2 students went on vacation part way through the test, result, 2 students not meeting expectations.  2 students broke down in tears because they were frustrated so we took the tests away, result, 2 students not meeting expectations.  I believe that 4 of those 5 students would have fully met expectations.  I had two students who would not be writing the tests for a couple of reasons and I used their identifications to show the students to log on.  Result- 2 more students not meeting expectations because I had logged them in and answered 1 question but did not click on submit thinking the results would not show, this was a boneheaded moment which I did not repeat.  When you have a cohort group of about 50 kids and 7 are not meeting expectations for reasons other than the results of the test itself is one thing that a school can address and explain to its community, when an outside agency is ignorant of what happens in the schools and publishes results is another.

Four years ago we had the exact same number of identified gifted students as student with learning challenges two years ago.  The school went from double digit number of students who were gifted to double digits with learning difficulties.  The cohort was completely different and something outside the control of the school.  The result was a huge drop in the number of students not meeting expectations.  I go over the results with the parents, give them the numbers as well as some background information.  I am able to explain why we have gone up or down.  Our community is aware of our context, an outside agency examining and working with the data is not.  When a school has their test data compromised because their electronic results have gone into an Internet abyss and suddenly 10 students results are automatically not meeting expectations and those results are published, the public perception of the school can be compromised.

From my vantage point, the FSAs can provide useful information for a staff to look at and plan our school goals.  We can use it as a measuring stick to chart our progress, knowing our context.  This is not shirking responsibility.  I still stand in front of our parents and share the data and listen to concerns that they may have.  My concern is that when a non-informed group that ignores or does not care about the contextual situations that can arise in schools, and publishes results that are challenged statistically it impacts the perception of the school.  Why are schools punished because H1N1 hit their schools hard one year?  Why are schools punished for not practicing for the tests for weeks and choosing to focus on the curriculum and using the FSAs as a snapshot as they were intended to?  Why are schools punished because the cohorts each year can be significantly different from year to year and give a false increase or decrease in the students' perceived success?  The number of students writing is also compromised, based on parent willingness to have their children write the tests.  There are some schools where only 30% of the eligible kids are actually writing the test.

If everybody is not writing the test at the same time, in the same way, under the same conditions, with the same number of kids in a room writing it together, with the same preparation, with ALL students writing the test regardless of abilities, with one large group of independent markers marking all of the tests in the same way and removing the possibilities of possible manipulation of the testing situations, intentionally or accidentally, the test is no longer standardized even if all of the questions are the same.  There are just too many outside factors possible to make this testing equitable.  Let schools use the information for the way that it was intended and not let outside agencies compromise the validity of a non-standardized standardized test.