Thoughts about the Board Work Session (PVAAS & PSSA results)

At the SCASD Board Work Session last night, the board reviewed Pennsylvania System of School Assessment (PSSA) and Pennsylvania Value Added Assessment System (PVAAS) outcomes for the district. Three handouts were available: a “Non-parametric trend analysis of SCASD PSSA performance”, and two memos from Dr. Francis to Dr. Mextorf on 2009 District PSSA/PVAAS Data Report.  I have not been able to locate these online on SCASD’s website, so will summarize points of interest below.

The trend analysis looked deeper at the rankings highlighted in an earlier post. Using all 500 PA districts’ PSSA math and reading scores, each district was ranked on the basis of the percentage of those classified either “proficient” or “advanced” at each grade level in descending order. Changes in rank over time were analyzed by linear regression of rank versus academic year. A slope different from zero was considered a trend. Regression analysis yielded statistically significant positive slopes (declining ranks over time) in math for Grades 3, 5, and 8, and in reading for Grades 4 & 6. A significantly negative slope (improving ranks over time) was evident for Grade 7 in math and reading (p < .05). In brief, math ranks have reached new lows in 5 of 7 grade levels, as low as 265 of 500 in grade 6.

Dr. Francis explained that the PSSA scores are based upon achievement and target PA academic standards’ eligible content in mathematics (grades 3-8 & 11), reading (3-8 & 11), science (4, 8 & 11) and writing (5, 8 & 11). Do keep in mind that the PA standards have been rated poorly (“D”) and proficiency requirements in PA fall far below that required on the NAEP or in Massachusetts, for example.

The average percent of possible points in mathematics in each area by grade follow [if scored out of 100%, a possible "grade" is assigned for illustrative purposes here]

2009 PSSA Mathematics Reporting Categories
Average % of total possible points

                        Gr 3 	Gr 4 	Gr 5 	Gr 6 	Gr 7 	Gr 8 	Gr 11
N                       476 	471 	471 	490 	512 	548 	651
Numbers & Operations 	87 	70 	71 	69 	75 	71 	68
Measurement             85 	76 	62 	74 	70 	71 	72
Geometry                87 	72 	68 	71 	75 	74 	75
Algebraic Concepts 	83 	69 	76 	71 	71 	70 	77
Data Analysis & Prob.   94 	88 	67 	61 	69 	79 	69
Average %               87.2 	75 	68.8 	69.2 	72 	73 	72.2
"Grade" (out of 100%)   B 	C 	D+ 	D+ 	C- 	C 	C-
 

Dr. Francis noted Grade 3 & 4 were strong in Data Analysis & Probability, and Algebraic Concepts, and a need to focus on measurement and conventions was evident (notably, it was acknowledged that “the core curriculum doesn’t have an emphasis on conventions, so is supplemented with Calendar Math”. Aren’t math conventions an essential component of math? Hmm.). Grade 5 “is slipping”. In middle school, Grades 6 & 8 show a downward trend over the past 3 years; Grade 7  is “moving in an upward trend”. In high school, slippage is noted in 2009 scores with patterns in reporting areas similar across years. The areas in need of more attention are Numbers & Operations, and Data Analysis & Probability at that level.

The PVAAS scatter plots of PVAAS growth measure (x-axis) against PSSA % Proficient or Advanced (y-axis). The growth value ranges from -20 to 35, where “a value of 0 indicates that the cohort has experienced a full year of growth for the year in school. Any value to the right (positive) indicates more than a year of growth for a year of schooling. Any value to the left (negative) of the 0 line indicates that there has been less than a year’s worth of growth for a year of schooling.”

Whereas for Grade 4 all SCASD elementary schools’ data points fall in Quadrant I (Adequate Achievement Positive Growth), Grade 5 data points have negative growth scores and ALL fall in Quadrant II Adequate Achievement Negative Growth. Grade 6 & 8 points fall slightly positive but just about on the 0 line; Grade 7 points fall more comfortably in Quadrant I. So, overall  it seems that our district’s results in mathematics are not looking so good, and are mixed at best.

Throughout the discussion at the Board Work Session, action steps were detailed, and a sense of “urgency” was noted. The “BIG Action Steps” listed were: “1) time for intervention and supplemental instruction for students, 2) Requiring students to participate in recommended instruction versus encouraging students to participate, and 3) Time for collaborating and data sense-making”.

I, for one, am pleased that the district is taking steps to look closely at these data, sees some serious concerns, and has identified the need to use formative, norms-based assessments to evaluate student needs. That said, SCASD adopted Investigations-2 for elementary and uses Connected Math in middle school as the core curriculum choices. Today, it is rather unfortunate that so much time seems focused on “identifying instructional resources to disseminate to teachers”, “adjusting curriculum”, and “reviewing instructional materials”.  If we had instead a good, solid, direct instruction-based curriculum that required mastery of skills in mathematics before moving forward rather than unproven, spiraling constructivist approaches that regurgitate topics repeatedly, would more instructional resources be needed? Would more time really be needed? Would supplemental instruction be needed?

Much discussion seemed to center around developing a summer school academy approach. Sounds interesting and may well be quite helpful for some students. At the same time, the notion of “requiring students to participate in recommended instruction versus encouraging students to participate” has me thinking. If the district can require parents to bring their kids in for extra instruction, why can’t we parents REQUIRE (rather than encourage) the district to use better, research-based and empirically proven instruction in our kids’ regular classrooms to begin with? Seems only fair to me!