While I was collecting NAEP data, I thought I’d see if the explosion in education spending from NCLB has any relationship to the percentage of students proficient or above in math and reading, so I downloaded the most recent data (8th graders, 2005, aggregated by state). First, though, I was curious to see if there was a statistically significant difference between the percentage proficient and above in math and the percentage proficient and above in reading, so I ran ANOVA:
| ANOVA: % Proficient or above in math and reading | ||||||
| Source of Variation | SS | df | MS | F | P-value | F crit |
| Between Groups | 48.5898039 | 1 | 48.5898039 | 0.94313513 | 0.33381604 | 3.93614278 |
| Within Groups | 5151.9451 | 100 | 51.519451 | |||
| Total | 5200.5349 | 101 | ||||
Interestingly, there is not. In order to disprove the null hypothesis, p must be 0.05 or lower, and as you can see, p is 0.33381604 (alternatively, you can see if the value of the F is greater than the critical value of F, and it is not) . So we cannot claim from these data that there is any statistically significant difference between the percentage proficient or above in math and the percentage proficient or above in reading.
Let’s turn, then, to per pupil spending, and see if it (our independent variable) has any effect on the percentage proficient in math or the percentage proficient in reading (our dependent variables). If we just eyeball the data it seems dubious that there is an effect:
| State | Per Pupil Spending | Proficient or Above Math | Proficient or Above Reading |
| Alaska | $16,665 | 28.7 | 26.4 |
| District of Columbia | $16,344 | 6.9 | 11.7 |
| Tennessee | $6,460 | 20.6 | 26.2 |
| Mississippi | $6,387 | 13.5 | 18.5 |
These are the two states that spend the most per pupil, and the two states that spend the least. Note that in reading, Tennessee scores almost as high as Alaska, and note the vast difference between the two top spending states, Alaska and DC (yes, I know DC is not a state, but it’s included as a state for the purposes of aggregating the data). Mississippi, which spends the least per pupil, ranks pretty low compared to both Alaska and Tennessee, but higher in both math and reading than DC, the second-highest per pupil spender. But to see if per pupil spending has a statistically significant effect on the percentage proficient or above in math and reading, we need to run regressions, first on math:
| Regression Output: Per Pupil Spending (x) and % Proficient or Above (math) | |||||
| Regression Statistics | |||||
| Multiple R | 0.03949115 | ||||
| R Square | 0.001559551 | ||||
| Adjusted R Square | -0.018816785 | ||||
| Standard Error | 7.760173608 | ||||
| Observations | 51 | ||||
| ANOVA | df | SS | MS | F | Significance F |
| Regression | 1 | 4.609102257 | 4.609102257 | 0.076537358 | 0.783209568 |
| Residual | 49 | 2950.794427 | 60.22029443 | ||
| Total | 50 | 2955.403529 | |||
| Coefficients | Standard Error | t Stat | P-value | ||
| Intercept | 27.08558202 | 4.728833956 | 5.727750703 | 6.13097E-07 | |
| Per Pupil Spending | 0.000141012 | 0.000509707 | 0.27665386 | 0.783209568 | |
The value of p is 0.783209568, not 0.05 or lower, so per pupil spending has no statistically significant effect on the percentage of students proficient or above in math (note that the correlation coefficient between the two variables is only 0.03949115). So how about reading?
| Regression Output: Per Pupil Spending (x) and % Proficient or Above (reading) | |||||
| Regression Statistics | |||||
| Multiple R | 0.048217771 | ||||
| R Square | 0.002324953 | ||||
| Adjusted R Square | -0.018035762 | ||||
| Standard Error | 6.687537468 | ||||
| Observations | 51 | ||||
| ANOVA | df | SS | MS | F | Significance F |
| Regression | 1 | 5.106856808 | 5.106856808 | 0.114188199 | 0.736868904 |
| Residual | 49 | 2191.434712 | 44.72315738 | ||
| Total | 50 | 2196.541569 | |||
| Coefficients | Standard Error | t Stat | P-value | ||
| Intercept | 28.39898531 | 4.075199326 | 6.968735277 | 7.41626E-09 | |
| Per Pupil Spending | 0.000148431 | 0.000439253 | 0.337917444 | 0.736868904 | |
The value of p is 0.736868904 so per pupil spending has no statistically significant effect on the percentage of students proficient or above in reading (and note that the correlation coefficient between the two variables is only 0.048217771). So these data–the most recent NAEP data, for 8th graders, aggregated by state), support no statistically significant relationship between per pupil spending and math or reading proficiency.
However, one of the data tables I downloaded for these analyses included the mean NEAP math scores by parents’ level of education, aggregated by state (again, 8th graders for 2005). It seemed almost a waste of time to run ANOVA on the scores for all four levels of education, so I ran ANOVA on adjacent education levels, first no high school diploma and only a high school diploma:
| ANOVA: Parental Ed and NAEP mean math score (No HS diploma, HS diploma) | ||||||
| Source of Variation | SS | df | MS | F | P-value | F crit |
| Between Groups | 1304.6544 | 1 | 1304.6544 | 24.7627 | 2.7727E-06 | 3.9381 |
| Within Groups | 5163.2512 | 98 | 52.6862 | |||
| Total | 6467.9056 | 99 | ||||
The p-value is 2.7727*10-6, so the NAEP math scores for students whose parents who have no high school diploma and those whose parents have only a high school diploma are statistically significant. And indeed, we see a statistically significant difference between all adjacent groups:
| ANOVA: Parental Ed and NAEP mean math score (HS diploma, Some college) | ||||||
| Source of Variation | SS | df | MS | F | P-value | F crit |
| Between Groups | 4080.6544 | 1 | 4080.6544 | 77.0904 | 5.3483E-14 | 3.9381 |
| Within Groups | 5187.4712 | 98 | 52.9334 | |||
| Total | 9268.1256 | 99 | ||||
| ANOVA: Parental Ed and NAEP mean math score (Some college, College grad) | ||||||
| Source of Variation | SS | df | MS | F | P-value | F crit |
| Between Groups | 1429.5961 | 1 | 1429.5961 | 23.0638 | 5.6337E-06 | 3.9381 |
| Within Groups | 6074.4778 | 98 | 61.9845 | |||
| Total | 7504.0739 | 99 | ||||
So the level of parental education has a statistically significant effect on childrens’ math scores even between adjacent groups. I find this somewhat surprising. I would have expected a statistically significant difference between the math scores of students whose parents who had never completed high school and those whose parents graduated from college, but these data indicate that the level of parental education has a significant effect even when comparing students whose parents went to college but did not graduate and those whose parents graduated from college.
While we see no significant effect of funding on scores, we do see a strong effect of parental education on scores. Interesting.
KDeRosa says:
I graphed the data for PA here.
I used SES instead of parental education, but the results were virtually the same for botjh. Parental education correlated a bit better than SES with student achievement.
Funding was random.
April 10, 2007, 10:10 amRight Wing Nation says:
[…] If you recall, I was somewhat surprised to see statistically significant differences between the math of students in 2005 and the level of education of their parents. Curious, I went back to the Department of Ed site cruising for more data. […]
April 14, 2007, 5:25 pm