Case Study problems solved with Statistical techniques

Assume you are working on a team that has been commissioned by a large school district to collect and analyze data related to a recent curriculum experiment designed to improve student scores on state-wide standardized tests. The schools in this district are predominantly large, urban schools. School officials are interested in knowing how successful the experiment was, and if the new curriculum should be incorporated district-wide.

Three years ago, the school district rolled out the experimental curriculum to 100 of the 400 elementary schools in the district. Those 100 schools were selected via a simple random sample. Your working budget is not large enough to collect data on the population of 400 schools. You can only afford to collect a sample of 80 schools.
Unless the question states otherwise, conduct all analyses at the 95% confidence level (α=.05).

1. You think that the best sampling strategy is stratified sampling. You'd like to list the characteristics of schools in this district and then randomly select 80 schools that roughly match the demographic characteristics of the entire population of schools in this district. Forty (40) of these would come from schools that had the experimental curriculum, and 40 would come from schools that kept the old curriculum.
However, people from the school board have made the following statements regarding sampling:
A. One well meaning school board member has argued that "If it ain't broken, there's no reason to fix it! We should sample 80 random schools with the old curriculum; as long as the students in those schools are performing acceptably, there is no reason to change curricula."
B. A second school board member has argued that "since you are looking for improvement, what you should do is simply choose the 80 schools that had the biggest improvement from three years ago to now and look at how many of those used the new vs. the old curriculum."
C. Finally, the School District Superintendent has told you that "as you know, I'm currently running for reelection, and it is important that this research turn out a certain way. I have a list of schools that I personally handpicked for your sample that I think really represent what is going on here."
Describe what is wrong methodologically with each of the three suggestions you received from the various school board members. You will want to focus on issues such as sampling error, the sorts of biases that will be introduced from such sampling methods, how you might expect to see those biases manifested in the data/data analysis, and how these issues will affect your ability to comment on the district's original question as to whether or not it would be a good idea to roll out the new curriculum to all schools in the district.

2. Having convinced the board members that a stratified sample is the most appropriate, you collect data from 80 schools. The collected data is provided in this file on the worksheet labeled "Data."
The following variables are contained in the data set:
? chgtestscores is the difference between average school test scores three years ago and average school test scores today. A positive value for chgtestscores indicate an increase in test scores at the school as compared to 3 years ago, while a negative number indicates that the school is now performing worse on these tests.
? cirriculum is a categorical variable that can have a value of either "old" or "new." New means that the experimental curriculum is being used.
? income ($000) represents the average annual income (in thousands of dollars) of the households of students from each school.
? school is simply the ID number of the elementary school.
The first step to your analysis is to generate some descriptive measures. For each of the following, create a chart or graph that best displays the data:
A. Show the makeup/composition of your sample by curriculum type (old vs. new)
B. Show the distribution of the change in test scores (chgtestscores) across all 80 schools.
C. Show the distribution of income across all 80 schools.

Additionally, you want to generate summary/descriptive statistics for your variables. You must show a minimum of 5 measures for each variable.
D. Create a table that shows summary/descriptive statistics for the chgtestscores and income variables using all 80 schools.

E. Create a table that shows summary/descriptive statistics for the chgtestscores and income variables separated by curriculum type.

F. Based on the graphs and tables created in parts a-e, what preliminary conclusions can you draw regarding the effectiveness of the experimental curriculum?

3. One of the criticisms levied upon the old curriculum is that it is outdated. It is so outdated, the board members argue, that it is causing standardized test scores to fall. You decide to test this hypothesis.

A. First, state the null and alternative hypotheses (H0 and H1)
B. Next, perform the statistical test of the hypothesis that the mean change in schools using the old curriculum is less than 0.
C. Calculate the p-value associated with your test statistic from Part b.
D. Interpret your results.

4. Because the school board's primary concern is whether or not the experimental curriculum leads to better standardized test scores, your next step is to conduct a simple analysis comparing test scores from schools with the old curriculum to the test scores from schools with the new/experimental curriculum.

A. Conduct an ANOVA to evaluate whether or not there is a significant difference in test scores between schools with the old curriculum and schools with the new curriculum.

B. Summarize and interpret the results of your test.

5. One board member is concerned about the impact of the experimental curriculum on schools in low income neighborhoods. To investigate his concerns, you need to run a multiple regression model.

A. Create a dummy variable for the new/experimental curriculum. Also, create an interaction variable that interacts your experimental dummy variable and the income variable.

B. Run a multiple regression that includes the experimental curriculum dummy, income, and the interaction variable as independent variables. Chgtestscores will be your dependent variable.

C. Calculate predicted values for the chgtestscores variable under both the new and old curriculum for income levels of $15,000, $30,000, $60,000, and $120,000.

D. Summarize and interpret the results of this model. What do you tell the board member about the effect of the new curriculum across different income levels?

6. Shortly after you publish your findings in a report, you receive a call from Mensa, an organization whose members have high IQs. Mensa wants to open a series of schools for gifted kids across the country; children with IQs above 160 would be offered places at these schools completely free of charge. Mensa is intrigued by your research, and wants to know, based on your findings, whether or not they might expect similar results if they use the experimental curriculum at its new chain of schools. What do you tell Mensa and why?

7. The variable the school board is most interested in understanding/explaining is the change in school-wide standardized test scores. You were also given variables that indicated the curriculum type and the income of the households of students from each school. If this were a real research project, what other data would you collect to use as control variables? Why?
In other words, aside from income and the curriculum type, what other variables might you collect that could have an impact on the change in test scores? Give at least 4 examples. For each variable you list, explain why you would include it.