Once you've created a format and drafted the materials, you need to test them to find out how they function in the real world, in your school or district.
This is an extremely important step. To understand why, consider existing curriculum. Let's take the example of 4th grade teachers who have had 30-year careers and have therefore taught specific units—say, rocks and minerals—30 times in their entire career. Each year, they might adjust the lessons a bit, but for the most part they're going to stay the same. If there are 400 4th grade classrooms in your district, then that's a lot of data you have about that unit: 400 x 30 opportunities to collect information on how well this lesson works. But what is your district's capacity to gather, analyze, and innovate off that data, and who will be responsible for those tasks?
The point is that if we hope to acquire a useful amount of data about how units function in the amount of time allotted to designing and rolling out new curriculum, we must be intentional about collecting enough information to base recommendations on. We need more than the experience of one or two teachers. We need to create replicable curricula that everyone can use with some level of consistency in student learning outcomes. Therefore, you must engage in actual testing that allows you to answer the questions listed below:
While testing materials, your district must attempt to answer a range of questions about the effectiveness of the newly formatted and drafted curriculum.
Again, each of these questions is meant to answer one overarching question: How does this curriculum work in the real world and, more importantly, how can it be broken?
So say we were to address the question "Can students be successful with varying outcomes given the materials provided?" Well, you don't want students engaging in engineering with a map in hand. That's not engineering; it's crafting. It's a toy-building class, and it does not lead to mastery. Instead, students need to be able to bring their own ideas to life, which means there should be natural variation in the small groups. To give students the space to do this successfully, we must minimize the potential for misinterpretation or accidental removal of learning value from lessons by resorting to the old "I do, we do" model.
Because when that happens, what's left for the student to do at the end of the day? What is left for the student to actually experience for the first time? What will actually challenge their skills? Nothing. If you're creating a resource, you have to think about how you're going to offer a full release responsibility—creating a "you do, we do" scenario. There should never be any "I do"s. Never create something for students to parrot when the Next Generation Science Standards performance expectations require that they be the ones engaging in developing critical thinking skills and creative thinking skills. The value and opportunity for growth exists where we allow for productive struggle.
Next, we must consider whether or not the hands-on materials are age and cost-appropriate. If what you've created involves a first grader putting a piece of string through a small hole and tying a knot on it, that's not age appropriate. Similarly, if teachers are spending money out-of-pocket, the materials are not cost appropriate. Relying on teachers to go out and buy supplementary materials in order to teach lessons is not professionally appropriate and will not work consistently, because it isn't a teacher's job and they might decide one year not to do it, creating a hole in learning. For this reason, materials must also be available in sufficient quantity. If you can't find enough of these materials, you must avoid relying on them.
One district we observed, for example, had built a curriculum around frog embryos that they scooped out of a pool cover in someone's backyard. This is no joke: They actually relied on the frog embryos for lessons. The problem is that person might move, fill in their pool or otherwise compromise the dependability of the embryo supply. If that happens, that entire piece of the curriculum is gone. So you have to think about materials from a risk perspective: If you lose them, what will happen to the curriculum? If the answer is that it would no longer work, then those aren't good materials design.
The nonfiction reading level must also be appropriate, or else it won't effectively challenge students to move beyond their existing comfort levels and develop new understanding. Lastly, it's important to think through who will be in charge of buying, storing, replenishing and distributing all the materials and how live materials will be handled. The larger your district gets, the more complex these questions and answers are going to become. On our end, a lot of time goes into testing materials and seeing how these questions get answered at different phases of the testing process.
