Our professional development programs are evaluated annually by Joellen Killion, Deputy Executive Director of Learning Forward. As a framework, she considers both the Alabama Quality Teaching Standards and Learning Forward’s own nationally recognized Standards for Professional Learning.
Drawing on confidential interviews, observations and ABPC program records, Joellen’s 2010-11 evaluation looked at how ABPC’s actions are promoting change among schools and districts that participate in our Powerful Conversations Network, Key Leaders Network and Superintendent Leadership Network.
We thought it might be most helpful to summarize the evaluation’s key findings and recommendations in a question and answer format, with Cathy Gassenheimer answering the questions. (If you’d like to review the entire evaluation, please get in touch with Cathy at 334-279-1886.)
What was the main focus of the 2010-11 evaluation?
We worked with Dr. Killion to create four focusing questions. First, we wanted to take a measure of how districts have used the professional learning opportunities offered through our networks and direct services to improve student achievement and spread effective practices to other schools in their systems. We also wanted to look at how well we’ve leveraged our programs to support other statewide efforts, such as our work with the Alabama Reading Initiative, the Alabama Math, Science, and Technology Initiative, and the new teacher evaluation system, EducateAlabama. And, perhaps most urgently, we wanted an objective analysis of our capacity to scale and sustain our efforts to help Alabama schools improve student achievement.
What does the Learning Forward evaluation tell us about the success of the PCN, KLN and SLN learning networks?
It’s clear that we are making a difference. Participants in all three networks – The Superintendent Leaders Network (SLN), the Key Leaders Network (KLN), and the Powerful Conversations Network (PCN) – report that they take what they have learned and use it back home.
Superintendents value the learning time they have together and the focus on leadership. Districts involved in the Key Leaders Network – organized for district and school instructional leaders – report that they adapt what they learn in KLN and use it in their schools. And school teams in our Powerful Conversations Network often talk about the importance of the “lateral learning” that occurs at the meetings. In other words, they like learning from colleagues from other schools and districts, particularly during the spring site visits.
Here’s some of what Dr. Killion had to say about the value of the professional learning experiences ABPC both provides and facilitates:
- Network participants commended ABPC staff for their expertise, dedication, commitment, and thoughtful planning in providing effective professional development during all meetings and the staff’s ability to link ABPC efforts with major statewide initiatives.
- In interviews, informal focus groups, and in feedback summaries from SLN, KLN, and PCN meetings, participants repeatedly cite increased learning in terms of higher graduation rates, increased test scores, more active engagement, learning at higher levels, personalized learning, and other benefits when schools and districts participate in ABPC networks.
- Teachers who participate in PCN have opportunities for more intensive professional development. Teachers are more engaged in collaboration and reflecting on their teaching and more likely to become instructional leaders in their schools. One district leader acknowledged that the way the district handled professional development was transformed through participation in ABPC programs.
The challenge, of course, is to document results in improved teaching and learning across the board in all participating schools. So, this year, we are going to work hard to gather additional data that can further connect our work to improved teaching and learning. I say more about that below where we discuss recommendations.
The evaluation also looked at ABPC’s direct-service partnerships with several school districts. What did you learn about the effectiveness of that model?
We are working intensely in two districts: the Talladega County Schools and the Tarrant City Schools. One district, Talladega, is largely a rural school system, while Tarrant’s students live in an urban setting. Both districts have large numbers of students living in poverty. (For Tarrant, it is 97% and in Talladega County, the average is about 70% countywide with some schools as high as 90%.)
Dr. Killion noted that the “intensive support is making a substantive contribution to overall district improvement efforts. Results for both students and teachers are clearly evident. Teacher leaders report increased collaboration among teachers. Onsite observations provided evidence that teachers are using formative assessment practices as a part of routine instruction. Teachers have a common language to talk about teaching and learning. School leaders and district leaders report having a stronger relationship focused on specific strategies for improving teaching and student learning.”
Talladega County is involved in every facet of our network activities. Dr. Suzanne Lacey is a member of the Superintendent Leaders Network. Talladega County participates in the Key Leaders Network, and many of their schools are in the Powerful Conversations Network. Additionally, Julie Hannah and I spent six days with Talladega district and school leaders where we focused on formative assessment (which I like to describe as “checking for understanding”) for both students and adults (EducateAlabama).
In Tarrant, our focus this past year has also been much the same: formative assessment to improve teaching and learning. Jackie Walsh and I provided six intensive professional learning sessions last year for district and school leaders. In both Talladega and Tarrant, our ability to get to know these leaders on a deeper level enabled us to customize our professional learning in more meaningful ways. The trust level shared between participants and the facilitators is strong.
Dr. Killion concluded that:
“Evidence of formative assessment abounds in the districts working more intensely with ABPC and is reported by others in their schools. In every classroom visited, formative assessment was a routine and constant part of teachers’ practice and students’ learning. Principals report seeing formative assessment in classrooms more frequently. Teachers, they report, have a strong sense of what students know and how to support their learning. Those interviewed report seeing evidence of protocols and practices learned in PCN in place in classrooms they visited. Instructional coaches teach the strategies to teachers.”
For several years, ABPC has facilitated cross-district visits by school leaders to experience the Instructional Rounds process. What did Learning Forward have to say about the impact of that project?
Instructional Rounds in Education is a network approach to improving teaching and learning, developed by four leading school improvement experts at Harvard. It’s intended to help education leaders and practitioners develop a shared understanding of what high-quality instruction looks like and what schools and districts need to do to support it.
During the Rounds process, we found that participants were making connections to their own practice. As Dr. Killion notes: “The instructional round process provided multiple opportunities for teacher learning both as they observed and hosted instructional rounds.” Observer comments included: ‘This has been a wonderful experience. I reflected on my practices as I went through this process. Seeing student engagement from the outside really helps me become aware of what really isn’t going on when I thought it was in my own classroom.’
The Rounds process also had a profound impact on the host schools, according to Dr. Killion. “Hosts of instructional rounds reported: ‘I felt rewarded as my students were able to explain purposes of lesson and benefits to their learning experience.’ ‘We learned so much from hosting. We can use the data collected to drive our PD for next year.’”
What suggestions did Learning Forward make for strengthening ABPC programs?
We need to continue to “tighten” the connection between our Networks to maximize our impact. Dr. Killion particularly emphasized linking the Superintendent Leaders Network more closely to our other Networks. We’ve already begun to put that recommendation in place by adding five new districts to the Key Leaders Network, all led by SLN members. We also plan to be more intentional about informing SLN members of the work going on in other ABPC networks.
Dr. Killion also urged us to make sure that KLN members strongly support their schools involved in the Powerful Conversations Network. As we prepared for this year’s work, we asked KLN district leaders to commit to doing exactly that. We plan to ask for descriptive evidence of their support at each meeting, both as a way of maintaining a focus on results and for sharing good ideas about maximizing what we all learn together through networking.
As we seek support to continue the valuable collaborative learning going on in our networks, it’s important to be able to connect adult learning with results for students. We’ll be pursing a 4-level evaluation this year where we ask schools and districts to provide us with evidence of impact, both through improved teaching and student achievement.
The evaluation’s final focusing questions reads: “What is the capacity of ABPC to scale and sustain its efforts to improve student achievement?” What did Learning First conclude and how will their findings shape ABPC’s work in the coming year and beyond?
In order to maximize our organizational resources, we must ask our network partners to share accountability for effectiveness. Dr. Killion identified this issue very clearly in the evaluation: “ABPC’s greatest challenge is taking action to ensure that parallel to the privilege of membership in any of its networks is accountability for results.”
One of my education friends told me long ago that it was important to make sure that we designed professional learning opportunities for results. She said, “It can’t just be about professional development for your participants’ own delight; it must be about taking what they’ve learned and using it in their classroom, school building or district.” We may have to make some tough decisions at the end of this school year about how to allocate our limited resources, and it will be important for all districts that enjoy working with us to be able to show some evidence of impact.
ABPC has two full-time staff members: Magan Merritt and me. Rennie Crabb, who is the office manager for A+, is also an essential part-time member of our team. Our work is greatly enhanced by two exceptional consultants: Dr. Jackie Walsh, who designs most of the professional learning we deliver, and John Norton, our communications advisor, who brings knowledge, experience, pithy writing, and passion to our work. Our other excellent consultant, veteran Alabama education leader Julie Hannah, recently returned to the Jefferson County Schools as interim deputy superintendent.
People marvel at our ability to do so much with so few resources. I’ll admit that we work hard and that most weekends I spend a lot of time at my laptop or reading one of my “new favorite books” about an important education topic. What they don’t often see is the great partnerships that help maximize our work, including our own A+ College Ready program staff, and the Alabama State Department of Education —particularly the Alabama Reading Initiative; the Alabama Math, Science, and Technology Initiative; and the Leadership and Evaluation Section. Our district partners are also wonderful colleagues and assist us in planning the professional learning by providing valuable feedback both before and after each session.
Dr. Killion offered the opinion that the leverage to increase ABPC’s capacity rests with its ability to link its networks and focus on significant statewide reform initiatives. She believes that with the advent of Common Core State Standards and upcoming changes in assessment systems, the state, districts, schools, educators, and students will increasingly depend on ABPC to increase our reach.
We strongly believe that our work can be scaled through these partnerships and we’re excited about the possibilities. But as we grow we will clearly need to add more people to our work. That, of course, is contingent on funding, which is always a challenge for a small nonprofit.
As you look over more than a decade of ABPC programs and partnerships, do you have any summary thoughts?
Our work is so rewarding. We are inspired when we work side-by-side with committed educators who hold high expectations for every student. On the other hand, progress is slow and we worry about children in Alabama who are not yet getting the type of excellent education they deserve and need to be happy, successful and productive adults. So we’re guided by both a sense of urgency and the affirmation we feel when we see positive results.
In the days to come, when the challenges sometimes seem overwhelming, I know we’ll find solace in some closing words from Dr. Killion’s 2011 evaluation. Speaking of educators’ engagement with our networked communities and other professional learning opportunities, she wrote:
The efforts of ABPC have a deeply transformative effect in districts where there is commitment and effort to improving student learning.
