Archive | Research and Evaluation RSS feed for this section

Principal 2.0 (new book!)

Principal 2.0

Remember last September when I made available for free my most recent book chapter, Supporting Effective Technology Integration and Implementation?

Well, that chapter’s still available to you for free, but I’m pleased to announce that the rest of the book is now available as well. Featuring a diverse set of authors, perspectives, and topics, there’s something for everybody in Principal 2.0: Technology and Educational Leadership.

Here are the chapter titles and authors (including all 4 CASTLE directors!):

  • Foreword: Ready Set Go! with Educational Technology, Governor Beverly Perdue
  • Introduction: Making the Case for Principal 2.0, Jennifer Friend and Matthew Militello
  • Augmenting Educational Realities, John Militello
  • The Role of For-Profit Firms in the Educational Ecosystem, Michael J. Schmedlen
  • Generation X Meets Generation Y: Reflections on Technology and Schooling, Jennifer Friend and Alexander David Friend
  • Zen and the Art of Technology in Schools: Multigenerational Perspectives, Matthew Militello, Ronald Militello, Dominic Militello, Luke Militello, and Gabriel Militello
  • Digital Storytelling for Critical Reflection: An Educational Leadership Story, Francisco Guajardo, Miguel A. Guajardo, John A. Oliver, Mónica M. Valadez, and Mark Cantu
  • Engaging Youth Voice: Collaborative Reflection to Inform School Relationships, Processes, and Practices, Christopher Janson, Sejal Parikh, Jacqueline Jones, Terrinikka Ransome, and Levertice Moses
  • There’s an App for That: 50 Ways To Use Your iPad, April Adams and Jennifer Friend
  • Student-Owned Mobile Technology Use in the Classroom: An Innovation Whose Time Has Come, Tricia J. Stewart and Shawndra T. Johnson
  • Affective Learning Through Social Media Engagement, David Ta-Pryor and Jonathan T. Ta-Pryor
  • The Central Texas Community Learning Exchange Digi-Book: Fostering School and Community Engagement Through the Creation of a Digital Book, Lee Francis, IV, Mónica M. Valadez, John A. Oliver, and Miguel A. Guajardo
  • Leaders Online: Enhancing Communication with Facebook and Twitter, John B. Nash and Dan Cox
  • Balancing Effective Technology Leadership with Legal Compliance: Legal Considerations for Principal 2.0, Justin Bathon and Kevin P. Brady
  • Connected Principals: In Pursuit of Social Capital via Social Media, Candice Barkley and Jonathan D. Becker
  • School Leaders’ Perceptions of the Technology Standards, Matthew Militello and Alpay Ersozlu
  • Supporting Effective Technology Integration and Implementation, Scott McLeod and Jayson W. Richardson.

A big thanks to Drs. Matt Militello and Jennifer Friend for inviting me to be a part of this publication. Happy reading!

State department educational policy advocacy: “Evidence-based” or puffery?

I teach. What's your superpower?

One of the Iowa Department of Education’s favorite phrases is ‘evidence-based,’ as in “schools and educators should be adopting ‘evidence-based’ practices.” That makes sense on its surface, right? So when the DE issued a press release today touting two Iowa districts that had adopted the National Institute for Excellence in Teaching’s (NIET) Teacher Advancement Program (TAP) – which focuses on teacher leadership and performance-based teacher compensation – I started digging around in Google Scholar for research about the program. I don’t live in the world of teacher leadership/compensation and thus don’t know much about TAP other than that it appears to support the DE’s policy proposals this legislative session (and that Dr. Brad Buck, an amazing school leader that I greatly respect, is the superintendent of one of the two districts). For all I know, TAP could be amazing or it could be puffery

What did I find?

And, of course, on its web site I found research from NIET itself claiming that TAP is awesome. Indeed, there’s a plethora of publications on TAP’s web site from NIET; agencies such as the Milken Family Foundation, the Joyce Foundation, or the Gates Foundation that have funded TAP; and ideologically-oriented educational organizations. But despite TAP’s claims of ‘a decade-long track record of growth and success in raising student achievement in high-need schools‘ (page 19), the evidence from independent researchers is decidedly more mixed and leans much more pessimistic when it comes to student learning outcomes.

So is TAP ‘evidence-based’ or puffery? Given the greater preponderance of negative findings by outside, independent scholars, it appears to be extremely arguable. Given that uncertainty and its own call for ‘evidence-based’ practices, should DE be basing its 2013 education reform package on the TAP model? (and should it be touting the model via press releases to the Iowa public and media, which know even less about all of this than I do?)

We already know that DE is willing to ignore decades worth of peer-reviewed research when it conflicts with its policy advocacy (see, e.g., recent incidents regarding 3rd grade retention and cutting elementary school recess). I don’t know if TAP falls into this category of willful blindness or not. But I am wondering if the studies listed above were ever presented to those in decision-making positions so that they could make a truly informed decision.

I hope that the initiatives go well for the two Iowa districts that are trying TAP. Their educators are going to invest a great deal of time, energy, and money in the model. Hopefully they will see the results for which they are striving. Until I see further independent research supporting TAP (send it if you’ve got it; maybe I missed some!), right now I’m less sanguine about whether we should be basing tens of millions of dollars statewide to implement the model here in Iowa.

Got any thoughts on this?

[Note that there's also a bigger question here that's always worth considering: What should we do if/when our own governmental agencies fail to apply the same standards to themselves that they wish to apply to us?]

Image credit: What’s your superpower?

Text complexity in the early grades: Shooting ourselves in the foot?

Cozy reading spot

Here are some quotes from the most recent issue of Educational Researcher regarding text complexity in the early grades, one of the hallmark pushes of the Common Core State Standards:

the CCSS text complexity standards for Grade 3 appear to be aspirational, much like the No Child Left Behind Adequate Yearly Progress targets (Shepard, 2008). The small set of studies that have examined text complexity over time does not show that text complexity at Grade 3 has deteriorated. Neither is there evidence that the accelerated targets in the primary grades are necessary for high school graduates to read the texts of college and careers. (p. 47)

AND

Another potential indirect effect on students may be their motivation and engagement. The engagement of reading among American students is already low, as indicated by a 2001 nationally representative sample of fourth graders from 35 countries that ranked the United States 33rd in an index of students’ motivations for reading (Mullis, Martin, Gonzalez, & Kennedy, 2003) and 35th out of 35 countries in the revised index of attitudes toward reading (Twist, Gnaldi, Schagen, & Morrison, 2004). At present, there is research indicating that motivation decreases when tasks become too challenging and none that indicates that increasing challenge (and potential levels of failure) earlier in students’ careers will change this dismal national pattern of disengagement with literacy (Guthrie, Wigfield, & You, 2012). (p. 48)

AND

Will the intended outcomes of higher levels of literacy for all students be realized by setting the bar arbitrarily at third grade? Our review suggests that the unintended negative consequences could well outweigh the intended positive outcomes. (p. 49)

AND

Increasing the pressure on the primary grades – without careful work that indicates why the necessary levels are not attained by many more students – may have consequences that could widen a gap that is already too large for the students who, at present, are left out of many careers and higher education. How sadly ironic it would be if an effort intended to support these very students limited their readiness for college and careers. (p. 49)

Hiebert, E. H., & Mesmer, H. A. E. (2013). Upping the ante of text complexity in the Common Core State Standards: Examining its potential impact on young readers. Educational Researcher, 42(1), 44-51.

Image credit: “Cozy” reading spot

5 videos on connected learning from the Digital Media and Learning Research Hub [VIDEOS]

Here are four very powerful videos from the Digital Media and Learning Research Hub that are guaranteed to make you think hard about learning, teaching, and schooling. You can watch them all in less than half an hour. My quick notes from the videos are included underneath each one…

Engaged (7 minutes; Connie Yowell)

  • we are fundamentally starting with the wrong questions
  • we start with learning outcomes – and content defines everything – rather than “what is the experience we want kids to have?” 
  • our core question is around engagement; if you ask “is a kid engaged?”, you have to pay attention to and start with the kid
  • we have to make room for curiosity, we don’t have enough opportunities for kids to take things apart and wonder about them
  • little opportunities to fail and iterate are also opportunities to play with identity
  • we need opportunities to explore who we are in the world and how the world works, particularly as teenagers
  • we so decontextualize learning for kids, we’ve forgotten we have a passion for learning
  • in school they could care less, but in complex games kids demand that they learn how to do something so they can move on
  • as adults, we have to deeply connect content and students’ activity, otherwise learning has no meaning

Everyone (7 minutes; Mimi Ito)

  • we give responsibility for learning to professionals instead of remembering it’s the fabric that frames all of our interactions with everybody
  • connected learning networks force us to fundamentally rethink what we think is the problem and goal of education
  • it’s about expertise that’s widely distributed; anybody can help somebody else get better at something
  • if you have an educational system that always tell students what to do, you’re not building their capacity to make effective learning choices themselves
  • we used to have capacity bottlenecks for learning, so you had to go to school or a library – now we don’t have that problem but we still act as if we do
  • education isn’t bound to particular institutions anymore, it can happen anywhere
  • how does a kid find a mentor or peer that helps them develop their interest, make their interest relevant, find a sense of purpose, etc.
  • how do we use the capacity of the network to bring people together who want to learn together?
  • everybody can participate in a connected learning model
  • the great side benefit of interest-based, connected learning is that it fosters social connection and well-being: fulfillment, belonging, and purpose

Play (7 minutes; Katie Salen)

  • play creates for people a reason for them to want to engage
  • body and spirit are transformed by play
  • play is a state of being, a very different state of mind, openness to ideas and other people
  • not a closed, rules-bound place – the openness of the play space is extremely important
  • play is one of the most fundamental human experiences
  • play is a practice space, we play to get better at something, it helps us build confidence
  • kids are driven to want to share with you what they’re doing, what they’re making, what they’re learning
  • at school, we cordon off a time for play (recess) and then you’re not doing that anymore
  • when you get older, play becomes embedded in objects (video games), you can activate play when you pick up that object
  • when we’re young, play is the frame for how we experience the world
  • adult life becomes about a set of responsibilities rather than a way of engaging your soul in the world

Creative (5 minutes, Nichole Pinkard)

  • we’re just now getting to the place in America where we realize it needs to be different everywhere, not just in some places
  • we have to completely overhaul how we think learning happens, where it happens, and what people are capable of
  • technology transformations show us the world is going to be different
  • they are going to have to be more nimble and more proficient with technology to communicate and to learn, or they’ll be a new form of illiterate
  • we no longer live in a world where you can only write and read text and you will be successful
  • we have to teach these new literacies and then let kids be creative in how they express themselves with these literacies
  • schools always have been about ‘the right answer’
  • now we care more about how kids find information, think about information, communicate information

The DML Research Hub also has an 8-minute summary video, Essence, which includes some of the best pieces from each video above plus some new stuff.

  • there’s no longer a promised future for all kids
  • how do we create environments that delight learners at all ages?
  • open up the question of who contributes to learning
  • how do we help kids grow up to become curious, engaged citizens?
  • kids say over and over that schools are (merely) a node in their network of learning
  • we have an embarrassment of (information) riches but we still have to figure out how to bring those pieces together
  • learning principles need to start with the idea of connectedness 

Finally, be sure to check out the core values, learning principles, and design principles of connected learning:

  • Core values: equity, social connection, full participation
  • Learning principles: interest-powered, peer-supported, academically oriented
  • Design principles: production-centered, openly networked, shared purpose

See also the infographic below. There’s a lot here to digest. Thoughts?

Connected Learning

Publishing in a print journal is only minimally ‘public’

Let’s face it. “Publishing” one’s work in a print journal that languishes on a dusty shelf in an academic library is a fairly minimal definition of making something public. So the real issue here is that we are simply not accustomed to our work being discussed in public. Ever. Period.

Alex Reid via http://www.alex-reid.net/2012/10/twitter-academics-and-the-public.html

Parents believe that technology can improve students’ learning

research from the CEA showed that three-quarters of U.S. parents of students in grades K-12 “agree” or “strongly agree” that technology greatly improves students’ learning experiences, and that two-thirds “agree” or “strongly agree” that they personally have seen their children benefit from educational technology implementation.

Further, more than half of respondents from the general public “agree” or “strongly agree” that K-12 students should be provided a computer for their education, the research showed.

via http://blogs.edweek.org/edweek/DigitalEducation/2012/10/ed-tech_among_five_tech_trends.html

When children are simply taught

studies have found that when children are simply taught, they don’t explore and test multiple hypotheses

[In one study of preschoolers,] an experimenter held a toy that had four tubes. Each tube did something different — for instance, one lit up and one made a squeaking sound.

In one case, the experimenter accidentally made the toy squeak by bumping into it and then left the room. The children experimented with the toy and figured out the three other features.

But when the experimenter made the toy squeak on purpose and then handed it to a child, he or she simply repeated what the experimenter did and never explored the toy’s other features.

via http://www.nytimes.com/2012/10/02/science/scientific-inquiry-among-the-preschool-set.html

“Time on task” is less important for higher-level understanding and creativity

the nature of the task helps to determine the relationship between time and achievement. It turns out that more hours are least likely to produce better outcomes when understanding or creativity is involved. “How much is learned by rote is a direct function of time and effort,” acknowledges literacy expert Frank Smith. “But when the learning is meaningful we learn much faster. . . . Having to spend long periods of time in repetitive efforts to learn specific things is a sign that learning is not taking place, that we are not in a productive learning situation.”

Sure enough, researchers have found that when children are taught to read by focusing on the meaning of the text rather than primarily on phonetic skills, their learning does “not depend on amount of instructional time.” In math, too, even the new-and-improved concept of “engaged” [time on task] is directly correlated to achievement only if both the activities and the measure of achievement are focused on rote recall. By contrast, there is no “linear positive relationship for higher level mathematics activities, including mathematical applications and problem solving.”

Alfie Kohn via http://www.joebower.org/2012/08/limits-of-time-on-task.html

Teachers are only part of the student learning equation

Here’s a slightly-modified version of a comment I left over at The Des Moines Register regarding teachers’ impacts on student learning outcomes…

Sure, what happens in the classroom matters. But peer-reviewed research shows over and over again that between 2/3 and 4/5 of student achievement is based on non-school factors. Schools only contribute about 20% to 33% to students’ overall learning outcomes.

In addition, teachers are only part of the school equation. They’re the most important part, but non-teacher factors such as administrators, curriculum, other students in the school, available learning resources, and so on also impact student achievement. So teachers are responsible for about half (or so) of the school impact, but the rest lies outside their domain.

When you add all of this up, good teachers clearly are absolutely critical to student academic success. But their overall impact on student learning falls around 10% to 17%. Other in-school and out-of-school factors account for the rest. What this means is that – the occasional tale of heroic, exceptional teachers and schools aside – we should be making state and national policy based on what the research shows generally occurs, not exceptions, anecdotes, personal intuition, or unsubstantiated policy/political claims. And we definitely should not be holding teachers 100% accountable for outcomes for which they’re only 1/6 to 1/10 influential.

We need a much broader (and smarter) conversation about what it means to educate our nation’s children.

Image credit: Bigstock, Teacher and students

About those PISA scores…

Here are PISA scores for 15-year-olds as typically presented by politicians. When you see these, it’s easier for Americans to be more alarmist (oh no, others are ahead of us!).

Average scores

AveragePISAscores 

Percentage of high scorers

 PercentagePISAhighscorers

But we also should look at the data another way.

Number of high scorers

NumberPISAhighscorers

Our size benefits us, and we need to remember that too. [of course we have many more LOW scorers as well…]

More data at American Achievement in International Perspective.

Switch to our mobile site