January 23, 2009
Here’s my latest Science Teaching Tips podcast — As any teacher knows, the ability to ask good questions — and use students’ questions — is a valuable skill to have in your teaching toolbelt. In this podcast, TI staff biologist Karen Kalumuck describes how she tries not to answer every question that’s asked during a class, however tempting it may be. Instead, she’s learned how to guide her students to discover ideas for themselves.
Episode 66: That’s a good question!
December 16, 2008
That long blog post title is the summary of a very interesting piece of research just written up in Cognitive Daily. This is worth going over and taking a peek at the original post, because it’s quite an interesting piece of research.
The research question was whether people’s memories follow a predictable pattern. After all, we seem to remember more stuff from our 20′s and 30′s than, say, our 60′s. The researchers found, basically, that more life-defining events happen in our 20′s and 30′s (like marriage, having kids) and those events create more long-lasting memories. I’m grossly paraphrasing here, so take a look at the original post for the clearer picture.
Cognitive Daily says:
This corresponds well to other researchers who have found that immigrants remember more details about the years surrounding their time of immigration than non-immigrants. So if you immigrate in your 30s, you’re more likely to have memories from your 30s than someone who immigrated in her 20s. Other studies have found a memory bump in people from Bangladesh corresponding to a period of political unrest in that country. So it seems that our memories are affected more by the events in our lives than just the physical development of our brains. We’re not all destined to remember more of our teens and 20s than other years; we’re just more likely to experience significant, life-changing events in those years than others.
November 20, 2008
This is the last in a series of three posts on Dan Schwartz’s work on preparation for future learning, or helping students learn skills instead of rote facts so that they can apply their knowledge to new situations. All pictures in this post are courtesy of Dan Schwartz.
In the previous post, I showed Dan’s use of contrasting cases in helping students understand density and ratios. Why is it important to show students different cases, instead of the best single example of something? Well, he said, think about perception. Consider this circle:
We immediately recognize it as a circle. It is, after all, not a square.
But, in fact, it is many things. It’s a empty circle. It’s a circle created with a black line. It’s a largish circle. Here are a bunch of contrasts to this circle:
We’re abstracting “circle-ness” from the single example, but that’s because we recognize circle-ness already. These contrasting cases would be important if we were first learning about circles.
Here are some contrasting cases of something familiar to us:
After all, what is the best way to teach Japanese speakers to say the sound “L,” which doesn’t exist in their language? Give them the purest example of an “L” sound that you can find? No, it’s to let them hear “R” and “M” and all the other sounds, so they know what the “L” is NOT in addition to what it IS.
But, this is what we do in instruction! We give students the purest example of something that we can. Consider, for example, this picture.
This is a perfect example of this breed. Now, tell me which one of the following is the same breed?
An expert will look at the width of the ears, the curve of the nose. But a novice can’t look at these pictures and see the immediate resemblance to the example picture. (I forget which one was the correct answer, but I think it’s the last one. The hair length is an extraneous feature, the ear shape is most important.)
It would have helped if, first, an expert had used the following picture with contrasting cases to help you learn about ear shape (what does “rounded” ear shape look like? How wide is “wide”?). You need to be oriented to understand the key structures in what you’re seeing. You can’t just look at the picture below and learn from it, though — a bunch of different examples are confusing to a novice. The expert’s role is to help them make sense of the different cases.
An example activityHere, for example, is his activity where he asks students to invent a reliability index for a pitching machine. He gives them several different cases so they have to find a general solution which fits all these cases. This, after all, is what we do in science – to find a general solution that fits many cases.
In my previous post, I gave his activity for teaching density using clowns in buses.
The way he uses these in the classroom is to have students explain their classmates’ solutions to each other. That means that each student’s solution has to be written clearly enough so that someone else can understand it. This act of public “publishing” of the results gives students a bit more motivation to come up with a good solution. On the other hand, the goal of this task is NOT to come up with the “right” solution! It’s to prepare students to understand the expert solution (in this case, the idea of variance) when it’s presented.
As experts in a subject, we know an amazing amount. What we’ve learned has been compressed into a bunch of huge steps. We don’t recognize the huge number of things that we’re doing when we do what seems to us to be a single step (such as computing a ratio). We need to decompile our knowledge for the novices. In order to do this, it’s good to have an intelligent novice around — someone to ask us a bunch of questions at every step so that we can see what it is that we are doing in any task. Once you’ve discovered some key, fundamental idea that is needed to solve the problem, that’s a great place to put an invention activity. Examples are density, vectors, variance, and other fundamentals.
Dan’s main point is that our assessments need to change in order to use this kind of instruction. If we value students’ showing that their learning is adaptive, we have to give them a chance to demonstrate this on a test, to demonstrate an expert level of perception.
What do I mean by expert level of perception?
What do the images below say to you?
The novice answer (“car,” “bird”) is not very precise.
The expert answer (“2007 BMW X5″ or “indigo bunting”) is much more precise, and relies on deep recognition of various features. We should test students on this more broad ability to apply their knowledge. For instance, geology students should be able to extract some important features from this picture of a landslide:
This doesn’t have to be a perceptual test — in the previous post, the “green people” vs. “blue people” example relied on students ability to recognize the variability in a data set.
I think this stuff is incredibly powerful. Let me know of any more activities that you come up with or you know about!
November 17, 2008
In my last post, I wrote at length about Dan Schwartz’s work about teaching students how to learn by having them create a solution to a problem before you give them the standard lecture about how to solve that kind of problem. I wanted to give you an example of this kind of “Preparation for Future Learning” activity, in addition to the batting machine example in the previous post. All images are courtesy of Dan Schwartz at AAA Lab.
This one is to help students learn about density. The task is below.
And here are the graphics for the task.
The key to notice here (and in the previous batting example, though I only showed you one example of the batting machines) is that he uses contrasting cases to teach this concept. There are different buses with different amounts of clowns. These cases are chosen carefully so that the student must come up with a solution that satisfies all these different cases. For example, the number of clowns in the bus does not distinguish between the very first and very last cases shown on this sheet (for which the answer would be “2″ for both cases, which are clearly different).
He found that those students who first invented this density ratio were better able to then use this knowledge to understand spring constants (another ratio) than those were were just told the formulas for density. That data is shown below.
More on how to write your own preparation for future learning activities in the next post….
November 17, 2008
We had a visit from Stanford education researcher Dan Schwartz last week, and what he told us about how people learn just rocked my world. I always enjoyed his work (and it was a real pleasure to tell him how much he’s influenced my thinking about education), and have blogged before about his A Time for Telling paper. Still, spending many hours with him over two days was a transformative experience for me. Let me try to tell you why.
All images in this post are courtesy of Dan Schwartz. His research website is here.
For example, students can learn how to do permutation problems using, say, cars or marbles as examples. But when you ask the marble-trained students to answer a question using cars, they struggle. Similarly, we see our own students do well on homework and chapter-level tests, but not on the final. They know the formulas and ideas, but don’t know how to apply them. This has been called a problem of transfer from one domain to another.
But, he argued, how is it that we could fail to transfer what we know? After all, we learn something at home and we bring it to work, or we learn something at home, and we bring it to school. We seem to transfer all the time. And, as my colleague Noah Finkelstein has argued, in order to believe in transfer, we have to believe that there is some thing to be transferred. What is transferring? Some static little packet of knowledge? There’s no tangible chunk of knowledge that we can bring from place A to place B. Knowledge is about skills and process and understanding, it’s not a static thing.
So, Schwartz argued, it’s not really a problem of transfer. The problem is that we’ve trained people to do things quickly – efficiently – not to adapt to new situations. We train people to recall lists of words quickly, or take timed tests, or tell us what causes the seasons when asked on the street. So we’ve trained efficiency over adaptation. While efficiency is important for routine tasks, experts readily adapt their knowledge to a new situation.
Schwartz did a fascinating study to see what helps students learn to adapt to new situations.
So, how did they each do on assessment? On a traditional factual test, group 1 (reading and lecture) and group 3 (graphing then lecture) do equally well. Group 2 (graphing only) did very poorly — without some expert guidance they didn’t really learn much from just playing around. Those data are to the right.
OK, so does that mean that it’s equally good to have students
read and hear a lecture as to play around with the data before hearing the lecture?
Nope… he then gave them all a test that required them to use their understanding in a new situation, and those data are to the right here. Those who first played around with the data and then heard the expert lecture did much better on that test. They were approaching adaptive expertise more quickly than the others! The differences in performance on this test, above, are stunning — these students (who, he argues, were prepared to be able to learn during the test by the instruction they were given) did more than twice as well on this test.
This is particularly appropriate to remember in math and science. Math, for example, is usually presented as an efficient way to solve problems. What if, instead, students found that math helped them understand science and manage complex problems? For example, he took a class of 9th grade students and taught them statistics by asking them to find a way to rate the reliability of pitching machines. Below are two examples of student approaches to this problem.
This forced them to create ways to deal with variability in data before being given a formula for computing variability (eg., standard deviations). He found that, even a year later, these students did better than college students in understanding formulas for variability, and were much better able to understand variability in data and its importance. Below is the task that he gave these students — those who had struggled with variability before hearing the lecture were able to recognize that the IQ scores of the green people had more spread, even though the IQ of the blue people was higher, on average.
He argued that this emphasis on efficiency is very American. We train people to become expert at routine tasks, but what we need to emphasize instead is innovative experiences. Let go of what you’re told, and try something new. For one, when students innovate a solution first, then they have a context for what they’re learning. When given the solution first, they don’t have a context for it. Telling people the answer works if they have a lot of prior knowledge (and that’s why talks at conferences, for example, can still be decent ways to get a lot of information across). But when you’re learning something new, don’t tell too soon!
You can find out more about Dan Schwartz’s work here.
October 16, 2008
I’m siting right now in a fascinating meeting for my physics education group, and we’re hearing about research on stereotype threat, which I’ve written about before. Stereotype threat is the idea that when you spark cultural stereotypes (like “girls aren’t good at math”) then those people get very stressed about their performance, feeling that however they do (say, on a math test) will reflect poorly on their group (women). That makes them perform worse on that task than they might.
It turns out that if you give women the same test and say that it’s not diagnostic of their ability (for instance, by saying that it’s just a pilot test and you’re looking for feedback to improve it), then women and men do similarly on the test. That’s because women don’t feel threatened in that case, it’s a low-stakes test.
Similarly, when you tell subjects that men and women perform differently on the test, women score worse. You’ve made them aware of the stereotype by doing that.
Interestingly, when you tell subjects that men and women perform the same on the test, women score better and men score worse (closing the gap between their scores)! For some reason, when men feel that they’re not getting some advantage from their gender, they don’t perform well.
They’ve seen similar results with:
They’ve done some work to see how understanding and knowledge of this stereotype threat can be used to help close the gap between some of these traditionally lower scoring groups. Psychological processes can have a significant impact on students’ performance — we need to be aware of that as teachers, and address it. They did some work on mentorship, for example, and found that motivation is fairly malleable based on whether the mentor gave strict negative criticism, vs negative criticism that’s buffered with positive feedback. The positive feedback appeared to help. Sparking students’ positive ideas about themselves also seem to help. Basically, it is important to highlight factors other than the stereotype that are relevant in performance.
October 10, 2008
“A Time for Telling” is the title of one of my favorite papers of Dan Schwartz (Professor of Education at Stanford). In it, he argues that lecture isn’t all bad. We complain that lecture (or “direct instruction” in ed-speak) doesn’t result in a lot of learning for our students. This has been shown again and again, in a lot of studies. But it’s pretty hard to completely eradicate lecture from our universities (or high schools, etc.) — it’s a pretty efficient way of communicating information. But if students first struggle with the ideas and concepts, then they’re prepared to learn from it. This is called Preparation for Future Learning.
For example, you could imagine (and it’s been shown) that students who first invent the idea of density (by being given the task of coming up with a way to describe how many clowns there are per square foot at a circus) will be better able to answer a question about the density of water than, say, a student who was just given the formula for density and shown a worked problem using gold. And a recent study by Schwartz shows just that, that those students who first invented the solution were better able to transfer the idea to a new situation. He writes:
Direct instruction is important, because it delivers the explanations and efficient solutions invented by experts. To gain this benefit without undermining transfer, direct instruction can happen after students have engaged the deep structure, per the Invent condition. [The students who invented the solution on their own] performed just as well on a subsequent test of word problems about density and speed. Direct instruction becomes problematic when it shortcuts the appreciation of deep structure. Across conditions, students who encoded the deep structure of the clown problems were twice as likely to transfer. It is just that fewer students in the Tell-and-Practice condition encoded the deep structure, because they had received direct instruction too soon.
Similarly, he later cites a study that found:
For example, college students learn more from lectures and readings when they first work with relevant data compared to when they write a summary of a chapter that explains the same data .
In some instances, he says, it is useful to just receive direct instruction because the goal is to build rote, routine skills. But in math and science, this isn’t the case:
In math and science, instruction cannot exhaust all possible situations. Transfer and adaptation are important. Although automaticity is important for some facts such as “2 x 3 = 6,” real situations rarely come with formulas attached, so students need to learn to recognize the relevant deep structures. Moreover, the cumulative curricula of math and science mean that students should build a base of knowledge on deep structures from which future learning can grow and adapt.
But teaching this way brings up the problem of assessment:
In the current milieu of high-stakes testing, standardized assessments largely measure routine expertise; namely, efficient recapitulation. If educators want students to become adaptive, innovative citizens who keep learning through changing times, current assessments do not fit. A better fit would map students’ trajectory towards adaptive expertise. Ideally, assessments would examine students’ ability to transfer, particularly for new learning. Such assessments would include resources for learning during the test (for example, a simulation that students can freely manipulate).
September 26, 2008
Especially for K-12 teachers, check this baby out. The National Science Digital Library has Science Literacy Maps online. For a bunch of different topics (Math, Technology, Physics, Nature of Science) you can click to get a concept map of a set of topics. In physics, for example, you can click on waves to see a map of all the concepts related to waves. Even better – there is a link within each section to see a list of student misconceptions (with references) related to those topics.
September 25, 2008
I just read an interesting article (Shmader and Johns, Converging Evidence That Stereotype Threat Reduces Working Memory Capacity, Journal of Personality and Social Psychology, 2003, Vol. 85, No. 3, 440 – 452) about why worrying about stereotypes can make women and other minorities perform poorly on tests. They gave subjects a test of working memory (matching equations and words). They were told this was a test of quantitative capacity. For half the subjects, they also said that gender differences in math performance could be related to differences in quantitative capacity. That created a condition called stereotype threat — the fear that your behavior is going to confirm existing negative stereotypes of your group.
They found that – for both women and latinos – the stereotype threat condition caused a decrease in working memory capacity. In the non-threat condition, women and men, latinos and whites performed equally on the task. Men’s and whites’ scores were not affected by the different conditions.
They wondered whether it was just that women and latinos became more anxious due to the threat condition and that this emotional response affected their test performance. That doesn’t seem to be the case — women didn’t become more anxious (though latinos did). Even if they’re not aware of feeling threatened, the mere presence of the negative stereotype may consume critical cognitive resources. Other research has shown that subjects show physiological signs of stress when under stereotype threat. Perhaps that interferes with the ability to remember items.
Myself, I know that when I’m in a position of stereotype threat (which happens all the time… I mean c’mon, I’m a woman physicist, and one with a relatively shoddy physics background at that) I become very aware of it. Just the other day I found myself the only woman in a room of men discussing gender differences in physics classes. I stayed quiet. I didn’t want anything I said to become indicative of “what women think.” I also find myself discussing physics with groups of men quite often. I’m very quiet in those conversations too. Of course, worrying about “looking stupid” is tied up with many other factors (my personal ego, the culture of physicists, group dynamics), but it’s also tied up with my gender, even if only implicitly. I’m not one of those super-brilliant female physicists. I’ve got my PhD, I’m no dummy, but I don’t compete well with the fast-and-furious discussions of physics. In part, I often feel I have fewer cognitive resources at my fingertips, and maybe that’s what this study is highlighting. I could participate in the physics discussions at a slower rate, with a book and some time to think about it. I don’t pull things out of my head if I haven’t been thinking about them recently, the way that I see the men around me do.
September 22, 2008
There was an interesting post, and comment thread, over at Built on Facts — on How to Be a Good TA. I’ve been wanting to respond to it for two weeks and have been too busy. It is interesting that this discussion came up just as I was forwarded a great article about TA Training — Growing a Garden without Water: Graduate Teaching Assistants in Introductory Science Laboratories at a Doctoral/Research University (Luft et al, Journal of Research in Science Teaching, vol 41, pp 211-233, 2004). That article delves into the dearth of training giving to graduate TA’s, who bear a large brunt of the work of communicating science to undergraduates [I'll send you a copy if you ask]. They write:
In the past 3 decades there has been a rising concern about the instructional support afforded to Graduate TAs, and an acknowledgment by faculty that expertise in teaching does not occur instantly in higher education.
No kidding. Even faculty don’t often get this kind of training. And then they’re supposed to teach the next generation of worker bees. (One of the exemplary training programs is called Preparing Future Faculty). TA’s are called on to make all sorts of decisions about their courses (curriculum, what concepts to emphasize, how to evaluate students) and faculty aren’t guiding them very much. Faculty aren’t often well-informed about undergraduate education reforms, anyhow, which suggest that there are better ways to teach and assess students how we were traditionally taught.
The blog post from Built on Facts, in some ways, exemplifies these problems. I have no doubt that Matt is a great TA. He understands that it’s important to engage students in the process of learning. But many of his comments suggest that being a great TA is just about doing traditional instruction the best that you can. Here is what Matt said about his extensive experience as a graduate TA:
What (students) need in recitations is only so much theory as is needed for an understanding of the concept, with lots of worked example problems. Lots of them. Do them as interactively as possible, so that instead of just working through the problems yourself in front of sixty glazed-over eyes the students are actively involved in figuring things out. …
Put real thought into how you present your lectures. What seems beautiful and elegant to you might be obscure and overly complicated to a new student. Try to be clear with concepts and buttress each new idea with a concrete example problem. A real one, not a toy problem that’s orders of magnitude easier than what they’ll face on the homework.
He also suggests giving students extra practice in working problems by giving quizzes and review sessions.
I think a lot of these methods would work if there was good evidence that lectures work. But so far, the evidence suggests that students don’t learn by telling, they learn by doing. As long as you’re up there in front of the blackboard, you’re stuck in a classroom structure where information is supposed to travel from teacher to student. I don’t think that’s the best approach, based on the evidence. Get the students talking to each other, working through problems, discussing and arguing. Then act as their “guide on the side” (not the “sage on the stage”) to help them learn. You can’t teach anybody anything.
Now, I’m really not slagging on anything that Matt’s saying (or any of the other good suggestions in the comments of his post), just that the initial structure of the teaching environment he’s using is flawed. For instance, I can’t argue that it’s good to give clear explanations, to think about your lectures in advance, and to give example problems. I love his suggestion of giving quizzes — research shows that the act of trying to recall information increases your memory of it (even if you don’t get the answers), so taking as many tests as you can is a really good thing. But the “good lecture” techniques only go so far. Students plead for us to give them example problems often because they want to see something that “looks like” the homework so that they can follow it as a recipe.
The comments to Matt’s post suggest that at least the better students don’t want those boring example problems, though — like Matt says, they want “real” problems — interesting, tough problems that get them engaged in solving it. I’ve seen that desire in our physics majors here as well. What would be great is if we could really model to students how we go about solving such a problem — taking wrong turns, thinking back to worked examples, looking at limiting behavior, etc. But that takes a very long time, and is hard to do justice in front of a class.
One idea that I’ve found really compelling is called Preparation for Future Learning. The idea is that sometimes there is a time for telling (for the “theory” part of the presentation, tying things together, giving out facts), but it is after a student has already struggled with the ideas. One way to do this is to give them a canonical problem and ask them to come up with the solution. For example, ask biology students to come up with a strategy for eagle conservation. That’s a huge, open-ended problem (they don’t have to be that unstructured) but after students come up with a bunch of (poor) strategies, they are better equipped to hear and understand a lecture about conservation techniques.
TA’s aren’t well-trained and teaching is undervalued
But when would a TA learn these kinds of techniques to teach? The article I mentioned at the top of this post (about TA training) argues that it’s not enough to know the content (in this case physics) — you also have to have Curricular Knowledge (instructional methodologies) and Pedagogical Knowledge (how to take the content of your particular discipline to the learner). And graduate TA’s are taught neither of these — they’ve been prepared for research careers, for the most part. Teaching is often seen as a lower-tier calling than research. Thus, TA’s aren’t rewarded for working on their teaching, and their faculty mentors aren’t well prepared to help them in these endeavors. TA’s feel that teaching is important, but an interest in teaching doesn’t really help their professional development as scientists.
Here is a faculty’s comments on the lack of importance of teaching for a TA, from that paper:
Sydney did think that teaching was important, but there is a reason that it is not emphasized. He goes on to add that at the graduate student level it is perceived as being more prestigious to hold an RA appointment instead of a TA appointment. At the faculty level, research productivity is important in the yearly reviews, not teaching. Faculty may talk about the importance of teaching, but during the department reviews the focus is on research and funding. At the national level, grants are funded for research and not teaching. When grants are funded, they pay more for RAs, not TAs. Sydney pauses again and states that it is clearly a cultural thing.
TA training is poor
In keeping with these cultural expectations, TA training meetings aren’t sufficient to teach such a complicated and difficult task as teaching. Here is one TA’s comments on the usefulness (or lack thereof) of weekly training meetings, from that paper:
The staff meetings address what the lab is about. They are necessary, but are not done well.
Some TAs just like to talk and so we listen to them and they take up so much time. I just
don’t get a good view of what the lab is about from the staff meetings. I’ll ask a
question . . . and the laboratory coordinators can’t answer the question and I get frustrated.
I know that they try really hard, but it’s not exactly what I would want. I guess I need
more clarity than the other TAs and the 2-hour staff meeting is just not an efficient use of
my time. I end up going to Monday lab before I teach my sections to really get a sense of
the lab.
TA’s are left on their own
One other quote reminds me a lot of what Matt said about his teaching, since it sounds like he was pretty much on his own as he decided what to cover in recitations:
The lack of faculty involvement was also evident when GTAs discussed their preparation for teaching each laboratory. No GTA indicated seeking out the assistance of faculty members or even the laboratory coordinator when planning for their classes. Instead, as Samie stated, she often “. . . read through the laboratory manual, making sure that I understand the order of things
and what it’s asking. I interpret the lab, reword things, make the objectives clear, and think
of ways to introduce it to students and think over what I want to lecture on in the
laboratory. “
These sorts of decisions are fairly complicated for a beginning teacher to make!
The article concludes quite strongly:
In this study, GTAs and laboratory coordinators who were involved in preparing GTAs had limited opportunities to enhance their instructional abilities. The constraints of the working environment often led GTAs to make intuitive decisions, or decisions based on their own experience as students; thus their practices were often disconnected from the literature base in education.
The title of this article, ‘‘Growing a Garden without Water,’’ represents the expectations and potential of GTAs in the absence of adequate support to facilitate their growth. GTAs have an essential role in universities and colleges, but without proper instructional support they may not achieve their potential. Furthermore, it is estimated that by the year 2014, 500,000 new professors will be teaching American college students (Jones, 1993). Many of these professors will have served as GTAs. Improving the education of future students depends on the thoughtful, careful, and purposeful training of future faculty members. To meet the needs of the community, the garden must be properly tended by involved caretakers, and it will yield its fruits.
They say that rewards and incentives should be given for good teaching, and TA training programs should draw on the research base in education that informs us how we best learn to teach science.
 | sciencegeekgirl on The Salty Science of the Saltw… |
 | mannixjr on The Salty Science of the Saltw… |
| Stephanie Chasteen on Things to do in a microwave #1… |
 | Placedb.org on Things to do in a microwave #1… |
 | Tom Tahmassian on Myth 3: Does water swirl count… |
