Evaluation

You are currently browsing the archive for the Evaluation category.

This morning I cut down the survey we ask teachers to fill out after taking part in I’m a Scientist. The old survey was a hefty 41 questions long, and the new one has just 18 questions.

By looking at teachers’ previous responses to the 41 questions we were able to identify the questions which worked, and the questions which didn’t. Using this information we’ve made the survey simpler.

It’s also more quantitative. Funding from the Wellcome Trust for the next 3 years allows us to improve the way we evaluate I’m a Scientist, and move to evaluating outcomes through more quantitative measurements.

My next task is to do the same for the scientist and student surveys, and cut them down from 28 and 25 questions.

How can we evaluate the impact on students taking part in I’m a Scientist? Can we measure if they’re more likely to take a STEM subject at A Level? If they’re more likely to study science at University? How should we use the large amounts of data generated by online projects? How can we share our evaluation in a more useful way?

These are just some of the questions we’re trying to answer about evaluating I’m a Scientist and other Gallomanor run projects. Judging from the first in a series of seminars looking at Evaluating Impacts of Public Engagement and Non-Formal Learning, last Friday 4th November, others are thinking along the same lines.

The Core Issues & Debates seminar kicked off the series at the Dana Centre in London, and bought together a range of researchers, evaluators and learning and communication practitioners. Future seminars focus on areas such as how to reach new audiences, evaluating online engagement and using qualitative evaluation methods.

The 7 speakers approached evaluating impacts from different views – funding, strategy, science festivals, academic, and museums/science centres. There were some key themes that emerged during each of the 20 minute talks and the resultant Q+A sessions. (It would have been useful to have a bit more time for Q+A discussion after each speaker, as the allocated 10 minutes were quickly eaten into.)

  1. Evaluation needs to be shared with others so all projects are ‘learning projects’. The British Science Association’s Collective Memory is a good place to start. It’s worth constantly thinking about how to improve evaluation during a project, such as changing evaluation questions so they return more useful responses.
  2. Evaluation is very important right from the grant application stage at the start of a project, but shouldn’t be done for the sake of it, or just because funders ask for it.
  3. There are still lots of questions unanswered about how to evaluate and measure the impacts of an engagement project. Is it really possible to measure if students are more engaged with or interested about science as a direct result of one activity? Is it enough to accept your activity is one of many factors that may have influenced a change seen? These will hopefully be explored further, and maybe even answered, in future seminars in the series.
  4. Negativity can be hard to capture in evaluation. Evaluation studies can therefore be designed to try and capture negativity, such as framing questions to encourage participants to think not just about the positives of the event.
  5. Bad evaluation that draws inaccurate or invalid conclusions from data can be more damaging than no evaluation.

Overall it was a useful introduction and summary of how impacts are being evaluated. Armed with my 7 pages of dense notes scribbled during the seminar we’re now working out how to put some of these ideas into practice with I’m a Scientist. This will likely spark another post in due course.

Click on the RSS symbol at the top left of the page to subscribe to the blog, or register for email updates.

The I’m a Scientist Debate Kits project is now over (for the moment). Here’s a summary of the evaluation findings, or you can download the full report at the bottom.

Summary

I’m a Scientist Debate Kits is a project to help get more debate and discussion in the UK’s science classrooms. It was a Gallomanor project funded by the Wellcome Trust

  • Over the last 18 months we have produced and distributed four kits, on different biomedical topics.
  • Altogether roughly 4,500 printed copies, on three biomedical topics, have been printed and distributed, plus a further 4,000 copies downloaded.
  • They have proved extremely popular with teachers, apparently meeting a teaching need for high-quality, simple-to-use resources to stimulate debate and discussion in the science classroom.
  • Teachers judged them to be very effective.
  • The kits got students seeing different sides to an argument, expressing their opinions and backing them up with facts, and developing their confidence and discussing science issues.
  • They also increased teachers’ confidence and skills at running debates and saved teachers time in lesson preparation.
  • Teachers told us they also supplied far more in-depth coverage of each topic than they would have had time to provide.

Key findings

Total kits distributed (print and electronic): 8,521.
98% of teachers would recommend the kits to a colleague.

The kits

  • Were effective at prompting in-depth discussion.
  • Engaged young people.
  • Were easy to use and saved teachers’ time.

Download the full evaluation report (pdf)

Download kits, or make your own

For more info please email
debatekits@imascientist.org.uk

A student looks at a log in card for I'm a ScientistIn our pilot event, we did nearly all of the evaluation in-house, to save money. We tried to be as objective as possible, but of course it’s much better practice to have an independent evaluator. This year, we’ve been able to do that, and we appointed the lovely Kate Pontin to evaluate the project for us.

We are publishing the evaluation report on 2010 today (and also presenting it at the Science Council). You can also read the team’s early thoughts on this year in our write up of our Science Online London session, or see our evaluation report on the pilot.

Download full I’m a Scientist Evaluation Report 2010 (pdf)

Summary of I’m a Scientist 2010 evaluation

This report summarises the findings of the first year of formal evaluation of I’m a Scientist, Get me out of Here! and thus acts as an interim report.

I’m a Scientist creates opportunities for scientists to answer questions from students via the internet, opening dialogue, while developing skills and understanding of the science process. It also gives students an idea of what it is like to be a scientist and the types of careers available.

Evaluation data was collected from scientists, teachers and pupils via surveys, and interviews. Observations of a number of sessions in school were also undertaken.

Data suggested that the IAS programme is very successful, providing a unique approach to the communication of science to students. It shows that:

  • Students gained a lot from the experience. Their feedback shows they thoroughly enjoyed the event, especially the live chat sessions. They developed key skills including focused questioning and gained confidence in scientific discussion. They found it interesting to discover more about what it is like to be a scientist and were surprised that scientists were not as stereotypical as they originally thought.
  • Teachers found that IAS supported their need to develop different approaches in the classroom and also to encourage pupils to think about science and who it is actually undertaken by. They found the resources very useful, in particular the debate kit and were also pleased with the support they received from the team during the running of the event.
  • Scientists also gained or developed skills in communicating with the public, explaining their work (often complex science) clearly to pupils aged 13 onwards. This in some also helped inspire and revive their enthusiasm for their own studies.

Minor improvements suggested are:

  • Helping teachers to prepare pupils especially in their development of clearly focused questions
  • Promote and signpost other debate lesson packs like the IVF debate, and to help conclude the event, with perhaps an assessment element.
  • Further initial guidance for scientists giving guidance on the time it might take, but also for those less familiar with students of this age some general information on KS3/4 and the diverse knowledge base.

Evaluation will in 2011 focus on finding out more about how to attract scientists and about the impact their involvement has on their organisations. Longitudinal studies will look at the impact of students in the longer term (for example in take up of science or exam results). It is also hoped to develop baseline and plenary activities to find out more about change in knowledge and attitude to science.

I’m a Scientist enabled students using an innovative approach to inspire and enthuse. It worked with pupils from high achieving classes but also those across the ability range and from a wide range of backgrounds including groups from BME families.

Key figures for 2010

  • Two events – March and June
  • 7,459 questions asked
  • 125 scientists were involved, in 25 zones
  • 94,909 visits to the site
  • 6397 students took part
  • 648,563 page views

About the evaluator

Kate Pontin is a former geologist who has been working in museums and evaluation for over 20 years. She specialises in the experiences of young people in informal learning environments.

People have made some brilliant suggestions of topics for the last debate kit. Thanks!

Please vote for your preferred topic below:-

Topic for the last debate kit

View Results

Loading ... Loading …

BTW, There are some suggestions I’ve not included, for one of three reasons:-

1. It’s not a biomedical topic (e.g. renewable energy) – the Wellcome Trust fund us to promote debate and discussion about biomedicine. But we’d be happy to produce kits on other topics if someone else would fund it!

2. It’s a topic we’ve already done. Previous kits are on IVF funding, Stem Cells and ‘Are we too clean?’. You can download earlier kits here.

3. I couldn’t see a way to make a debate kit on the topic. e.g. Nature vs Nurture just seems far too big a question. It’s a fascinating one though and I’d love to produce resources of some kind on this. In fact I’ll probably be distracted for the rest of the day thinking about that now:-)

We’d never been to the Science Online conference. And we’d never given a presentation about I’m a Scientist, Get me out of Here! But when the organisers asked if we’d like to do a session on the event we jumped at the chance.

We then closed the office for the whole of August, so we had to write and organise our highly interactive, possibly risky session in 2 days when we got back last Wednesday. At the same time as do all our catching up from the holidays.

Last week was an interesting week…

So here is a summary of our session (last Fri, 3rd Sept) and what we, and others, said.

Our presentation

Photo of Sophia speaking, in front of prezi presentation and a twitterfall about the session

Sophia presenting at Science Online coference London

At Gallomanor we like to walk the walk, as well as talk the talk. So we wanted to make the session as interactive and audience-led as possible. After a short (2 min) intro, we showed the audience the 12 topics we’d prepared and asked which 6 they wanted to here about.

Here’s the prezi presentation, with a slide on each topic. I’ll summarize below what we were going to say on each – including the ones we didn’t cover on the day.

IAS Solo on Prezi

1. Film – we showed our ‘Intro for the classroom’ film.

2. Our Philosophy

The audience didn’t vote to hear about this, which was a shame as I think it’s really important in explaining why we do what we do, and how. Fools, fools!;-) Maybe we should have called it something more exciting sounding…

Anyway, key points would have been:-

  • We reverse normal power structures (kids in our event are given some decision-making power and the chance to ask what THEY want to ask). This engages pupils who are normally turned off. See more on my thinking here in this piece on the Secrets of Engaging Teens.
  • Making it fun and game-like is not a gimmick and doesn’t make something not educational. It makes people pay attention and be interested! (e.g. this research)
  • It’s not just about getting kids to study science. Not all teenagers will grow up to be scientists, but they will all grow up to be people, and need to have a relationship with science. More on this point in this article for Wellcome Trust blog.
  • Scientists have a lot to gain from engagement too – they aren’t just doing everyone a favour. They can be energised, challenged and made to think by the huge variety, and inventiveness, of students’ questions.

“a 4.00am Eureka! moment solved a problem that’s been bugging me for the best part of a year. It came from a seed of an idea planted in my head by a simple question from a 13/14 year old, absolutely bloody marvellous! Next day I was skipping into work like a refugee Munchkin from the Wizard of Oz, bleary eyed but elated.”

3. History of event

Shane thought of the idea for our sister event I’m a Councillor, Get me out of Here! while drinking Guinness. It’s been running since 2002, helping councils and local councillors to engage with young people in their area.

We came up with the idea of doing a science version in 2007 and ran the first, pilot event in 2008.

4. Funding

The pilot was funded with a People Award from the Wellcome Trust. We then successfully applied for a follow up grant from Wellcome’s Society Award scheme.

We love Wellcome and they are fab. We approached lots of people – people like BIS and Dept of Education who you’d think would be the right people to approach, and we got nowhere. All roads led back to Wellcome.

If you’d like to apply for science engagement money from Wellcome, our advice would be:

  • Make sure you understand what they value and what they are looking for, and that your project fits in with that.
  • Contact them early and talk to them.
  • Make sure you have really thought it through and explained your plans (not just vaguely). They will fund imaginative, even risky stuff, but they want to know you aren’t going to piss the money up the wall.
  • Realise the importance of formative (and summative) evaluation.

5. Site detail

Shane was just going to run through what’s on the site and how it works. You can prob just have a look for yourself.

6. Site build

It was in WordPress, cos we like and support open source, etc. And also cos WordPress is great. Apparently the site is doing things with WordPress that no one has done before. This is of course very exciting.

Developed by total legend Mike Little. Here’s Mike’s presentation about the project at Wordcamp 2010.

7. Scientist Recruitment

We pursued as many routes as possible to recruit scientists, including contacting learned societies, universities, research institutes and companies. Personally, I also bore people to death at parties and force my card on them if they are scientists or know any.

Because we’d done formative evaluation (i.e. talking to possible end users) as part of the development, we were able to focus communication and explain what the benefits would be to the scientists.

Once we’d run the pilot, word of mouth was one of our most powerful tools, as the scientists who took part really loved it. We also had evaluation evidence to back up our claims of the benefits.

To select the scientists (as we are now oversubscribed) we involved our end users. For the last event we took each scientist’s one-sentence description of their work and put it up blind (i.e. just their words, no information on age, gender, ethnicity, organisational affiliation) on a website where students and teachers rated each description. We, and a representative of the Wellcome Trust, also rated each scientist and we combined the scores.

We still want to find new ways of recruiting scientists, and particularly reaching the scientists we aren’t reaching at the moment. If you can help, let us know! And if you want to get involved, sign up here.

8. Teacher recruitment

Timeline for debate kit sign ups, kit 1

Again, used many routes. As you can see from this graph of sign ups for debate kits, some of the most powerful methods are direct mail, the Planet Science e-newsletter, and posting on the TES message board.

Also, for the big event in June, we got a significant number of sign ups from the STEMNET newsletter and a few from many other sources. We think it’s worth casting the net widely.

Again, word of mouth incredibly powerful. Especially because we recruited a ‘teacher panel’ of teachers to help us develop the project, before the pilot, and they became very invested in the project and really helped us a lot. This emphasis on consultation with teachers also meant we really were providing something they wanted, in a way that was useful to them.

Cross-marketing from our debate kits project was also incredibly useful. It enabled teachers to get a sense of where we are coming from, and the quality of what we provide, before making the commitment of signing up to a two week event.

We are oversubscribed with teachers too, and here part of our selection procedure is to deliberately pick as varied schools as possible (geographically, type of school). This has included Special Schools, Pupil Referral Units and Bristol Hospital Education Service. And also schools in Shetland, Northern Ireland and Oman.

We also try to get a representative sample results-wise. By which I mean, high-performing grammar schools are overly represented in the schools who apply, we try to even things out by picking more of the less academically achieving schools. We think it makes a bigger difference to those kids.

As with scientist recruitment, this is still a work in progress. If you can help us get the word out to teachers (taking part is FREE to schools), please do! Or if you are a teacher who would like to get involved, sign up here!

9. Evaluation strategy

Formative evaluation has been absolutely key. We started talking to scientists and science teachers before we did anything else. Asking them about what they wanted, what would work for them, what motivates them.

For teachers, we recruited a teacher panel of people interested in the project, who could give us instant feedback via email on everything from lesson plan ideas to terminology on the site. This also meant that they were invested in the project, when the pilot came around, and understood it and what they could do with it.

We also included young people, for example testing possible designs on them. And we made several school visits to observe science lessons.

We had a limited budget for the pilot, so much evaluation had to be done in house. But we set aside money to appoint an external evaluation consultant, Yvonne Harris, to spend a few days on the project. She advised us throughout, and also conducted some independent interviews with participants at the end, and audited our report. This was absolutely invaluable as she could check things like questionnaires and methodology as we went along, and suggest solutions we would never have thought of, and bring an independent perspective to the whole thing.

It also helped that we had grown the project organically, as we had developed over the years questionnaires that worked, and found (and corrected) many useability issues as we went along.

You can read our evaluation report on the pilot here.

We now have a much bigger budget for the roll out of the project, and we have devoted far more of that to external evaluation. Kate Pontin is now our external evaluator, and she has been invaluable in helping us think with clarity about what we need to find out and how we can do it. She has also been able to do far more schools observation than we can, as we are busy running the event while it’s on! This has been extremely useful.

Kate’s interim report will be unveiled at a special event at the Wellcome Trust on 20th October. This will be part of a special ‘Beyond Blogging’ event, curated by us. Tickets will be very limited, but do get in touch if you’re interested in attending.

10 Evaluation findings

Shane outlined some topline figures so far from the 2010 events.

TopLine Web stats for IAS June 2010

  • 4,667 students
  • 100 scientists
  • 171 teachers
  • 6,580 questions
  • 3,085 comments
  • 4,744 votes

More in-depth evaluation results will be published on 20th October, as above.

In the meantime, you can read our full evaluation report on the pilot here.

A summary of the pilot evaluation report here.

Or our short evaluation report on the 2009 event here.

11. Summary of the strengths and weaknesses of this format

Strengths

  • Power reversal truly engages and empowers.
  • Online gives access to scientists (for schools) and students (for scientists), without having to go anywhere.
  • Doing it online also creates intimacy (makes it easier to ask real questions and break down barriers), compared to a scientist in person giving a talk.
  • It also ‘levels the playing field’ – quieter and less confident students participate more.
  • You’re reaching all the students in the class – not just the very keen, as you find in science clubs, etc.
  • A conversation develops over time – over two weeks, students can read about the scientists, go away and find out more, ask questions, think about the answers, ask more questions…
  • For teachers, the preparation work is done for them and they can concentrate on facilitating learning.
  • For scientists, it saves time – no travelling to a school, they log in from their desk. Every moment spent participating is spent in engagement.
  • Scientists also find the range and energy of the questions rejuvenating, thought-provoking and inspiring.
  • And scientists get into the competitive aspect and have fun.

Weaknesses

  • Some scientists (and some teachers) don’t like the informality of the project, although we think it has real value in making connections and breaking down barriers.
  • We’ve disguised the learning and made it fun, so some teachers/scientists/students don’t see that it’s there and think it ‘won’t help them pass exams’. (God help us if that is the only thing some people think education is about).
  • Schools IT (sigh!). Often school firewalls are over-enthusiastic and we do have some problems with schools blocking the site.
  • It can be hard work for the scientists. Some had ~700 questions to answer!
  • It’s expensive to run (although not compared to many other projects).

12. The Question Game

Our question comedy improv game! OK, so not very comedy, and not actually improv, but kind of a game, inspired by Whose Line is it Anyway? Audience members shout out a word, any word, and we search the site to see if there are questions (or answers) containing that word. Intended to give an insight into the enormous range of questions.

Here’s some results

Search dinosaur

Search space

Search evidence

If you want to play the question game for yourself (be warned, browsing the site can be addictive!), just go to the main page and type in the search box near the top.

Live chat

After the presentation, we wanted to give the audience a feel for what live chats are like, and why they are so popular with scientists, students and teachers. So we had a live online chat, with the audience taking the place of students. We gave out log in details and everyone in the audience who had a laptop (quite a few, it being Science Online) could log in and take part. We also showed the chat on the projector.

Some of the key benefits of live chats:-

  • Access to people who couldn’t be there otherwise: We had a scientist in Michigan, one in Sydney, a teacher in Shetland, and a student who was in school. All of them had taken part in the event and answered questions from the audience about what it was like.
  • Immediate and friendly
  • Fun
  • Newer comments appear at the top, so you need to read upwards.
  • Discussion isn’t threaded – we’ve found that breaks up the chat too much and stops it being a communal experience.
  • Chats can be difficult to follow at first, but you get used to it quickly. Students are often quite familiar with chatrooms and don’t find it’s a problem.
  • There are two chatrooms side by side, one for students, one for the scientists. (In this chat that means one for audience and one for our participants). This makes it more difficult to understand an archived chat, however, we’ve found from experience that if the scientists and students are all in the same box, the scientists’ replies get lost in an avalanche of comments from students, so this works better.

Archive of tweets relating just to our session, hashtag #iassolo.

He doesn't know either: Gilbert demonstrating the magnet before Queen Elizabeth /Wellcome Images

Last year I saw A C Grayling talk on happiness and it’s importance. He said that he bans his students from using the word ‘happy’, that it’s a lazy portmanteau word. He thinks that if you are forced to choose a different word – hopeful, exhilarated, content – you’ll think about what you really mean far more clearly.

I think the same can sometimes be true of jargon. Scientists taking part in I’m a Scientist have told us before that explaining yourself without using jargon is hard work, but unexpectedly rewarding. It means you have to think through what you mean and it exposes your mental shortcuts.

We saw a great example of this in Imaging Zone. A fairly innocuous-seeming question (Why do magnets attract and repel?) pretty much lead to the scientists realising that they don’t actually know how magnets worked. Not really. Not when they tried to actually explain it to other people, without using jargon.

Now these scientists include a man who spends many of his days working with an fMRI scanner, containing a magnet so powerful that you have to remove any ferrous object from your person before entering the room. But still, magnetism turned out to be one of those things that he learned about years ago, and sort of assumed he understood.

I’m expecting that some of you have the same feeling I did when reading that Q+A – ‘OMG, I don’t REALLY understand how magnets work either. How did I not notice that before?’ I think the thing is, most of us rarely discuss how magnets work. And when we do, we use technical words (‘dipole’, ‘electromagnetic’, ‘electron shell’) which we and our listeners all know, which can obscure the fact that you don’t truly understand the underlying mechanism.

By all accounts this question, and the attempts to answer it, lead to an awful lot of magnetism-related discussions at scientific breakfast tables and coffee machines around the country. So, one outcome, of just this question, has been much thinking about and discussing the mysteries of physics, by scientists, with colleagues, and others, about a subject they all thought they understood, but actually it turns out they’ve got lots of questions about it.

If the essence of science is asking questions and taking nothing for granted, then I’d call this a result.

This post started life as part of a mammoth post I’ve been writing about how June’s I’m a Scientist event went. The post has taken about three days so far and we’re up to 3,000 words. So I thought I should really break it into bits and start bunging some up now.

Well, of course, I’m going to say YES to that question.

What’s This?

Here is the evaluation report on the first half of the debate kits project, comprising the first two kits (from Summer and Autumn 2009). Obviously I was supposed to publish this MONTHS ago, but it’s fallen by the wayside, until now.

Why now?

We have just brought out the third IAS debate kit, on Stem Cells. We posted out 1,500 stem cells debate kits earlier this week.  I got an email from a teacher this morning saying ‘Just got the Stem cell pack…fantastic!’ so I’m hopeful they’ll be as popular as previous kits:-)

What are the debate kits?

The debate kits are designed to get students in their science lessons debating and discussing science issues, seeing things from different points of view and learning to back up their opinions with evidence. Each kit is developed in consultation with teachers, and fact checked with scientists.

Evaluation summary

“This is great I couldn’t have done a debate without it”

I’m a Scientist debate kits is a project to help get more debate and discussion going in the UK’s science classrooms. We are developing and distributing debate kits to UK science teachers. The kits are designed to give a teacher everything they need to run a structured debate on a set topic and help their students develop their discussion skills. The activity gives them a structured way to start discussing and gets them practicing the behaviour. The Wellcome Trust have funded Gallomanor to produce and distribute three sets of kits in 2009/10.

The first kit was developed as part of I’m a Scientist, Get me out of Here!, in 2008. This first kit was re-printed and distributed to 1,500 teachers and science communicators in Summer term 2009 as the first nationally distributed IAS debate kit. We also provide an electronic version of each kit, which more people can download from the website. The second kit was produced in Autumn term 2009. The topic (suggested and picked by teachers) was ‘Are we too clean?’ The third kit will be produced and distributed in Spring term 2010. A fourth kit will be developed in Summer term and distributed electronically.

Outcomes

Key aims met:-

  • Kits well used
  • The resources worked ‘straight out of the box
  • They really engaged the students
  • They developed students’ discussion and critical thinking skills

Kits also:-

  • Were good way for students to learn new content
  • Were effective with a range of students
  • Got quieter students talking
  • Inspired teachers to start making their own!

Recommendations

  • Next kit in Summer term 2010, on stem cells (done!)
  • Produce an electronic large print version (done!)
  • Produce DIY debate kit templates for teachers to use (done!)
  • Increase print run to 2,000 (done!)

More details in the full Debate Kits Interim Report (pdf).

If you would like to order a copy of the latest debate kit (they are free!) go here

If you would like to download a copy of one of the kits go here

If you’d like to know more about the kits or the project there’s more info here, or feel free to get in touch and ask us!

We launched our I’m a Scientist debate kits on 18th June and have now distributed all 1,500 we printed. We’ve been really pleased by the response from teachers. The next kit will be coming out in the Autumn. We’ve written a report on this first phase of the project and we thought a summary of that may of interest to others, so here you go:-

Project: I’m a Scientist Debate Kits

A Wellcome Trust funded project to distribute structured debate kits to science teachers

Key milestones

18/06/09    Promoted kit by email to our contacts and on twitter
22/06/09    Mailers arrived in 5,000 UK secondary schools
23/06/09    Posted on psci-comm
26/06/09    Featured in Planet Science
01/07/09    1st kits arrived in schools
06/07/09    Posted on TES messageboards
27/08/09    Final kits posted out

Other marketing

  • Complimentary kits sent to the 323 specialist science colleges and 177 largest secondary schools plus various IAS contacts (e.g. at BSA, SLCs, ASE) with request to pass info on to teachers who may be interested.
  • Ongoing twitter and blog coverage.
  • Contact with Aimhigher who agreed to help publicise us to their target schools.
  • Attended Wiltshire STEM ambassador event.
  • Presentation to student teachers and to their mentors at UWE.
Orders over time

Orders over time

I think it’s clear that the events I’ve highlighted have had the biggest effect on teachers ordering kits. The dates of other interventions did not coincide with a spike. However, other activities, especially word of mouth between teachers, are invaluable and contribute to the long term take up of the kits, but will not give the same sort of spike. We are still seeing new visitors to the site every day with teachers signing up to hear about the next kit.

Distribution

Comps: 523
Online orders: 644
Offline orders: 283    Total: 1450 sent out
50 distributed in person

Additionally 167 people have downloaded the online version.

Schedule for kit 2

31/08/09    Ask teachers for topic suggestions
11/09/09    Email feedback reminder to teachers
14/09/09    Put up topic shortlist for voting
21/09/09    Decide topic for kit 2
………………….Put up info about it and start taking orders
………………….Make any topic-related connections necessary
28/09/09    Start research/writing kit 2
………………….Make decision about whether we produce a mailer for this kit
01/10/09    Meet w mailing house about arrangements for batch 2
05/10/09    Kit 2 to design
07/10/09    Kit 2 to printers
19/10/09    Post out 1st batch kits

Hi everyone, here’s a summary of how the March 2009 event went (pdf download). I was supposed to write a couple of pages so of course it’s 12 pages long (I’m the same with cooking – if I have a couple of friends round I make enough food for ten and we’re eating leftover curry ’til thursday…). Fortunately Shane (my boss) is busy being terribly important in London running his G20 bloggers tent, and hanging out with Bob Geldof (pictured), so he can’t tell me off about it.

shane-karina-and-st-bob

Shane (just behind Sir Bob) at the G20Voice

What we found was pretty similar to the pilot (but it’s important to check everything is still working:-)). One of my favourite quotes was the one I used as the title. Students were over and over amazed that ‘scientists are just like real people’, and that they were taking time to talk to young people. They also couldn’t quite get over that we were giving them a say about something, “[I liked] being able to vote as a child and make a difference.” I find that quite moving – we all want to make a difference, don’t we? Do we give young people so few opportunities for doing that?

As for the grown-ups, all the scientists and all the teachers (who filled in the feedback survey) would take part again, and recommend the event to a colleague. Everyone enjoyed taking part, the scientists developed their communication skills and got inspired about public engagement, “I got a tremendous amount out of it, and I think I probably learnt a lot more from the students than they learnt from me!”, and the teachers felt their students benefitted enormously.

The key elements mentioned, once again, were:-

•    Exciting and intimate medium for interaction
•    Real scientists and real science
•    Taking young people seriously and giving them actual decision-making power
•    Supported by thought-provoking classroom discussion activities

But looking through all this data has confirmed for me again how important each of the elements is and how they work together.

For example, one thing that works is that live chats are an intimate and familiar medium for young people – so they can ask questions when they might normally be shy in class. But it’s the fact that they have a real decision to make about real science, primed by appropriate classroom discussion exercises, that gives some purpose to that conversation, and means it’s not just students asking what nipples are for.

We can’t claim this is entirely due to our genius as event producers (although I’m sure it’s a factor;-)), it’s also to do with the organic way I’m a Councillor, and then I’m a Scientist have developed*. We’ve seen what works and what doesn’t as we’ve gone along and built the event up. We also owe the biggest debt of gratitude to the teachers, scientists and young people who’ve helped us. Thank you everyone! I was going to talk about that lots more, but I have to go and catch a train to see my baby sister’s band play so it will have to wait for another time. Have a great weekend everyone!

*I particularly like how this theory confirms all the prejudices of my biologist worldview: Evolution, much better at engineering than engineers are.

Tags: , ,

« Older entries