Meet some of our members
Below, in an occasional series, we profile some of our members.
Tēnā koutou katoa. My ancestors originated in England and Ireland. My father’s great grandfather sailed from Lyme Regis in Dorset (celebrated for discovery of the first dinosaur fossil) with the Canterbury Association settlers in 1850. A hundred years on, my mother emigrated from Northern Ireland, benefiting from the ‘£10 Pom’ assisted passage policy. The third of four children, I was born and raised on the Port Hills in Christchurch with a view of the Southern Alps. I now live in Wellington, my home for the last 20 years, and have a view (just!) of those same Alps, looking out across Raukawa Moana. I live in Ōwhiro Bay with my partner Richard and two children, Lewis aged 10 and Zora aged 7.
Views, nature and the wilderness have always been important in my life, perhaps driven by my need to see the bigger picture and connect to the place, land and space I inhabit. In a similar vein the appeal of evaluation for me stems from the pleasure of sense making, of putting the pieces of the puzzle together for that better, clearer view.
My path to evaluation came with joining the Work and Income ‘CORE’ research and evaluation team in early 1999. I had completed my masters in psychology a numbers of years earlier, with a very quantitative experimental design emphasis. The role at CORE offered a rich learning experience, exposing me to new qualitative methods and a blossoming evaluation community. Mine was long affair with the public sector. I spent almost 16 years working at the Ministry of Social Development and then the Families Commission. Over the years my work has involved evaluating a range of social sector policies, but with emphasis and specialisation in the areas of parenting practice, child development, integrated service provision and cross-sector partnerships.
Of late, I have become enthused by the ideas and practice of social innovation, design thinking and collective impact, in particular the tools and underlying principles that assist a collective way of working to create social change. As a precursor to this theme, I have a history of evaluating and providing advice on collaborative initiatives, including the Social Sector Trials, Investing in Services for Outcomes (ISO), the Community Response Model, High Trust Contracting and SKIP. My most recent work was an assessment of the Children’s Teams – an integrated service response to address protecting and supporting vulnerable children.
I recently left my role as principal analyst at the Families Commission to work independently. Putting into practice my belief in the benefits of collaborative alliances, I have teamed up with like-minded colleagues to work collaboratively as ‘Ripple’. I am currently involved with supporting the design thinking stage of a new parenting collaboration as well as keeping conversations boiling about supporting and measuring collaborative and collective impact efforts.
Collective impact is about working in unison to create large-scale social change. It requires putting the pieces of the puzzle together to create combined influence and reveal the full picture. Maybe it’s the wider view that I am drawn to, wanting to appreciate the whole vista in all its tangled glory.
Click here to view my recent presentation to the ANZEA Wellington Branch on collective impact.
I am of Ngāti Porou, Rongowhakāta, and Te Aitanga-Mahaki descent. Born and raised in Gisborne, I lived in Auckland for many years immediately after completing High School before shifting to the Wellington region where my wife and I reside.
I am an independent evaluator-researcher and have worked in evaluation and research for the last 15 years; 5 years as an employee at Phoenix Research Ltd (Auckland), and 10 years as an independent evaluation and research consultant. I have a degree in Social Policy from Massey University, Albany, completing two years of study towards a BA in Social Work before switching to Policy. I have also completed some post-graduate study in programme evaluation, at the University of Auckland.
While I’m passionate about health, and education, particularly for Māori, the scope of my work, mostly for the government sector, extends to social development, corrections, justice, disability, and employment.
As a long-time member of Aotearoa New Zealand Evaluation Association (ANZEA), I’ve been an ANZEA Board member and am currently a committee member of the ANZEA Wellington branch.
Current areas of evaluation practice
My current areas of evaluation practice mostly include Health, Education, and Corrections projects.
Over the last three years, I have project managed two three-year projects. One of these is a formative, process, and outcomes evaluation of a model of improved responsiveness to Māori with diabetes and cardiovascular disease. The implementation of the model was through demonstration services in primary care settings. Central to the model was the requirement for services to engage whānau in Kaupapa Māori ways, with the expectation that this would improve clinical uptake by whānau and result in improved lifestyle behaviours and wellbeing (Ministry of Health). The final report is due in December 2014. The other project was conducting action research/evaluation of Whānau Ora provider projects (Te Puni Kōkiri). Both of these projects end in 2014.
I am also in the first year evaluating two three year Te Ao Auahatanga Hauora Māori innovation projects (Ministry of Health). Both projects require formative, process, and outcomes evaluation activities.
One project is primarily about building resilience in the Māori learner and developing a technological tool to support resilience-building. Key stakeholders include one school (for now), a local Hauora, and iwi, and looks to involve whānau and community as it develops. The other project is primarily about improving tamariki/student health and wellbeing, and their engagement in school through the use of local Kaumatua (including Kuia) who deliver a programme based on the Te Pae Mahutonga model that is tailored to complement existing school curriculum. Key stakeholders include four schools, a Māori charitable trust, and a Kaumatua group. Both projects are proving to be very interesting, and relatively complex.
Tenā koutou katoa. Ko Julian King ahau. Ko Tamaki Makaurau taku kainga noho.
My ancestors are English and Irish, with a smattering of Spanish. I was born in Auckland. I grew up in West Auckland and Toowoomba, Queensland. I am married with two amazing, beautiful daughters and I live in Mt Albert. My spiritual home is Medlands Beach, Great Barrier Island.
My career pathway has had a few twists and turns. I started an architecture degree which morphed into a BSc. At the same time I trained as a commercial pilot and worked as a flight instructor for a while. In the early 1990s, while living in Wellington, I met some people who were policy analysts and my eyes lit up – here was a calling that sounded dynamic, challenging and a place where I could make a difference! I found myself a job in Government and returned to uni to complete a Master of Public Policy.
Two decades on and I’m still loving it, though I blame it for my coffee addiction. I’ve been lucky to have some diverse roles in the Department of Social Welfare and the Ministry of Health, Health Canada and KPMG (where I first discovered evaluation). I married Carolyn just before we moved to Ottawa. Our eldest daughter is Canadian. After a few years we were ready to move a little closer to home. Our second daughter was born in Melbourne.
In 2002 I set up Julian King & Associates and moved home to Auckland, initially representing Health Outcomes International in Aotearoa and subsequently leading their NZ consulting business. In 2010 I joined the Kinnect Group.
My evaluation interests (which include health economics, investment appraisal, social justice and Māori self-determination) are not as eclectic as they might seem. My current PhD research at the University of Melbourne represents a deliberate attempt to work at the intersection of these interests, to develop a trans-disciplinary, evaluation-specific approach to evaluating value for money that might enhance the traditional economic approaches in some contexts. Cost benefit analysis is not the gold standard!
Kāi Tahu, Kāti Mamoe, Waitaha, Rongomaiwahine, Ngāti Kōnohi
My Poua (great-uncle) and Taua (great-aunt) had a rock-solid mokopuna-focus, which we grew to be grateful for much later in our life once we realised the taonga that it was.
Home was Pahia on the southwest coast of Murihiku (Southland), between Riverton and Fiordland – ancestral homeland with small village and small primary school. Come secondary school age I was packed off to a convent boarding school in Invercargill to become a “young lady” and my brothers to a similar setting in Gore to become young men.
Whānau eventually had to uproot from our rural setting and shift to an urban one for employment opportunities. So it was to Gore we went and there I married and had two daughters and six sons.
At some point, it became obvious to me, that something was not sitting right about being Maori in our own country, although the old people had gone to great lengths to ameliorate our experiences so we could get by.
I grew to understand that it was Te Tiriti o Waitangi that hadn’t been sitting right and so worked with both mainstream and Maori audiences to raise awareness and knowledge of the impact on whānau, hapū and iwi, the efforts being made to tip the balance and what those audiences could do to contribute.
My whānau-raising life has seen me involved in care and protection, Te Kohanga Reo, Maori provider development, community boards and a mental health team.
Meanwhile our tribal efforts to have Te Kereme (The Ngai Tahu Claim) finally ended a seven-generation journey, culminating in a settlement with the Crown.
We shifted to Christchurch in 2000 to join in the tribal rebuild – initially working in the tribal corporation then as a self-employed contractor where evaluation work became a focus.
A self-employed indigenous development consultant I get contracted to organisations who wish to make change at some level. I continue to do my share of sitting on boards to represent. Reviewing Ngai Tahu 2025 – the tribal strategic plan – and presenting it at an ANZEA seminar were two evaluation highlights.
Currently through Te Awatea I am co-evaluating practice and outcomes of Family Group Conferences for the CSW Office – the experience of whānau the priority for my contribution to this work.
Christchurch has become our urban home, earthquakes and all, and we are proud to announce the pending arrival of a 13th mokopuna. Our greatest whānau effort has been in pulling together a whānau ora plan, which is designed to intentionally create a different future for these mokopuna – it will always be a work in progress.
Tēna koutou katoa. Ko Pākehā te iwi. Nō Kōtirana, Ingarani, Aerana me Tenemāka ōku tīpuna. Nō Turanganui-a-Kiwa tōku whānau mai rānō i te tau kotahi mano, waru rau ono tekau. I tipu ake ahau i Turanganiui-a-Kiwa me Rotorua. I ēnei rā, e noho ana e mahi ana hoki ahau I Pōneke. Ko Rae Torrie tōku ingoa.
Hi all. I’m a Pākeha New Zealander. My ancestors originated in Scotland, England, Ireland and Denmark. My family has been living in the Gisborne area since the 1860s. I grew up in Gisborne and Rotorua, and now live and work in Wellington. My name is Rae Torrie.
I grew up on the gorgeous East Coast with memories full of the beach and hot days and sun. That was also where I first noticed differences in how people were treated – Māori and Pākehā, women and men. I became a social worker in the 1980s and then moved into Equal Employment Opportunities (EEO). This is where I began learning the evaluation trade, as Government sought to report on its investment in EEO. I became self-employed in 1998, increasingly taking on evaluation work, and completed a Postgraduate Diploma in Social Sector Evaluation Research in 2009.
Robyn Bailey and I joined forces in 2010 and set up our small company, Evaluation Works Ltd. We work predominantly in the social, youth and health sectors. Examples of our work include evaluations of initiatives developed to recruit and retain more rural midwives, to provide a clinical secondary triage response to 111 callers, to reduce the risks associated with swimming at our beaches. This variety is one of the enjoyable challenges about evaluation.
In 2011 we received funding from the Health Research Council to develop a youth outcomes framework and to undertake a small impact evaluation for, and in partnership with, Kapiti Youth Support (KYS). There has been interest in this work from both the youth and evaluation sectors, and we are releasing a summary report about it in February 2014. We continue to work closely with KYS and have received additional funding to revise the model and measures after which we are aiming to trial it with other Youth One Stop Shops across the country.
An ongoing area of interest and enquiry for me is what is means to be Pākehā and working in evaluation. Since the 2012 ANZEA conference a small group of Pākehā evaluators have been meeting together and exploring this. At the 2013 conference we presented a paper on our thinking to date (presentation notes) . We will be involved in planning for the Pākehā forum at the 2014 conference where it is being offered for the first time, alongside the Māori hui and Pasifika fono. We have also drafted a chapter for a second edition of the book The Role of Culture and Cultural Context, edited by Stafford Hood, Rodney Hopson and Henry Frierson.
As a predominantly qualitative practitioner I am intrigued by the role of the evaluator, and the impact of identity (whether conscious or unconscious) on our evaluation practice. I am engaged by Patton’s idea that the evaluator is the instrument of qualitative methods, and consistent with a feminist and critical theory perspective, believe that ongoing reflection on how ‘we’ impact on our evaluation practice is an important element in ‘honing the evaluation instrument’.
I’m from Hamilton originally, the youngest of six children, offspring of farmers and shopkeepers in the Ohaupo area. I am now slightly taken aback to realise that Auckland has been my home, with some breaks, since the mid-1980s.
My journey into evaluation began with a politics degree at Auckland University, back in the days when mullets were semi-acceptable. Joining what felt at the time like an inevitable drift to Wellington, I fell into health policy, but soon developed a strong interest in the role of policy and planning in the health of people and communities. This ultimately led me to do a PhD at SHORE-Whariki, where I became increasingly exposed to evaluative thinking and took part in a range of evaluation assignments, alongside the more research-oriented doctoral study.
Over the past 20 years, my work has travelled between research, policy, academia and now the private sector. The most satisfying parts have been when I have worked alongside others to make sense of multiple data sources, and to build knowledge and insight in the process. This became central to my work when in 2006 I joined Synergia, an Auckland-based consultancy, where I led the development of their evaluation portfolio.
For me, evaluation is an underlying way of thinking that I’ve been able to bring to life in my work. I really enjoy the opportunity to apply evaluation in supporting projects and initiatives across the health and social sectors.
Over recent years, I’ve been working to build evaluation capacity with community organisations. We work with them to help make sense of the issues they are dealing with, what they are trying to achieve, and to develop straightforward approaches to data collection that describe in some ways the impact they are having.
Another area of work with Synergia is evaluating different types of workforce innovations, which at the moment includes physician assistants and care coordination roles in primary health care.
I’m keen to be part of building the evaluation sector. So it was really pleasing to be able to support a former summer student we had working with us, to publish some of the learnings from a project, and gain a scholarship to the Australasian Evaluation Society conference in 2013. I am also currently involved in the ANZEA-SuPERU collaboration to develop evaluation standards, as one of a group of evaluation practitioners in the project.
Evaluation is often a challenging space. We need to give effect to the interests of evaluation funders, respect and capture the perspectives of providers and participants, and through this understand the nature of what a programme or innovation has achieved. But it’s hugely rewarding for those very same reasons.
When I’m not doing all this, I’m often on my bike or tending what is generally a fairly shambolic garden, and doing my best to be a dad who is seen around the house as much as possible.
Links to publicly available reports
I’m part of the Nunns’ whānau which includes Ray, four step-children, our daughter and nine mokopuna. My tūrangawaewae is Kairakau in southern Hawkes Bay (see photo) where we’ve had many fabulous family holidays.
Evaluation whakapapa … While working as a policy analyst in the late 1990’s, I was asked to evaluate a policy initiative despite having no evaluation experience. Fortunately Dr John Owen from University of Melbourne and Dr Jenny Neale from Victoria University ran an “introduction to evaluation” course which I attended along with other people who are now ANZEA members. I joined the research and evaluation team at Housing New Zealand in 2003 where I worked until 2007 when I became self-employed. I was in the first student intake for the Post Graduate Diploma of Social Sector Evaluation Research at Massey University which I completed in 2008. I started a PhD with Associate Professors Robin Peace and Karen Witten from Massey University as my supervisors in mid-2010 (more about that below).
What I’m working on … I do policy work, and policy-related research and evaluation projects for government agencies. One of my current projects is a three year evaluation for the Ministry of Health as part of a team which includes ANZEA members Mathea Roorda, Debbie Goodwin, Louise Were and Julian King. We’re evaluating a new approach to supporting disabled people to realise their goals, based on the principles of inclusion, personal development and self-determination. The approach also aims to encourage communities to become more inclusive and supportive of disabled people.
I’m thinking, writing and talking about … evaluative reasoning which (unsurprisingly) is my PhD topic. My “elevator” definition of evaluative reasoning is: ‘what’s involved in getting to an evaluative conclusion that is justified and defensible’. I became convinced of the importance of evaluative reasoning via my first PhD topic which was evaluation quality. Following months of reading about different perspectives on evaluation quality, I realised that sound evaluative reasoning is the foundation of evaluation quality … the reason being that unless an evaluation is based on sound evaluative reasoning, any other attempts to enhance quality will be futile. I presented some interim findings at the 2013 ANZEA Conference. More findings are on the way!
My parents were “ten pound poms” on the last assisted passenger scheme from England to Australia in 1969. They lived in Australia for ten years and headed home soon after my brother and I were born. I was brought up in Kent in England where my father was a coal miner. After a year of strikes my mum studied to become a social worker and my dad became the primary care giver. They were super proud when I became the first person to attend University from our family. I am hoping my daughter might be the second; she is currently 5 – no pressure!
I completed a BSc (hons) in Psychology in the UK and then got a job evaluating prison alcohol and drug treatment services with the University of Kent. My Head of Department introduced me to evaluation and I absolutely loved it. So instead of psychology I completed a Masters in Public Health. This developed my qualitative and mixed methods skills, but nothing developed my skills more than working in the field alongside other evaluators.
After my Masters I met my husband-to-be and came to Aotearoa in 2004. I got a job as a Research Fellow at the University of Auckland with Associate Professor Janet Clinton. I got married and then decided it would be a good idea to have a baby and do a PhD in evaluation at the same time. On reflection, it was only a good idea because I have a wonderful husband. He has certainly earned a lot of brownie points.
I am now a Senior Consultant at Synergia and lecture at the University of Auckland. We are working on a number of exciting health and social care evaluations. Our evaluations of community development approaches to reduce the social exclusion of disabled people and bullying are particularly exciting. We are learning lots about evaluating these approaches within the demands for tangible outcomes. I am also currently collaborating with the University of Auckland to evaluate the Shorter Stays in Emergency Department targets and local After-Hours services. I love the complexity that health systems bring to evaluation and the challenge of understanding the delivery and achievements of health services, policy and innovations.
To support the development of evaluation practice and my own professional development, I am also currently supporting the ANZEA conference working group. This is a new role for me and I am enjoying the connection to other evaluators that this brings. I am also supporting ANZEA and SuPERU in developing evaluation standards, with a view to supporting stakeholders and evaluators to maintain and develop quality evaluation practice.
I am very passionate about the influence of evaluation. Feedback from my last ANZEA and AES presentations identified the challenge of enhancing evaluation’s collective influence. I am often thinking about this. Where do we sit at the policy table? How can we increase the use of evidence in decision-making, not just at a policy level but throughout the sector and its programmes? What is our role as evaluators and what is beyond our reach? These are questions I am continuing to grapple with.
Tenā koutou. Ko Michael Blewden tōku ingoa. Kei Tamaki Makaurau ahau e noho ana. Nō Kirikiriroa ahau. Ko English Ko Norwegian oku iwi. Ko Blewden Ko Brock oku hapū. Tenā koutou, tenā koutou, tenā tatou, katoa
My pathway to evaluation practice began in the early 1990s while undertaking the Community Psychology post-graduate training programme at the University of Waikato. I was excited by the prospect of an applied form of social inquiry that ticked many boxes of interest for me, not least the praxis between social science theory and practice. Utilization-focused evaluation was all the rage. I was hooked in by Michael Patton’s use of stories and myths, his wisdom, his challenge against orthodoxy, and his determined practical focus on utility. My first evaluation (1991) was a process and outcome evaluation of a host responsibility programme; my MSocSci thesis (1992), a retrospective evaluation of a healthy aging programme. My evaluation journey had begun!
My interests in prevention and environmental/system level analysis lead to an initial role in public health promotion and opportunities to integrate research and evaluation activity into this work. Social research and evaluation positions followed, initially within the Alcohol and Public Health Research Unit (APHRU), Department of Community Health, University of Auckland, and then later within Forsyte Research and Gravitas Research and Strategy. These latter roles provided opportunities for me to continue to work in public health, health promotion and social marketing as well as to undertake social research and evaluation across the diversity of local, regional and central government.
More recently, I have responded to a yearning (and need) to further develop the theoretical basis of my practice, initially through completing the PGDipSSER and then through a PhD in evaluation. During my PhD studies, I worked part-time within SHORE, Massey University and again enjoyed working within an academic and applied research and evaluation environment.
My current role as an internal evaluator within Te Pou responds to my desire to work on the ‘inside’ for a change. The possibilities of developing evaluative thinking, frameworks, and practice within a large and complex organisation, are providing me with a whole new set of evaluation challenges and rewards. I am constantly reminded of our privileged and unique position as evaluators and of our responsibilities to practice with care and conviction.
Ngā mihi ki a koutou katoa
I grew up in South Canterbury with Aoraki as my mountain, and Rakitata as my river, which provided our family with huge salmon, and many happy holiday memories at the family batch.
There were 100 kids in the third form at Temuka District High School. I was one of the lucky ones with parents who could support me to further my education: Sixteen of us made it to the sixth form (Year 12), and two of us ended up with university degrees.
At Canterbury University I first studied English and Social Studies, and became a high school teacher, but then went back to Uni to study Psychology. At first, I used my psychology degree in a number of counselling roles, but when I ended up at Queen Mary Hospital in Hanmer Springs, the American CEO was passionate about evaluation, so I evaluated the addiction programmes, and in 1996 went to my first evaluation conference in Wellington.
In the next two years I completed the Post Grad Diploma in Evaluation online through Melbourne University, and in early 1999 our little family of three moved to Wellington, where I undertook to manage the pilot evaluation of the first three Family Start sites. Suddenly, I was working in the Public Sector, based at (what was then) the Ministry of Social Policy, and jointly managed by Health, Education, and Social Policy. It was a huge shock to the system!
Since 2001 I have worked for Te Puni Kōkiri, the Ministry of Justice, and the Department of Corrections, where I have now been for more than three years. I generally do a mix of undertaking my own research and evaluation projects, working together with contractors, or simply managing the contractors.
Currently, I have three main projects in this financial year that take my time: an impact evaluation of a new design for Community Probation buildings, and how that affects the practice and engagement of Probation staff with offenders and their families; a formative evaluation of the implementation of the change in legislation that led to children up to the age of two years being able to stay with their mums in prison; and a pilot of a new initiative (called ‘Right Track’) at Auckland Prison that equips frontline staff with strategies to proactively engage with prisoners and motivate them to make changes.
So you can see that the work is varied. My latest challenge is to set up systems and processes for monitoring the roll-out of ‘Right Track’ to all other prisons in NZ, and to build real-time reporting into that. As always, there is a lot of negotiating to do with a wide range of stakeholders – but I enjoy that aspect very much, and it is one of the advantages of being an evaluator within an agency.
Nō reira, tēnā koutou, tēnā koutou, tēnā koutou katoa.
|Kia ora koutou katoaNō Hōterani me Āirihi me Ingārangi me WīWī ōku tipuna.
I whānau au i Te Papa-i-oeia.
I tipu ake ahau i Te Tihi-o-maru.
E noho ana tōku whānau i Tāmaki Makau-rau i tēnei wā.E noho ana ahau i Te Whanganui-a-Tara ināianei, ā, he kaiarotakenga ahau mō Te Puni Kōkiri.Ko Paula White toku ingoa.
My ancestors are an eclectic mix of Scottish, Irish, English and French.I was born in Palmerston North. I grew up in Timaru, and these days my parents and siblings live in Auckland.I currently live in Wellington, where I work as an evaluation advisor for the Ministry of Māori Development. My name is Paula White.
As a Pākehā evaluator working in a Māori context with people from backgrounds and cultures that are different from my own, my current work can be challenging, even confronting at times. Having said this, the insights I gain working alongside a variety of culturally-centred initiatives that aim to realise the potential of marae, and Māori communities, whānau and young people, I find personally very rewarding.
In my early career as an evaluation advisor for several government agencies, a question I have grappled with is how to provide relevant evaluation support to initiatives that are grounded in implicit cultural values, and emergent in both policy and operations. For such complex contexts, a useful strategy I have found is to facilitate Developmental Evaluation (DE) as an initial phase in a longer-term evaluation framework. It allows innovation, helps to set a new initiative up for success, and lays a foundation for formative and summative evaluation later.
I first became interested in evaluation ten years ago because I saw it as a potential ‘bridge’ between applied policy research and social policy and programme development, which to me appeared disconnected at times. I still believe evaluation has this potential – provided it is done well. And so I’d like to share three things I’ve learned so far that relate to doing evaluation well.
Find a mentor as a professional sounding board! My ongoing professional development in evaluation owes much to the invaluable real-world advice that several more experienced evaluation counterparts share with me.
Relationships are everything for evaluators, especially in Aotearoa. In my experience, relationships and how we manage them will ultimately direct how ‘good’ our evaluation work will be. See my published reflections on this that I co-authored in 2011 with Dr Amohia Boulton in the New Directions for Evaluation journal.
Effective public sector evaluation is made possible through quality evaluation scoping, quality commissioning and quality managing. See this hot tips presentation which offers strategies for avoiding common pitfalls, co-presented with Dr Jane Davidson at the American Evaluation Association Conference in 2012.
I always enjoy chewing the evaluation fat with others, you can contact me here.
I am a fourth generation Pākehā New Zealander, with ancestral roots to the Orkney Islands and Kent, in the United Kingdom. I grew up in Murihiku (Southland) at the southern end of South Island. Annie and I now live in Napier. We are proud parents of three adult children and three (so far!) grandchildren.
I have been working in education evaluation for the past 22 years, in early childhood, primary, secondary and tertiary sectors, mainstream and kaupapa Māori contexts, public, integrated and private provision.
I fell in love with the idea of evaluation thanks to some powerful formative experiences in my early years working in the Education Review Office. How on earth do you evaluate the quality of a complex policy area like education, do you look at the quality of provision, the learning gains of students, the quality of teachers/teaching, all of the above or something else? The challenges are huge but I became hugely interested in trying to develop some credible, educationally valid and useful answers these questions.
For the past 12 years I have been involved in developing evaluation policy, including evaluation frameworks and methodology, and practical tools such as key evaluation questions, evaluation indicators (merit criteria) and performance criteria rubrics. I have also been extensively involved in training and supporting evaluators involved in implementing the evaluation process.
In recent times I have been interested in developing evaluation frameworks that can apply in other policy areas as well, not only in education. Applied evaluative thinking and practical frameworks – these are the keys I think. I love the challenge of working things out, figuring out what is really happening, the people, the subject at hand, the challenges.
Kia ora koutou! Ko Jane Davidson taku ingoa.
My kiwi whakapapa: I’m a 6th generation Pākehā, descended from Scottish railway workers and English farmers. My dad was an electrician and my mum a nurse. All practical lines of work! I was the first generation in my family to go to university.In my family we like it practical and we like it clear and simple. When people lose me with their big words, complicated theory or analyses, I am always deeply suspicious this is due to a lack of clear thinking on their part (rather than just myself being verbally challenged!). I like cut-to-the-chase thinking and ideas I can really use.
My evaluation whakapapa: Early on, I worked in quality assurance and ran quality circles. Later, I studied industrial/organisational psychology, where I learned methodologies like the Critical Incident Technique for creating performance rating scales. I was frustrated at how overcomplicated it was in practice though, and ended up developing a set of rubrics instead, the first of many for performance appraisal.
I left for Claremont Graduate University in 1997 to do my Ph.D., discovered and loved programme theory early on. A year later Michael Scriven arrived, and I started learning about evaluation logic and methodology. I liked his incisive and critical thinking, but found some of it very high powered and theoretical and was searching for ways to translate it into practice. This brought me back to rubrics, and I worked on developing ways to apply them (alongside programme theory) to draw value-transparent conclusions.
What I work on these days: My interest in organisational learning has led me from “doing evaluations” into infusing evaluative thinking (“what makes us think this will work” and “how will we know early on if it is”) at the front end. I help people build outcome-focused evaluative thinking into policy and programme design. Sometimes I do this with them and coach/learn as we go; sometimes I do it through workshops or webinars.
What I’m thinking, writing and talking about: I wrote my textbook, Evaluation Methodology Basics: The nuts and bolts of sound evaluation just before my first little girl arrived. I’ve found it easiest to juggle having a young family (now 8, 4½ and 4½!) with short writings on the Genuine Evaluation blog I run with Patricia Rogers. She and I also co-authored a chapter on Australian and New Zealand evaluation theorists for Marv Alkin’s latest edition of Evaluation Roots.
I’ve recently started writing low-cost e-minibooks that summarise key concepts from my workshops. My first was Actionable Evaluation Basics: Getting succinct answers to the most important questions. I have a few more on my ‘to do’ list, including causal inference, commissioning evaluation, and evaluation rubrics.
I’m also slicing smaller pieces off these to offer as webinars because many people like to hear explanations ‘live’ and ask questions, and this is a low-cost way to be able to offer that. I also find it’s a great way to sharpen my own thinking.
For useful resources and links, or to get in touch, join me on http://RealEvaluation.com Kia ora!
Dr. Carol Mutch
Dr. Carol Mutch was on TV 1 Breakfast and National Programme Mediawatch talking about the role of the media during Pike River and the Canterbury earthquakes. Here is the Mediawatch link Mediawatch for 2 September 2012 An outsider’s view of the media response to disaster and recovery in Canterbury and Pike River; one year on from the death of the UK’s top-selling tabloid, is the world better off without The News of the World? (35′26″)Download: Ogg Vorbis MP3 | Embed