Kia Piki ai te Oranga a Nga Whānau me Nga Hapū
Impact Research NZ worked in collaboration with Shea Pita who were commissioned by The Ministry of Health to provide a series of reports to inform the Kia Piki te Ora Māori suicide prevention post evaluation programme in Aotearoa New Zealand communities.
The first report presented a literature review to understand the context of Māori suicide by analysing available Māori suicide data, understanding the diverse causes of suicide for Māori, the current health policy framework underpinning New Zealand’s response to Māori suicide prevention and an understanding of effective interventions. The second report involved presenting a landscape review and comparative analysis of relevant strategies, programmes or initiatives that may impact upon the Māori suicide prevention landscape.
The third report involved outlining a draft model to support outcomes-focused Māori community leadership in suicide prevention.
Weir, A., McEwan B., & Shea, S. (2016) Understanding the drivers, impact and ‘what works’ for Māori suicide prevention within Aotearoa New Zealand communities. Wellington: Ministry of Health
Shea, S., & Weir, A. (2016). A landscape and comparative analysis of Māori suicide prevention within Aotearoa, New Zealand. Wellington: Ministry of Health.
Shea, S., & Weir, A. (2016). Kia Piki ai te Oranga a Nga Whānau me Nga Hapū : A draft model to support Māori communities to lead outcomes-focused suicide prevention, within the overall context of Community Wellbeing. Wellington: Ministry of Health.
START Taranaki Results Based Accountability (RBA) Evaluation
START Taranaki commissioned Impact Research NZ to work with them to undertake and organisation review of their evaluation and reporting practices as well as to develop and implement a Results Based Accountability (RBA) framework. The project also involved providing START Taranaki staff with training in RBA, linking the strategic and operational plans to an organisational evaluation plan.
Three groups of stakeholders were surveyed utilising an RBA surveys. The perspectives of youth were surveyed via a focus group and a paper-based survey. Youth were asked about their expectations and experiences of the programme, their relationship with their key worker, and suggestions for potential programme improvements. Youth were also asked whether they had changed as a result of being on the programme, if they had developed new skills and coping strategies and if they would change their behaviour in the future.
Individual interviews and focus groups with staff and management of START Taranaki asked about their perceptions of the START Taranaki organisation, and the programme fit with other available programmes. Key stakeholders were asked for their perceptions of the underlying philosophy of START Taranaki and the benefits of their organisations’ relationship with the programme.
An overall evaluation report outlined stakeholders’ perceptions of the effectiveness of the START Taranaki programme, the extent to which it met its intended outcomes and suggestions for ways in which the programme could be improved. A Results Based Accountability manual was also created to enable START Taranaki to continue to evaluate the programme and monitor its success for themselves.
Weir, A., McEwan, B., & Holmes, J. (2016). START Taranaki Results Based Accountability (RBA) Survey Report. Impact Research NZ, Auckland.
Family Works Resolution Service RBA Evaluation
Impact Research NZ was commissioned by Family Works Resolution Service to undertake and evaluation of its service by asking clients to participate in a paper-based or online survey which asked them to indicate how well they believed the service met their expectations, and how effective it was in assisting them to reach a lasting arrangement for their care of their children. A total of 77 clients responded and a report of the survey. Findings will provide details of client demographics and level of their involvement with the service. In addition clients’ experiences of the service, the benefits to clients and their children of their involvement and suggestions for improvements to the service will also be presented.
Weir, A., & Holmes, J. (2016). Family Works Resolution Service: Results Based Accountability Survey Report 2016. Impact Research NZ, Auckland.
Enliven Home Based Support Results Based Accountability (RBA) Evaluation
Presbyterian Support Northern (PSN) commissioned Impact Research NZ to undertake an evaluation of the Enliven Home Based Support service. In 2016, 370 clients responded to a paper-based survey. Questions asked clients to indicate how effectively the service met their needs and situations, the positive benefits experienced as a result of their involvement with the service and suggestions for service improvements. Six Enliven sites each received a report outlining their regional results and one overall evaluation report which provided information on the effectiveness of the Enliven Home Based Support service, the extent to which the service met its intended outcomes and the positive benefits that clients experienced as a result of their involvement as well as suggestions for ways in which the service could be improved.
Weir, A., & Holmes J. (2016). Enliven: Results Based Accountability Survey 10 Service Report 2016. Impact Research NZ, Auckland.
Social Workers in Schools (SWiS) RBA Evaluation
Presbyterian Support Northern (PSN) commissioned Impact Research NZ to undertake the Social Workers in Schools (SWiS) RBA Evaluation. At the time of the survey a total of 126 schools based in the upper North Island were eligible to participate. There were795 children involved with the service which was delivered by 65 social workers. Children and their parents and caregivers were provided with a paper-based questionnaire to complete and members of school staff responded to an online survey. Questions asked each group to indicate how effective they thought the service was, what the benefits of being involved with the service were for children, parents and caregivers and school staff and suggestions for ways in which the service could be improved. In 2016 responses were provided by more than 440 people surveyed. Eight PSN sites each received a report outlining their regional results to present to the schools in their area. One overall service evaluation report provided information on the effectiveness of the SWiS service, the extent to which it met its intended outcomes and clients’ suggestions for improvements to the service.
Weir, A., Holmes, J., & Schonberger, S. (2016).
The Royal New Zealand Plunket Society – Family Works Northern research on the joint Early Intervention Service
The Royal New Zealand Plunket Society and Family Works Northern collaborated together to provide a short-term early intervention service. Impact Research NZ was commissioned to provide an evaluation of the effectiveness of a newly established Early Intervention Service in the Waikato area. Clients of the service participated in interviews to establish how well they thought the service met their needs, the positive outcomes experienced as a result of their engagement and suggestions for service improvements. Staff of Plunket and Family Works Northern participated in an online survey which asked them to indicate opinions relating to the success and strengths of the service and the importance of aspects of service delivery that contributed to its success. Staff were also asked for their suggestions for service improvements. An evaluation report presented the findings from the client interviews and staff online surveys to provide an insight into the ways in which the service was appropriately provided to clients and the extent to which the service provided the desired and intended outcomes for clients.
Weir, A., Holmes, J. (2015). Evaluation of the Royal New Zealand Plunket Society-Family Works Northern Huntly Early Intervention Service 2015. Impact Research NZ, Auckland.
Family Works Resolution Service RBA Evaluation
Family Works Northern and Anglican Trust for Women and Children Youth Workers in Secondary Schools (YWiSS) Check and Connect Service, Results Based Accountability (RBA) Pilot Survey (2015)
The Youth Workers in Secondary Schools (YWiSS) Check and Connect service is jointly provided by the Anglican Trust for Women and Children (ATWC) and Presbyterian Support Northern/Family Works Northern (PSN/FWN). Impact Research NZ was commissioned to develop an RBA framework to evaluate the effectiveness of the YWiSS Check and Connect Service and its intended programme outcomes. Four groups of stakeholders were invited to participate in an online survey. Youth involved with the service, and their parents and caregivers were surveyed for their perspectives on how well the service was received, the impact of engagement for youth and suggestions for improvement to the service. Staff from seven schools in the Auckland area were surveyed for their perceptions of the YWiSS Check and Connect service and its impact on their roles within the schools.
YWiSS Check and Connect staff and members of management were also sought and questions included perceptions of the service, their roles and relationships with youth, parents and caregivers and school staff. A total of 73 responses were received and an evaluation report was provided which presented the results of all surveys. The development of the RBA framework and associated questions, agreed in consultation with ATWC and FWN were demonstrated to provide measurable data which assisted in the evaluation of the ability of the YWiSS Check and Connect service to achieve its goals and meet the needs and provide positive outcomes for clients. This approach also enabled an evaluation of clients suggestions to inform future service improvements.
Weir, A., Holmes, J. (2015). Family Works Northern and Anglican Trust for Women and Children Youth Workers in Secondary Schools (YWiSS) Check and Connect Service, Results Based Accountability (RBA) Pilot Survey Overall Service Report 2015. Impact Research NZ, Auckland.
Family Works Northern Results Based Accountability (RBA) Casework Evaluation Client Survey Report
Impact Research NZ was commissioned by Family Works Northern to undertake an evaluation of their Family Works Northern Casework service, which provides support to individual clients and families/whānau through the provision of social work and family counselling. In 2016 a mailing house was trialled to invite the three stakeholders groups to participate in a client survey. Children aged 7-12 years were surveyed regarding their happiness, experience of school, problems, safety, ability to manage challenges, knowledge of where to obtain help and opinion of their Family Worker. Adult and youth clients were surveyed regarding their experiences with the service, the degree to which the service had helped them or their family/whānau and suggestions for improvement to the service. A total of 178 client responses were received and the results of all surveys were evaluated and compiled into a report. The RBA evaluation provided measurable data on the effectiveness of the Family Works Northern Casework service at meeting the needs of clients and producing positive outcomes. The evaluation also incorporated client suggestions for improvements to the service. The trial of the mailing house resulting in fewer responses than previous methods of survey delivery.
Weir, A., & Holmes, J. (2016). Family Works Northern Results Based Accountability (RBA) Casework Evaluation Client Survey Report 2016. Impact Research NZ, Auckland.
Family Works Resolution Service: Results Based Accountability Pilot
Impact Research NZ was contracted by Family Works Resolution Service to develop, trial and evaluate a survey mechanism to gain evaluative feedback about the provision of Family Works Resolution Service. A combination of survey and interview data is being collected from Family Works Resolution Service clients, utilising a Results Based Accountability (RBA) approach.
In addition, survey and interview data is also being collected from key stakeholders, including call centre staff, contracted mediators and preparatory counsellors, and managerial personnel. The client survey mechanism is aligned with the Ministry of Justice and Family Works Resolution Service’s Key Performance Indicator (KPI) requirements, as part of monitoring and reporting on the provision of services, including the production of management and funder reports. KPI and RBA reporting, when combined, will form a comprehensive evaluative framework.
Developing NGO Capacity to Undertake Evaluation
2015 – Current
Impact Research NZ has been working with a large NGO (that offers a broad range of social services for families in South Auckland) to undertake an organisational review of their tools used to evaluate their programme and services. Workshops have been conducted on developing an organisation and programme evaluation plans linked to their Annual Plan and Strategic Plan. Other workshops focused on developing and implementing RBA and innovative models of evaluations. The NGO was also interested in its organisational responsiveness to Maori and Pasifika and both hui and fono were held facilitated by Impact Research NZ and reports provided to the NGO for follow-up.
Community Initiative – Process Evaluation
Impact Research NZ, in partnership with David Turner Research Ltd, undertook a process evaluation of a community initiative pilot project funded by MSD currently being trailed in two Auckland districts: Manukau and Waitakere. The pilot project is targeted to assist low income individuals and families to improve their circumstances. A process evaluation assesses whether a program suits the context for which it was designed, if it has been appropriately implemented, and if it has reached those persons for whom it would provide the most benefit.
The evaluation methodology utilises a combination of interviews and document analysis to achieve the evaluation objective. The evaluation focuses on the quality and quantity of services delivered through the programme, both from a programme perspective and from the client perspective. The process evaluation was based on the programme’s intervention logic, which describes the problem to be addressed as well as identifying programme inputs, outputs, and intended outcomes. The evaluation l was used to inform future ongoing monitoring of the programme.
Turner, D., Weir, A., McEwan, B., & Holmes, J. (2015). Community Finance Pilot: Process Evaluation 2015. Ministry of Social Development: Wellington.
Social Workers in Schools RBA Evaluation
Presbyterian Support Northern commissioned Impact Research NZ to undertake the Social Workers in Schools (SWiS) RBA Evaluation. 126 schools in the upper North Island participated in the survey. The number of SWiS in schools = 71. The number of children in SWiS service = 1653. In 2015 the service evaluation included responses from more than 650 responses. Children and their parents/caregivers participated in a paper-based survey and the school staff in an online survey designed to find out how well the service is doing, suggestions for improvements and what are the outcomes of the service for children, their parents/caregivers and the school staff. The ten regional sites each received a report to share with their schools and an overall report provides information on how well the services is doing in reaching its intended outcomes and what services improvements are to be made.
Family Works New Zealand RBA Pilot Survey
The Ministry of Social Development funded the establishment of a common outcome measurement framework, including conducting a baseline RBA pilot survey of clients, across seven Presbyterian Support regions throughout New Zealand. Impact Research NZ was commissioned to undertake the development and implementation of the RBA pilot surveys of children, youth and their parents/ caregivers. The development of the survey instruments was done in collaboration with the seven regions. Each region received a report on their survey findings including client demographics, quantitative and qualitative data, as well as an overall report looking at the Family Works NZ service RBA survey results as a whole.
An External Review of the Response Made by The Salvation Army to the two Canterbury Earthquakes of September 2010 and February 2011
Impact Research NZ was commissioned by The Salvation Army to undertake an external review of its response to the two Christchurch earthquakes of September 2010 and February 2011; with the aim of discovering what could be done to ensure that The Salvation Army’s response to future disasters was appropriate, effective and consistent with its Mission Statement. The researchers worked in collaboration with senior Salvation Army staff to formulate the overall research design, and to identify the necessary research documentation and interview participants. A literature review was undertaken to identify key research findings in relationship to best-practice disaster management.
The research project undertook two stages of data collection. In stage one, a document review and analysis was undertaken assessing a range of earthquake-response documents including: Salvation Army-based daily field reports, email communications between staff, quarterly and annual Board reports, financial reports related to earthquake fundraising and expenditure, website press releases, and two low-level review reports completed by the Emergency Services. In stage two, a mixture of face-to-face and telephone interviews were undertaken with senior Salvation Army staff; and key informants selected from government, local government and NGO personnel involved in the Canterbury earthquakes.
The research findings, in conjunction with the ‘best practice’ disaster-response research literature, identified a number of organisational opportunities for The Salvation Army to consider when reviewing its Disaster Response Policies, Protocols and Practices.
Impact Research NZ has been contracted by Family Works Resolution Service to develop, trial and evaluate a survey mechanism to gain evaluative feedback about the provision of Family Works Resolution Service. A combination of survey and interview data is being collected from Family Works Resolution Service clients, utilising a Results Based Accountability
This evaluation of day programmes for people living with dementia, aims to identify the elements of an effective client-focussed programme, identify the methods employed by organisations to measure the quality of outcomes of programmes, and lastly to determine the efficacy of the methods adopted.
In Collaboration with the Royal New Zealand Plunket Society and Family Works Northern, Impact Research NZ have designed and implemented an evaluation plan, ultising a Results Based Accountability (RBA) framework, to access the outcomes of an early intervention partnership pilot. This evaluation aligns with finding and recommendation from an earlier evaluation on the pilot service conducted by Impact Research NZ, which identified that the ongoing collection of robust outcomes-focused data would support the success of the service.
Ⓒ 'Dancing with Data' Programme
This practical evaluation programme was offered in partnership with Auckland Communities Foundation and funded by SKYCITY Auckland Community Trust Board. Three cohorts of participants from social service NGOs completed the programme, which comprised three workshops of two hours duration, with an expectation that development activities would be completed between the workshops; an overall commitment of approximately 12 hours over the duration of programme.
Impact Research NZ have were commission in 2013 and 2014 to undertake qualitative thematic analysis on data collected from multiple sites from clients on their experiences and outcomes of engaging with Family Works Northern’s social support services.
Enliven Stakeholder Survey
March 2014 - March 2013
An online survey of stakeholders, including District Health Boards and Needs Assessment Agencies, was designed and conducted on behalf of Enliven in 2013 and 2014. The survey gathered both quantitative and qualitative data in order to gain an understanding of the Enliven service’s performance from the perspective of its stakeholders.
Impact Research NZ was commissioned by ACC Home and Community Services to undertake a scoping exercise in order to determine the how the six lead providers of services currently collect and use client data to support the quality of their service provision. Multiple data collection methods were used to gather data from a range of stakeholders.
This evaluation of Enliven’s home support services for clients of ACC Home and Community Support Services designed an piloted a Results Based Accountability (RBA) survey, in order to measure outcomes of the service.
The Royal New Zealand Plunket Society and Family Works Northern collaborated to pilot an innovative model of early intervention social service provision in two areas of high socio-economic over eighteen months. Impact Research NZ was commissioned to conduct an evaluation of the early intervention pilot utilising a participatory approach that engaged key stakeholders and used a range of data sources. The evaluation addressed the effectiveness of the service in terms of meeting its goals and objectives, identified factors that supported effective service delivery, and made recommendations for service delivery going forward.Read more
Waitakere Family Start is a collaborative service delivered by West Fono Health Trust and Presbyterian Support Northern. The service supports families/whanau through the early years of a child’s life, with the child at the centre of service delivery. Impact Research NZ conducted a pilot evaluation of the service, utilising two different approaches to the collection of feedback from clients, in order to assess outcomes for clients of the service, and whether there were advantages to either data collection method, in terms of supporting service outcomes and quality.
Presbyterian Support Northern, Whakamanhia Wahine Tauranga Women’s Programme Evaluation
June 2013 – November 2013
Impact Research NZ was commissioned by Presbyterian Support Northern to conduct an evaluation of their pilot programme for women offenders, funded by Department of Corrections as part of their Innovation fund aimed at reducing reoffending. The evaluation aimed to explore how effective the programme was in achieving its intended aims and included reviewing the programme implementation and how well it was received by participants and key stakeholders.
The goal of Child Youth and Family (CYF) is for children in care to be safe, happy and part of a family. If a child cannot return to their own family, permanent care is required. Care does not always address emotional and behavioural difficulties; this can lead to a cycle of failure and aggression. Read more
Inner City Interconnectedness
Connectedness refers to the relationships people have with people around them and their surroundings. It includes relationships with family, friends, neighbours and people they have met at clubs or through voluntary work for the community. Connectedness also encompasses the way people feel about the built environment: if there are good services available, if they can walk around their neighbourhood, if the public transport is easy to use and if they feel safe.Read more
Kidpower Healthy Relationships
In 2010 Impact Research evaluated the Kidpower, Teenpower, Fullpower, Healthy Relationships programme. The programme provided training in violence prevention, personal safety and self defence. It was designed for all ages and abilities, for individuals, staff and professionals. It included risk reduction and successful communication techniques. Read more
Kiribati – Environmental Migrants
Kiribati (pronounced “Keer-ah-bhass”) comprises three groups of islands in the South West Pacific. The people of Kiribati are not Kiribatese, but I-Kiribati (pronounced “Eee-Keer-ah-bhass”). Kiribati is expected to be the first country in which all land territory disappears due to global climate change. New Zealand has an immigration agreement with the government of Kiribati to relocate the first few environmental refugees.Read more
Midlands - Results Based Assessment
In 2010 Family and Community Services (FACS) commissioned Presbyterian Support Northern to conduct an outcomes-focused evaluation of Midland family and budgeting services. Impact Research (of PSN) conducted a Results Based Assessment (RBA) pilot and recruited 26 providers to complete three RBA surveys over a two year period. Read more
National Social Workers in Schools (SWiS)
This one year pilot was commissioned by Child Youth and Family (CYF) to evaluate the effectiveness of a Results Based Assessment (RBA) framework for measuring changes in SWiS services. Read more
This pilot-programme is trialling an early intervention that involves Plunket and Family Works Northern staff working together to support clients. It is expected that the two organisations will achieve more for clients through working together, than they could working independently. Read more
Results Based Assessment
Results Based Assessment (RBA) is a type of evaluation that measures outcomes. Government departments, not-for-profit organisations and educational services are increasingly relying on RBA to provide evidence for the success and quality of their practices. PSN has been at the forefront of RBA development since 2009. Using specially designed survey tools, PSN collects information from Family Works, Enliven, Coactive and Budgeting Services. Read more
Refugee Catch-Up Class
The refugee student catch-up class began in 2005 after Somali community members voiced concern about the under-achievement of their college-age, school children. The catch-up class is a collaborative effort between families and community. The majority of students are Somali and Eritrean. Read more
Safer Homes in New Zealand Everyday (Shine)
Shine delivers training on domestic abuse to a wide range of professionals who work in the social services or criminal justice system. A small number of these training programmes are open to the public. Read more
Social Workers in Schools
The Social Workers in Schools (SWiS) service is funded by Child, Youth and Family (CYF) under the Ministry of Social Development (MSD). The service was established in response to the impact social and family problems were having on students. Read more
The Theatre in Education Trust (THETA) Sexwise Programme
The New Zealand Health Curriculum states that the purpose of sexuality education is to provide students with the knowledge, understanding and skills to develop positive attitudes towards sexuality, to take care of their sexual health and to enhance interpersonal relationships. Read more
Waitakere Process Evaluation
In 2010 Presbyterian Support Northern (PSN) began a results based assessment (RBA) pilot with Social Workers in Schools (SWiS) based in Waitakere. Read more
Jun 2013 - Effective Client-Focussed Dementia Day Programmes in the Community: Improving the Quality of Life for People Living with Dementia and their Families
Nov 2012 - Evaluation of Safer Dance Parties and Large Events
Nov 2012 - Safer Parties Together: One Voice
Jan 2011 - Evaluation of the Kidpower Teenpower Fullpower Healthy Relationships Programme
Dec 2010 - Review of North Shore Family Violence Crisis and Advocacy Service Requirements