Evaluation


Evaluation study

ICN conducted an evaluation study of LFC™ during 2001 and 2002, focusing on impact and sustainability. The results are excellent. Findings indicate significant positive results, especially at the level of individual development. 

A final component of the evaluation is a longitudinal study of participants who have completed the programme. Due to commence in 2003, this study will help determine the long-term impact of LFC™ participants in areas such as policy development, community affairs and the development of others. It will also explore the impact and sustainability of LFC™ on participants’ career and professional development.

Over 75% of the governments, nursing leaders and health officials responding to a survey in the LFC™ evaluation considered the results to be sustainable. Like outcomes and impacts, sustainability depends on strong local support and effective follow-up programmes and activities. Where the financial capacities of a country are weak, special efforts will be required to keep up the momentum, sustain the positive outcomes and support long-term impacts. 

Strategies to support long-term sustainability include:

  • Fostering political will.
  • Fund-raising and resource-sharing between the public and private sectors.
  • Cohesive partnerships between ministries of health and education (or equivalent), and private training providers.
  • Involvement and support of top nursing managers.
  • Use of local training institutions.

Training Of Trainers (TOT) to implement the LFC™ programme at country level.

Executive summary

The following is the Executive summary of the Evaluation study:

This report summarises the key findings of a phased programme of evaluation of the International Council of Nurses Leadership For Change project, for those programmes implemented during the period 1996-2000.
The objectives of LFC™ are to enhance the contribution to improved health services of the participating nurses and their organisations. This is achieved by imparting new skills and knowledge to enable participants to manage and implement appropriate and proactive leadership strategies. 
As well as the nurse participants themselves, other potential beneficiaries of LFC™ include other nurses, managers and policy makers, the health care team, country governments and the public. 

The main elements of LFC™ include training workshops, planning and implementation of team projects, other structured learning activities, preparation of Individual Development Plans (IDPs), mentoring and coaching. In the case of the Phase 2 programme for the Caribbean and Latin America, there were team projects at both regional and national levels. 
Evaluation of LFC™ was an integral element within the overall programme. The main objectives of the evaluation were to determine the factors influencing long-term sustainability of results, the impact of the programme on the participants, and the main costs and benefits. The evaluation components were conducted by independent analysts over the period 2000-2001.They were based primarily on three interconnected elements:

  • A comprehensive document review, based on an analysis of project document.
  • An analysis of a questionnaire survey of stakeholders in participating countries.
  • 'Follow-up' case studies conducted in six selected participating countries.

The evaluation report is composed of five sections: 

Section 1 is the introduction and background of LFC™. 

Section 2 examines and reports on issues of relevance and validity of LFC™. It found strong support from participants and from other stakeholders for the continuing relevance of LFC™, with a majority view that the full potential of LFC™ has not yet been realised. As such, the responses highlighted that capacity building efforts should not be terminated after two to three years of operation, and that continued investment and efforts will be needed to consolidate the results and realise full potential. The report concludes that, on the basis of results from the three elements of the evaluation, LFC™ appears to have retained its relevance since it was first designed and implemented, and there is strong support for the continuation of the programme. 

Section 3 discusses issues related to the impact of LFC™. The vast majority of participants and most other stakeholders reported a general positive response to the effectiveness of the programme. It was also perceived that LFC™ was effective, in the sense that it has produced the expected outcomes in capacity building, creating/developing networks and contributing to policy strategy issues. 

In many cases, health leaders and health officials at ministries and elsewhere reported that LFC™ participants had assisted in planning/implementing change and that the programme provided a unique opportunity for developing leadership potential in young nurses. The response was also positive, but more muted, in relation to the question of contribution to policy formulation. Responses from stakeholders indicated that LFC™ primarily had an effect at individual and operational levels. The effect at regional or national levels was reported to be more variable and often less apparent. Participants and health leaders/officials attributed this to the difficulties individuals had in engaging with the policy-making process in some countries, and to the often relatively early phase of LFC™ activity in countries. 

Section 4 assesses the issues of sustainability of LFC™. The LFC™ strategy and its components have been designed and implemented with a view to sustaining the results both at the level of the participants and their nursing associations/institutions. Evaluation results suggest that most of the participants have acquired confidence and appropriate tools to organise and manage their work better, contribute to health reforms, and transfer their leadership knowledge and skills to other nurses. Most participants and other stakeholders were found to be broadly in support of LFC™ sustainability. Another aspect of sustainability is the extent to which there were unanticipated positive or negative effects of LFC™. Participants reported some unanticipated positive results and few negative unanticipated results. 

Section 5 highlights some of the key lessons learned from the evaluation. It is evident from the three evaluation reports (documents, questionnaires and case studies) that LFC™ has produced a significant impact in a number of countries. This impact has been most significant at the micro level (participants and mentors), and less significant at the meso (NNAs and employers) and macro (Government) levels because of some key macro constraints. Constraints included: a) the varying extent to which public sector reform initiatives might have created an environment more sensitised to issues of quality of care and the need for effective leadership and management, b) the varying capacity, power and resources of different local NNAs, and c) variations in the structure of national and regional nursing administration. 

Some of the capacities created as a result of LFC™ appear to be sustainable if continuous training is provided to participants and to additional groups of nurses (de-skilling or erosion of new capacity could occur if training is discontinued). One of the key constraints is the lack of domestic resources in many countries; another is the natural resistance to change of many institutions, often coupled with risk aversion. The report recommends that any successor programme should focus on direct support in education and development addressed to motivated, worthy nurse leaders, and institutional development to NNAs and influential stakeholders who have a bearing on LFC™ training, nursing policy, networking, programme preparation and self-evaluation. 

Last Updated on Friday, 20 May 2011 14:43