Application of M&E System Strengthening Tool (MESST) to assess the M&E systems for HIV prevention and treatment programme in Sri Lanka

Introduction: Global Fund to fight AIDS, TB and Malaria (GFATM) is one of the main funding sources for Sri Lanka to achieve its national objectives on HIV prevention, treatment and care. The GFATM HIV/AIDS programme is collectively implemented by the National STD/AIDS Control Programme (NSACP) and non-government umbrella organizations (UOs). Grant recipients have to show results when requesting funds. Therefore, it is agreed that country HIV M&E systems should comply with the standard 12-components M&E system strengthening (MESS) tool. The objective of this assessment is to identify current strengths and weaknesses of the M&E systems and to develop a plan of action for system strengthening. Methods: MESS tool was used for the assessment which has 12 Excel worksheets to be completed by the M&E stakeholders in the NSACP and UOs. Three multidisciplinary groups (10 stakeholders per group) were formed and each was given 4 sheets to be completed by participatory approach. Worksheets have M&E related statements to be collectively graded by stakeholders considering the country context and M&E experiences. Results: MESS tool generate summary dashboards with regard to the collective grading of statements of the tool under 12-components. It showed both strengths and weaknesses in the prevailing M&E systems for HIV in the NSACP and UOs. Organization structure, human capacity, national and sub national databases as well as supportive supervision & auditing are main areas identified by the tool that need strengthening. Conclusions: MESS tool is an important organizing framework to identify strengths and weaknesses and gaps in the M&E system of the NSACP and UOs. Results reflect the areas to be strengthen and rectified in order to show realistic programme outcome and impact results.


Introduction
Global Fund to fight AIDS, TB and Malaria (GFATM) is one of the main funding sources for Sri Lanka to achieve its national objectives on HIV prevention, treatment and care.The GFATM HIV/AIDS programme is collectively implemented by the National STD/AIDS Control Programme (NSACP) and non-government umbrella organizations (UOs).Grant recipients have to show results when requesting for funds.Therefore, it is agreed that country HIV M&E systems should comply with the standard 12-components M&E system strengthening (MESS) tool.The objective of this assessment is to identify current strengths and weaknesses of the M&E systems and to develop a plan of action for system strengthening.

Methodology
MESS tool was used for the assessment which has 12 Excel worksheets to be completed by the M&E stakeholders in the NSACP and UOs.Three multidisciplinary groups (10 stakeholders per group) were formed and each was given 4 sheets to be completed by participatory approach.Worksheets have M&E related statements to be collectively graded by stakeholders considering the country context and M&E experiences.

Results
MESS tool generates summary dashboards with regard to the collective grading of statements of the tool under 12-components.It showed both strengths and weaknesses in the prevailing M&E systems for HIV in the NSAP and UOs.Organization structure, human capacity, national and sub national databases as well as supportive supervision and auditing are main areas identified by the tool that need strengthening.

Introduction
According to the classification of HIV epidemics, Sri Lanka is currently experiencing a low level epidemic.HIV prevalence in the general population is less than 0.1%.It is estimated that about 3,000 people are currently living with HIV in the country.As of end 2014, 2073 HIV positive cases, 336 AIDS deaths and 71 cases of mother-to-child transmissions were reported to the National STD/AIDS Control Programme, Sri Lanka.Majority of the reported cases were males (62%) and the male to female ratio was 1.6:1.Based on the reported cases, majority (76%) of the infected people are in the 25-49 year group.Analysis of new HIV cases in 2014 (N=228) showed that mode of transmission was male-female sex in 55% of cases and 28% were due to male-male sex, and 1% were due to injecting drug use and about 16% the probable mode of transmission was not clear and unknown.(1) In accordance with the three ones principle of UNAIDS, Sri Lanka has one agreed HIV/AIDS action framework, one national AIDS coordinating authority (National AIDS Committee) and one agreed country level monitoring and evaluation system (National HIV M&E Plan).(2) Global Fund to fight AIDS, TB and Malaria (GFATM) as the main regional donor has been supporting Sri Lanka to achieve its national objectives on HIV prevention, treatment and care.The GFATM HIV/AIDS programme is the main funded country work plan which is collectively implemented by the government and non-government partners.GFATM and other international donors like WB, UN, WHO, USAID are putting lot of pressure to show results in their monetary investments for health.Therefore, it is agreed among donors that national HIV M&E systems should follow the standard 12component HIV M&E organizing framework (3) to show the HIV project and programme results as well as national outcome and impact results.Therefore, the application of 12-component M&E system strengthening tool (MESS tool) to government and non-government partners of the GFATM programme is one of the requisites in order to improve M&E capacities across all the partners.The objective of this assessment is to identify current strengths and weaknesses of the M&E systems for HIV prevention and treatment services in the National STD/AIDS Control Programme (NSACP) and other non-governmental umbrella organizations (UOs) and to develop an agreed plan of action for M&E system strengthening

Methodology
Methodology is mainly based on the grading of statements provided on Excel worksheets of the MESS tool and action points identified during the stakeholder group activity (MESS workshop).There are 12 worksheets for the twelve components (Table 1) of the tool and M&E stakeholders have to complete them with the MESS workshop facilitator.Initially, three multidisciplinary groups (10 stakeholders per group) were formed with almost similar composition and balance of power and each group was then given 4 sheets to be completed by participatory approach with group consensus.Worksheets have M&E related statements to be collectively graded by stakeholders considering the country context and M&E experiences.

The tool (12-component M&E system strengthening tool or the MESS tool)
This tool is a free and an open source material, initially developed by the World Bank Global AIDS M&E Team (GAMET) in partnership with UNAIDS, US government, Global Fund and other partners.All partners agreed and accepted this as an organizing framework for HIV M&E.Therefore, this standard tool (MESS tool) is an organizing framework, through which all aspects of an HIV M&E systems can be assessed, organized and strengthened.The tool has been broadly categorised in to three circles (Fig 1), outer circle is about the people, partnership and planning.Inner circle is about collect, capture and verify and the centre is for the use of data and decision making.12components under each circle are outlined in the table 1.If the answer to a statement is anything other than "Yes-completely", respondents are requested to provide an explanation in the COMMENT box provided in the Excel sheet.This will provide important information to consider in discussing the necessary actions to be taken.The electronic version of the tool is set up to facilitate selection of the responses and summarizing the assessment results at the end of the assessment.(3) The appropriate response can be selected from a drop-down menu in the ANSWER column.The "Yes-completely" responses will appear in Green, "Mostly and Partly" will appear in Yellow, and "No-not at all" will appear in Red.This visualization in colour will help with summarizing the assessment results and with prioritizing actions.A SUMMARY DASHBOARD (as in Fig 2 and Fig 3) of the results will be automatically generated and the distribution of scores (overall and for each of the 12 components) will be displayed.It provides a visual representation of gaps in the M&E system at different levels.A detailed review will help to prioritize the actions to be taken.(3) Application of the MESS tool MESS tool was applied to assess the Sri Lankan HIV M&E systems of the two principal recipients (PR) of the GFATM grant (Global Fund to fright AIDS, Tuberculosis and Malaria).Principal Recipient 1 (PR1) is the government National STD/AIDS Control Programme (NSACP) of the Ministry of Health and the Principal Recipient 2 (PR2) is the non-government umbrella organizations (UO) working in partnership with the NSACP under the GFATM HIV programme.
A One day workshop (MESS workshop) was conducted in Colombo, on 12 th of December 2014 with the participation of 30 multi-sectoral and multi level stakeholders having responsibilities of M&E work in the NSACP and UOs.
Potential participants were initially identified by using a stakeholder analysis.The workshop facilitator made a Power Point presentation on the MESS tool format and relevant instructions about how to complete worksheets in the tool.
Then three mixed groups were formed adjusting the equal sectoral representation and power within the group.Each group was given a laptop computer with installed MESS tool and was instructed to open the 12-component workbook in the format of "all stakeholder categories" using the relevant drop down option on the cover page of the tool.Four separate components (4-sheets) of the tool were given to each group to be completed during the MESS workshop.
Participants were given instructions to complete sections (columns of excel sheets) relevant to the NSACP and umbrella organizations (UO).Umbrella organizations comprised of Family Planning Association, Heart to Heart, Community strength development foundation (CSDF), Alcohol and Drug Information Centre (ADIC), Saviya Development Foundation (SDF) and Sri Lanka Red Cross (SLRC).
Each group completed and reviewed their respective allocated components (Excel sheets), and at the end, each group was given time to discuss the findings of the assessment in front of all the stakeholders.During the presentations, tool statements, relevant dashboard bars and action points were discussed and suggestions were noted down by the members of the respective smaller groups.Then again a period of time was given to re-grade the tool statements and also review the action points based on the discussions and suggestions.Then the facilitator consolidated the work sheets to a master workbook.A SUMMARY DASHBOARD of the results was automatically generated once the sheets were consolidated and these dashboards and the summary of action point list are the final output of the tool.

Results
MESS tool generates summary dashboards with regard to the collective grading of the statements of the tool under 12-components.HIV M&E organizational framework was assessed using the MESS tool in the NSACP and UOs.It provides a visual representation of strengths and weaknesses in the HIV M&E framework at NSACP level and at UO level. .National M&E work plan was not available and was on the process of development at the time of the administration of the MESS tool.Furthermore, the tool generates a sheet of action points under each of the 12 components based on the actions identified by the group which will help to focus on activities to be carried out in order to strength the system.Partnerships between the NSACP and UOs are maintained relatively at a satisfactory level because national level reporting needs are mandatory in the performance base funding system of the GFATM grants.However, there are deficiencies in data reporting formats in some entities and also deficiencies in supportive supervision and data auditing and verification which need to be strengthened for better M&E outputs.Furthermore, the tool generates a list of action points which can be used to develop a future action plan in order to carry out strengthening of the HIV M&E system.

Fig 1 :
Fig 1: 12-components of the MESS tool arranged in circles National STD/AIDS Control Programme (NSACP) Overall, M&E system in the NSACP has strengths as well as weaknesses and gaps.M&E partnerships, National HIV M&E plan, M&E advocacy communication & culture and evaluation & research, use of M&E data are relatively better at the NSACP.Performance of surveillance and surveys are having about 50% strength.However, Organization structure, human capacity, National and sub national databases as well as supportive supervision & auditing are main areas the warrant strengthening at the NSACP level (Fig 1)