Canadian Government Executive - Volume 23 - Issue 09

December 2017 // Canadian Government Executive / 29 Allowances had to be made for adaptations to the Provincial Evaluation Plan to address gained insight, respond to any gaps in information, and ensure key perspectives were accounted for. comparisons and roll-up findings across the province. In addition, as the provin- cial evaluation progressed, it became clear that baseline data was not always available for some of the indicators and planned data sources (e.g. a Core Pro- vincial Patient Survey) would not be ob- tained in time. The Answer: A Collaborative and Adaptive Approach to the Evaluation The success of the provincial evaluation of A GP for Me was founded on the follow- ing key principles that guided the work of what was a hybrid evaluation team, made up of staff from the Doctors of BC as the contracting authority and MNPLLP as the contracted evaluator. Collaboration: Members of the Doc- tors of BC team and the MNP teamworked closely together, such that the provincial evaluation was a truly collaborative un- dertaking. This extended beyond fre- quent, joint check-in meetings to include the co-facilitation of consultations as well as the assigning of work across the com- bined teams. It was also critical to coordi- nate with the work of other evaluators, be it the ones engaged by divisions or those working on components of the Provincial Evaluation Plan. Integration: The provincial evaluation had to integrate the foundational work of the Divisions of Family Practice and that of several other external evaluators, as well as administrative data from the Ministry of Health. With the divisions, a tiered, thematic analysis was carried out of thirty “Final Evaluation and Project Implementation Close-Out Reports” in the form of a high-level summary, divi- sion highlights, and direct report extracts. In addition to these close-out reports and what was referenced above, there were the results of surveys (e.g. of family physicians, with the GPSC, and the Cana- dian Community Health Survey), inter- views with health authorities and the BC Ministry of Health, and facilitated focus group sessions. Adaptation: Allowances had to be made for adaptations to the Provincial Evaluation Plan to address gained in- sight, respond to any gaps in informa- tion, and ensure key perspectives were accounted for. It was also critical to follow an iterative approach to validating early findings and interpretations. This took the form of a staged confirmation of the findings with a variety of stakeholders, covering the vari- ous components of the analyses (e.g., in- terpretations of the administrative data). The Takeaways: Reflecting on the Experiences of the Evaluation Just as A GP for Me has illustrated that there is not one solution to address the complex challenges of BC’s health care system, the provincial evaluation high- lighted opportunities for future practice. There are clear benefits to be gained from adopting hybrid evaluation teams, bringing together experienced staff and third-party contractors. Yet, it is impor- tant to also keep in mind the mainte- nance of harmony and balance in teams, be it with respect to size, well-defined and agreed upon roles, and contributed areas expertise. Having a well-thought-out plan, or framework, for an evaluation is another factor of success. The value of this plan- ning though, is nicely summed up by Vol- taire, “don’t let the perfect be the enemy of the good.” Ongoing socialization of an evaluation plan and allowing for moments for adaptation along the way should be embraced. With initiatives that serve to foster inno- vation, especially across geographic and other communities of interest, there can be a concern of imposing structure, which can come in the form of evaluation. Just as innovation depends on some arrange- ment for implementation, evaluators need to consider how findings will be combined in a coherent way. Finally, the pursuit of an all-inclusive, retrospective evaluation may be better replaced with an iterative strategy that al- lows for the staging of evaluative studies over the course of a major long-term ini- tiative like A GP for Me. In this way, the evaluation approach will naturally evolve with the initiative, and the latter stages of evaluation will yield findings that are in- formed by refined priorities, strategic di- rection and focus. During the final stages of the A GP for Me evaluation, the GPSC was looking for- ward and updating its strategic direction to include the patient medical home as the foundation of an integrated system of care. The provincial evaluation informed the evolution of the primary health care system in British Columbia, by providing important insights into local health care challenges; raising awareness of commu- nity-driven innovations that build on the important work of physicians, divisions of family practice, and health authorities; and furthering an environment for con- versation on how to continue to improve patient access and care. B ill R eid , Partner, Advisory Services, MNP and Past President, BC and Yukon Chapter, Canadian Evaluation Society, bill.reid@mnp.ca J asmina F atic , Manager, Advisory Ser- vices, MNP, and Past Vice President, BC and Yukon Chapter, Canadian Evalua- tion Society, jasmina.fatic@mnp.ca P etra L olic , Executive Coordinator, GPSC, Doctors of BC, plolic@doctor- sofbc.ca Evaluation