• About Us
  • The Commissioner of Official Languages
  • Mandate
  • Mission
  • Roles
  • History of the Office of the Commissioner
• Your Language Rights
  • Official Languages Act
  • Canadian Charter of Rights and Freedoms
  • Official Languages Regulations
  • Filing a Complaint
  • Court Remedy
  • Legal Interventions
• Official Languages and You
  • General Information
    • Frequently Asked Questions
  • Linguistic Duality and Cultural Diversity
    • Immigration and linguistic duality
    • Studies
  • Teachers
  • Parliamentarians
  • Institutions
• Newsroom
  • Search
  • News Releases
  • Daily Highlights
  • Backgrounders
  • Speeches
  • Media Resources
  • Contacts
• Publications
  • Search
  • Reports
  • Studies
  • Report Cards
  • Language Laws and Rights
  • Audits
  • Newsletter
  • History of Official Languages
  • Youth
  • Videos
  • Posters
  • Other publications
  • Order Form
• Statistics
• Careers
• Archives
• Links
  • Travel and Hospitality Expenses
• Completed Access to Information Requests
• Proactive Disclosure
  • Contracts Awarded
  • Grants and Contributions
  • Position Reclassifications
  • Advertising and Public Notices Expenditures

Indicators of Vitality and Its Evaluation

Page 21 of 36

It is commonplace nowadays to use indicators to measure the results of public or community initiatives, especially when there is a requirement of accountability. This certainly applies to OLMC vitality. However, since the concept of vitality is vague but quite broad, it needs to be understood in a manner that is as comprehensive as possible. That is why we favour evaluation, using indicators as measuring tools. Without neglecting the role of evaluation in accountability, it also needs to be seen as a method of empowerment for the communities.

Evaluation

Evaluation is an exercise through which a judgment can be made on the value of an action, a project or a situation; it thus calls into question the thing being examined. This means that a consensus must be reached between the parties involved in an evaluation, specifically with regard to 1) the values and criteria that will guide the judgment, 2) the things that will be judged and 3) the indicators that will serve as units of measurement.

Evaluation is of interest to the communities to the extent that they are involved in it from the outset, participate in defining the issues, contribute to the interpretation of the results, and are thus able to light their own way as they pursue their quest for development. Evaluation is an accountability tool, but it is also a tool of collective empowerment. It needs to be carried out using a variety of analytical perspectives (Johnson, 2003), and the parties involved must work together.

There is growing agreement about the inherent interest of evaluating the vitality of OLMCs, but there are doubts and fears about the way in which this is done. The doubts are related to the choice of the most appropriate indicators to reflect so complex a set of issues, either those that will most accurately reveal the state of vitality (result) or those that will best express the factors that act on vitality (process).

The fears of the communities hinge on the fact that indicators are often imposed on them by the funding agencies, without the communities knowing either what they mean or, most importantly, how the results will be interpreted. For their part, government officials who deal with the OLMCs are worried about whether the results of such evaluations will be attributed exclusively to their programs, knowing that other forces are also in play. An investigation conducted by the volunteer sector in Canada identified other obstacles to evaluation: "Both voluntary organizations and funders agreed that lack of internal capacity such as staff or time, lack of money, unclear funder expectations, and lack of skills and knowledge were the main barriers to evaluation for voluntary organizations" (Imagine Canada, 2005).

The American experience sheds some interesting light on these apprehensions. A study analysing the practices used in evaluating community development over a 15-year span in the United States suggests that common approaches fail in several respects (Auspos and Kubisch, 2004):

• The deadlines set for the evaluations are too short to take into account the results of community development. The start-up phase (planning, upgrading capabilities, etc.) typically takes three years or longer. Evaluations that focus on this period take into account neither the maturity of the initiative nor its long-term impact.
• The theories of change that guide community initiatives are weak: they often consist either of pious hopes, or are too complex to be understood by the people involved.
• The data gathered overwhelm the players, and collecting these data exhausts their resources. It is necessary to choose what is to be measured in light of clear theories of the changes desired.
• The specific character of community initiatives is often difficult to reconcile with the overall parameters used for evaluation, and this makes comparisons difficult. The communities must identify standards that enable them to compare their own circumstances to others.
• The evaluators are viewed as detached technicians who impose unreasonable demands for data on the communities, analyse them from afar and fail to report the results to the communities.
• The evaluations deal with the results anticipated by the programs of the funders and do not focus on the processes of empowerment that are of concern to the communities.

Although those involved are right to be critical of the evaluation of community vitality, it is nonetheless true that evaluation continues to arouse interest, and this is expressed specifically around the use of indicators.

Previous Page | Table of Contents | Next Page