agregar comentario

Tools

Making Collaboration Count: A Tool for Tracking and Building Participation in Community Collaboratives and Coalitions

by Ashley Anglin, Chris Michael Kirk, and Sharon Johnson-Hakim

Download PDF version to access the complete article, including Tales and Figures.

Introduction: Principles and Practices of Successful Coalitions/Collaboratives

Community collaboration and coalition development is a primary focus of the field of Community Psychology (Society for Community Research and Action, 2012). Community psychologists are familiar with the call to action to bring together diverse community residents to combine and align efforts and resources to meet mutual goals. They also recognize the fundamental values of inclusion/participation and respect for and attention to diversity that guide their work. With the growing popularity of coalitions and the need to assure that collaborations are effective and sustainable, it is now more important than ever to evaluate the work of these groups, to identify best practices, and to develop tools that strengthen the work of community coalitions and collaboratives.

The first step in this process is to clearly define “coalition.” According to Wolff (2001),

a coalition is composed of community members; it focuses mainly on local issues rather than national issues; it addresses community needs, building on community assets; it helps resolve community problems through collaboration; it is community-wide and has representatives from multiple sectors; it works on multiple issues; it is citizen influenced if not necessarily citizen driven; and it is a long term, not ad hoc, coalition.

Inherent in this comprehensive definition are several important questions that each coalition must ask of itself to take the next step in assessing overall coalition functioning. For example,

Who is this coalition comprised of? Community members? Outside agencies?
What is the focus of this coalition? What sectors are represented? Are the right people at the table? Can it address multiple issues?
Is there true collaboration? What does collaboration mean to the coalition? Is it avoiding tokenism (i.e., empty participation)? Do the processes in place encourage participation? Are there multiple ways for individuals/organizations to be involved?
Is the coalition building on community strengths? Is it leveraging existing relationships?
Is the coalition going to be around long-term? What will involvement look like over time? Does its membership create opportunities for meaningful involvement? How will the goals change over time?

These questions constitute the basis of an evaluation of coalition functioning. To assist in answering these vital questions, this article presents a unique tool for evaluating the basic functioning of a coalition, with particular emphasis on participation and engagement. The authors argue that coalition evaluation should include more than just an outcomes evaluation (i.e., assessing the impact of coalition action in the community); it should also include a process evaluation that can provide actionable, real-time feedback on the ways in which organizations are working together in a given coalition.

The Engagement Scorecard is a spreadsheet-based tool, which allows users to input member information (Who is the coalition comprised of?); keep track of opportunities for participation and member engagement (Do the processes in place encourage participation?); measure coalition diversity by sector, geography or other category (What is the focus of this coalition? Are the right players at the table?); assess participation over time (Is the coalition going to be around long-term?); and disseminate a score card to members, allowing them to target their outreach and build on existing relationships and resources (Is the coalition building on existing community strengths?). In addition to outlining the steps needed to implement this tool, a case example is provided illustrating how this tool has been utilized within the North Jersey Health Collaborative (NJHC).

The “Engagement Scorecard” and the North Jersey Health Collaborative

The NJHC is a charitable not for profit 501 (c)(3) corporation whose members are working to leverage resources and expertise to maximize the impact on the health of our communities. Founded in 2013, the mission of the NJHC is to “coordinate the efforts and resources of public health, healthcare, and other organizations to maximize our impact on the health status of our communities and minimize avoidable illness, injury, and hospitalization” (North Jersey Health Collaborative, 2015). The NJHC is member-supported and member-governed, combining both regional and local expertise and efforts. We define health broadly, and at the time of writing have 24 member organizations (paying with voting rights) and over 80 partner organizations (non-paying), including healthcare, social services, education, pharmaceutical, faith-based, and grassroots organizations as well as private businesses.

As the NJHC grew in its inaugural year, it became clear that we needed a way to not only keep track of who was involved (which was especially difficult for partners, who often stepped in and out of the process, or who may send different representatives at various times), but also to assure that there were enough opportunities for organizations to be involved. Though a partner may be interested when introduced to the NJHC, limited or tokenistic opportunities to engage may quickly diminish this interest. We also wanted to target our outreach, but found it difficult to assess who else needed to be at the table for the NJHC to be effective when we did not have a clear picture of who was already involved and the extent of their engagement.

From these all too common struggles, the Engagement Scorecard was created. This Scorecard allows us to keep track of member organizations, the number and type of opportunities we provide for member/partner engagement, our geographic reach, and the distribution of our members/partners across sectors. It also allows us to increase personal connections between members, create shared ownership between members to “get people in the room” and maintain engagement over time, and motivates us to create additional opportunities for engagement.

Using the following steps, this tool can be implemented in any collaborative/coalition, regardless of the size or scope. Specific examples of how the Engagement Scorecard has been implemented within the NJHC will be provided along with the steps for creating and using this tool. Readers can also click here to view a sample sheet, which can be copied and edited to meet specific needs.

Choose a platform: The first step is to decide what software/web tools will be used to track information. Both Excel and Google Sheets are good options, and using the steps below, users will be able to insert the formulas directly. There are other options, however, such as EditGrid and Numbers (Mac only). For the NJHC Scorecard, we use Google Sheets because it is free, accessible on multiple platforms, downloadable, and promotes transparency (we can share the link to view and/or edit with members/partners). The following instructions work best with Google Sheets. If any reader decides to use Excel or another platform, he or she can contact the lead author for additional instructions.
Input member/partner information: Next, users will need to input basic information about the collaborative/coalition members. This can be done at the individual or organizational level or both. For example, in the NJHC, there are many member organizations, most of which have more than one representative who participates in the collaborative. Therefore, each individual has their information in the Scorecard (name and e-mail address), but the overall score is calculated by organization, so there is also organizational information included (name of organization, county(ies) the organization serves, and sector (e.g., healthcare, education, faith community)). It works best to create separate cells for each type of information. If users will be calculating scores by location, for example, and some individuals/organizations work in more than one location, it is easier to create a column for each location and to insert a “1” if that individual/organization works in the locale (see Figure 1). Users decide which categories are relevant to each collaborative, whether geographic or otherwise.
Input engagement opportunities: A key component of the Scorecard involves keeping track of engagement opportunities (see Figure 2). Depending on the collaborative/coalition, this can include e-mail communications, regularly scheduled meetings (collaborative-wide and those for special committees/work groups), attendance at events, the number of community members/target population present, on-line surveys, focus groups, planning sessions, and contributions to the website/social media. Create a column for each event to the right of the individual/organization details, with a descriptive label and insert the date on the row below (see Figure 2).
Assign weight to the engagement opportunities: This is an important step as it is evident that attending a meeting or chairing a committee involves much more time and effort than simply opening an e-mail. The weight assigned to each engagement opportunity is up to the collaborative members. In the NJHC Scorecard, for example, opening and e-mail is worth 1 point, clicking a link in an e-mail is worth 2 points and attending a committee meeting is worth 10 points. For the NJHC, we also include bonus points for chairing committees or work groups. Users should insert the number of points assigned to each engagement opportunity below the date in each engagement column (see Figure 2). Note that the purpose is not to “grade” individual organizations, but to assess the function of the collaborative as a whole.
Input engagement data: For each individual/organization users track, input the number of points accrued for each engagement opportunity and place it in the corresponding row/column.
Calculate engagement scores overall: To calculate engagement scores, users need to first decide on three things: 1) time period, 2) level of calculation, and 3) maximum score. With the NJHC, we calculate engagement scores every 60 days (this is why the date field under each engagement opportunity is important). We calculate both individual and organizational scores (with individual scores feeding into organizational scores and organizational scores reported). For example, if Organization X had three people attend a meeting worth 10 points, that organization would accrue 30 points for that engagement opportunity. We keep track of individual scores in order to see exactly who is participating from each organization and so that we have their contact information on file. It also helps to know this information when communicating with leadership. We set the maximum score per time period at 100. This number is arbitrary, but a nice round number like 100 makes the charts easier to read/interpret (see Figure 3). For the NJHC, we also include a column for Active past 30 days. This helps us to see quickly whether or not an organization has been active lately. To create this column, create a new column to the right of the engagement score column, and insert the formula =IF (engagement score cell>0, “Yes”, “No”). For example, in Figure 3, the formula would be =IF(E5>0,”Yes”,”No”). Once you have typed in the formula in the first row of your data, you can simple copy the cell, “Paste Special”, and select “Formulas.” This will insert the correct formula for each following row (the row number will automatically change for each following row).
Calculate engagement scores by chosen grouping: In our experience, it is helpful to break down the engagement scores by group. For the NJHC, the groups important to us are County and Sector. However, groups could be broken down in other ways depending on what makes the most sense for another collaborative/coalition. To start this step, users will need to create new columns for each subgroup (e.g., for each county or each sector; see Figure 4). The idea is to generate an average score by subgroup (e.g., average scores for Morris County, Sussex County, etc.). To do so, users will need another formula that will pull the engagement scores of any individual/organization that falls within that subgrouping. For example, using Sectors, in step two, when I was entering individual/organization information, I made a cell for “Sector” and typed in the sector (from a pre-determined set of choices) directly into the cell (e.g., “Advocacy org,” “Pharma,” “Education,” “Fitness”). What the new formula will do is input the engagement score from the individual/organization into the new cell if and only if that individual/organization has that specific sector written in the column. The formula is as follows: =IF (the sector cell= “specific sector”, cell with that individual’s engagement score written as $column$row). In the example in Figure 4, the formula for the first row in the “Advocacy org” column is =IF (J5=”Advocacy Organization”, $E$5), where J5 is the cell in the “Sector” column and E5 is the cell with the engagement score for that individual/organization. If the formula finds “Advocacy org” in the sector column for that individual/organization, it will input the engagement score into the new cell. If not, it will input the word “FALSE.” With the NJHC, we also calculate engagement scores by county, but many organizations work in more than one county. In this case, the formula is slightly different. Looking at Figure 1, there is a column for each county, and if an organization works in Morris County and Union County, there is a 1 in each column. Therefore, when pulling the engagement scores by county, we aren’t looking for a specific phrase like we did with the sectors. Instead, we need the formula to input the engagement score from the individual/organization into the new cell if and only if that individual/organization has a “1” written in the cell for that county.
Create automatic charts: On a separate sheet in the same file, users can make charts that automatically update as they change/add to the data in the scorecard. This makes generating the Scorecard much easier. To start this step, first calculate average scores for each of the chosen groupings (see Figure 4). To do this, select the cell at the end of the first column, go to “Functions” and click “Average,” then highlight all of the data in that column (don’t worry about the “FALSE” entries; they are not calculated into the average). The average for that column should now show up in the selected cell. Do this for all groupings (see Figure 5).

Next, name the sheet with the data, by double-clicking on the sheet tab at the bottom. We use the name “engagement scores.” Then, open another sheet in the same file (at the bottom of the screen). On the new sheet select any cell and type in the formula =’name of your previous sheet’!the cell with the average score you are using. For example, in Figure 5, the average for Morris County (or cell AZ93) is 13. This is the number I want to show up on my second sheet in the table for average score by county (see Figure 6). Therefore, the formula for this cell is =’engagement scores’!AZ93. Follow the same steps for each of the average scores. Once users create a table for the average grouping scores (like the table shown in Figure 6 for engagement score by county), select the entire table and click “Insert” followed by “Charts.” Use the popup editor to format and select the chosen chart. We choose a bar graph to show average engagement by county and a pie chart to display engagement by sector.

Once satisfied with the charts, users are ready to create the final scorecard. First, download the charts (click the small box in the right-hand corner of the chart in Google Sheets, then select “save image.” Then use Word, Publisher, or other document-editing software to format the scorecard. Users can also create a standard template like the one shown in Figure 7. For the NJHC, we like to add other information to the Scorecard, such as website statistics (e.g., through Google Analytics), upcoming meetings, and Constant Contact data (e.g., records of who opens e-mails and clicks on embedded links). An example of a formatted scorecard can be seen in Figure 8.

Suggestions and Lessons Learned

It is important to recognize that promoting engagement and working collaboratively is always a work in progress that requires continual evaluation and modification. The NJHC went through several iterations of this tool before arriving at a format that works for us, and it took time to create a broader engagement strategy that complements this tool. It is good practice to work with members to agree upon the structure and format that would work best for any process metrics.
This tool can be adapted for other types and sizes of collaborative groups and can be modified to meet context-specific needs. For example, an Early Learning Center is Georgia is currently tailoring the Scorecard to evaluate the success of the center in engaging parents in their parenting program.
The Scorecard is a good jumping off point for meaningful conversations about the structure, function, and scope of the group, but the tool alone will not significantly increase participation. It is important that collaborative members engage in discussion around the Scorecard and opportunities for participation, ensure that it is disseminated widely, and make certain that members/partners feel a sense of personal responsibility for increasing engagement opportunities and getting the right people at the table. If they do not feel this responsibility, then the use of the Scorecard as a tool is limited. Supporting tools/strategies for use with the Scorecard include:
A collaborative website, which for the NJHC provides opportunities for online engagement (including surveys, contests, and forums); tracking over 150 health indicators; communicating meeting times and agendas; and highlighting topic areas, best practices, and other resources (visit www.njhealthmatters.org).
Use of Constant Contact (www.constantcontact.com) or other e-mail tools such as Mail Chimp (www.mailchimp.com) to track e-mail engagement (e.g., who opens e-mail, clicks on links, or registers for an event).
Focusing the process evaluation on both the amount and type of opportunities provided by the collaborative and the levels of member engagement. (e.g. evaluating the coalition at multiple ecological levels).
Actively encouraging members and leaders to use the Scorecard to address who else needs to be at the table and to reach out to their contacts.
Use of the Scorecard to promote connections between members based on chosen sub-groupings (e.g., county, topic area, sector).
Disseminating the Scorecard on a regular basis and highlighting month-to-month changes during in-person meetings.
Setting up and maintaining the Scorecard takes time. For the NJHC, it worked best to have a designated team of people, each working in three areas: 1) keeping track of who participates in each opportunity, 2) inputting the data into the scorecard and filling out the template, and 3) disseminating the scorecard and encouraging others to view the Google Sheet and subgroup lists (in our case these are county lists) to direct outreach to potential members and explore new opportunities for engagement.
New engagement opportunities and scores should be added to the form on a regular basis, if possible, to avoid a lag in information over time. We found it helpful to establish a process and timeline for when new information should be entered. This also allows for quarterly summaries and for tracking the ebb and flow of participation over time.
It is important to avoid placing blame or requiring more than members can provide. The capacity of organizations/individuals to engage must be considered. The Scorecard can generate some healthy competition between groups (e.g., one county working hard to “beat” another county next month), but members need to be sensitive to the fact that too much competition or competition that places blame on some individuals/organizations is not good for those individuals/organizations or for the collaborative as a whole. Some may also feel uncomfortable with the “surveillance” suggested by tracking participation. This is why discussion must happen early on to generate mutual goals for the use of the Scorecard and to assure that this information is communicated to all members/partners. There is also the option to keep individual names/organizations anonymous and to only present the information in aggregate to the larger group. This, however, erases some of the transparency, so it is important to weigh the pros and cons of each strategy. In the case of the NJHC, we have decided to compromise by providing aggregate data (like in Figure 5) to all members/partners, and the full data to the Executive Committee only.
Finally, we have learned that funders appreciate the information that the Scorecard provides. Those who have received grant funding know that funders often require detailed evaluations of programs and interventions. However, not many funders are aware of the structure and processes that go into community collaboration. From our experience, funders are very interested in the Scorecard and perceive it as another way to track the effectiveness of their funding and appreciate a glimpse into how the collaborative functions.

Summary

Due to the nature of the work, most community psychologists spend time working collaboratively with multiple individuals and/or organizations. There has been much written in the field regarding the importance of collaboration and the hallmarks of successful collaboratives/coalitions. However, there has been less discussion on the ways in which coalitions/collaboratives themselves are evaluated and how this evaluation can be used to promote engagement, continuous quality improvement, and give the coalition itself feedback on its functioning in order to give it the best chance to achieve its goals. This article describes a tool that can be used and modified to evaluate levels of participation over time, encourage engagement, create join ownership for collaborative processes, and track and expand upon the opportunities for engagement that the group provides to its members/partners. This tool has been used with success for a multi-member group called the North Jersey Health Collaborative and has proven to promote engagement and help NJHC members to avoid the common pitfalls of community collaboration.

References

North Jersey Health Collaborative. (2015). About Us. Retrieved from http://www.njhealthmatters.org/index.php?module=htmlpages&func=display&pid=1

Society for Community Research and Action. (2012). Competencies for community psychology practice: Draft August 15, 2012. The Community Psychologist, 45(4), 7-14.

Wolff, T. (2001). Community coalition building—contemporary practice and research: Introduction. American Journal of Community Psychology, 29(2), 165.