How to monitor and evaluate collective impact?

We did some work with the Public Health Agency of Canada on monitoring the key elements of vested partnerships or networks that affect system change. That is, how to monitor what the network can do that is greater than the sum of the individual partners? The work was based on a review and assessment of PHAC-Innovation strategy’s work in food systems and mental health initiatives. We found that live vested health partnerships and networks have:

  • diverse partners

  • a sectoral agenda

  • partner alignment

  • pooling of assets (financial and talent, extended networks)

A live network assumes strength in diversity ad governance of the partnership as well as demonstrating collaborative systems change. I tis important to monitor and measure not only the partners but the synergy and collective impact of the partnership. Outcome mapping can render pathways and assumptions explicit for partner deliberation around their common agenda.

Sophisticated partnerships and networks act like a murmuration of starlings, acting in concert to push learning, change practices and affect policy and societal norms. Capturing the movement of the flock is as important as counting the birds.

The changing landscape of organizing and learning online

The following is an excerpt of work that we did for Association for Women’s Rights in Development (AWID) for their membership strategy. Helpful to think about how to reclaim online space for critical learning and organizing.

———————-

Increasingly, organizing is finding creative tactics online to reinforce face to face mobilisation. Power and gender dynamics, of course, exist in these spaces too so it is important to enter these critically. Feminist and critical writers help others to navigate these spaces with intention. Aristea Fotopolou (2016, p. 360) talks about the importance of online feminist space “embodying activist practice” as important for both challenging current power structures and negotiating our identities within and across communities. She urges us not to fall into the trap of replicating the fast-based business cultures online. Digital spaces and social media platforms are not the only spaces where we dialogue and learn. It is where we represent ourselves, belong, labour, build and negotiate within and across communities. She reminds us what it means to enact feminist and queer activism in an accelerated digital age when culture, economy and politics are so intertwined and there is such a diversity of feminisms. 

How to navigate these tensions, spaces and communities while understanding that people learning, organize and network differently now because of these spaces? Movement organizations such as Mama Cash, Movement Strategy Center, Move to end Vioelnce, Greenpeace have adapted their way of interacting with members to take into account the fluidity with which we now move. This is not to dilhute campaign messages or learning but to acknowledge that online spaces between convenings tend to be most impactful as microlearnings that are provocative, inspiring and relevant yet minimal. Strong movements and alliances account for the fact that their members are fluidly moving across many communities, in and out, deep and skimming, depending on relevance and urgency. We are often, by default, part of many communities with lots of flexibility in how to engage. 

It is also increasingly possible to trace and make sense of online organizing in order to improve and deepen organizing opportunities within these spaces. Some movements have explored how they negotiate and frame campaign issues including how online spaces helps to frame shifts in thinking and narratives (McAdam et al, 2004). Hollaback! an online platform to crowd-source stories of harassment and gender-based violence uses this space to help shape gender narratives, campaign strategies and for women to feel less alone in their struggles. Similar online spaces in the Nordic context share gender-based insults and harassment using ”shamelessness as a feminist tactic of resitance.(Sundéna & Paasonen, 2018). HarrasMap in Egypt has found intersectional differences in perspectives and also found that anonymity helped uncover more severe forms of sexual harassment. 

Feminist activism that takes place in face to face ways will always be used to advance movement and cross-movement agendas. It is critical that offline organizing and cross-movement building is harmonized and each type of space leveraged for what it can do. Josephson et al. (2017) describe the powerful role that face to face activism has in mobilizing protests and calling attention to injustice. Newsom and Lengel (2012) found the ‘Arab Spring’ a joint operation of online awareness-raising and offline action. Both are, of course, critical. While women were integral to the face-to-face organizing of the uprising they have not really been recognized as part of the mass movement. 

For a global cross-movement organization like AWID, it is helpful to consider how issues and voices of different feminist movements, both global and local, are harmonized. Contemporary feminist movements have shifted to a more diverse cross-movement that integrates international standards and ideologies while establishing local precedence. Newsom and Lengel (2012) found that online helps women and trans people bring their different backgrounds and locales to their movements as well as be part of broader organizing. How AWID balances emphasis on their own strategic priorities versus amplifying the diversity of issues and campaigns arising from movement partners is a critical consideration for cross movement-building and co-creation. 

 

Networking and Learning Online

Critical reflection exercises

Critical reflection exercises

A key element of transformative learning is experimenting with real-world problems. David Kolb’s experiential learning cycle continues to be pivotal. Yet cycles, even spirals, don’t capture the dynamism (and dangers) now available with online and blended learning. Informal means of learning such as podcasts and viral social media posts abound. On one hand, frames have never had more possibility to engage with such a diversity of experiences, bodies, identities, and communities. How to bring feminist and critical pedagogy to these spaces? This paper works to critically frustrate and extend the learning cycle with a hologram which helps to capture online multi-dimensionality as well as distortion. 

Online there is both performance and projection. Holographic visualization offers multiple dimensions, movement, refraction, frustration, doubt. But online learners can be supported to move more consciously to meaning and their own deliberations while embedded in life, work, negotiations. This messiness and muddling is key to the transformative potential of online deliberations. Three key conditions seem to facilitate more transformative learning in online spaces and platforms:

  • allow multiple paths

  • piece the learning well into micro-learnings

  • embrace doubt and contested realities

Online spaces can be shallow and harmful. However, they also have potential to collapse gender and power dynamics in interesting ways as learners shape their own reflective paths at their own pace. These gendered power dynamics are not removed but can be fore-fronted in the learning. The lag between experimentation and insight can be used. Facilitators and peers can act more like coaches, real-time in real messiness. 

Sisters Ink has worked with AWID (Association for Women’s Rights in Development), the University of New Hampshire and the Coady International Institute on blended networking and learning strategies. Nanci presented a paper at the International Transformative Learning Conference, Columbia University on the experience with the University of New Hampshire “Frustrating the Experiential Learning Cycle.” Write us for a copy.


Feminist and Gender-transformative research

Screen Shot 2019-03-28 at 1.01.46 PM.png

Source: This work above represents the work of Harassmap. http://harassmap.org/en/ The diagram is adapted from Rao & Kelleher’s gender work (2005) with additions from the power and gender work by Cornwall (2014), Batliwala & Pittman (2010), Gaventa (2006), Veneklasen & Miller (2002) and Kabeer (1999).

******

Sisters Ink did a review of IDRC’s research for the last ten years to support dialogue and understanding of what makes gender-transformative research. Having rated 42 projects, 6 case-studies of IDRC research were selected for more in-depth analysis of what make gender-transformative research. These include research that:

-       Introduced fish processing and gender relations through participatory dialogue in Zambia and Malawi

-       Influenced gender narratives and power relations and policies in small scale and artisanal mining in Central and East Africa  

-       Enhanced Indigenous engagement, intercultural health practices and policy in Peru

-       Supported dialogue between women, their communities and local governments for more accountable and gender-equitable settlement services and water and sanitation infrastructure in India

-       Provided a digital crowd-sourcing platform for reporting gender-based violence, raising awareness and changing norms in Egypt

-       Empowered girls, youth clubs to dialogue negotiate with others in their community around early marriage in West Africa

Though the cases had very different contexts, gendered issues, influence strategies and actors involved in the research, there were some common principles.

Gender-transformative research: 

·      Addressed root causes. Intersectionality, how gender intersects with other aspects of social identity, is key to understanding root causes. 

·      Engaged systemic partners.  Actors and partners engaged were a fit for the system and context being changed. Influence is situated. In some contexts, the best partner was a women’s movement organization. In another a mix of stakeholders including elders were involved. In others, media and social media were the strongest influence. 

·      Embedded and embodied the change in gender relations locally. Even in the time-frame of funding early influence is possible. Some focused on building local research capacity or negotiating power. Some on beginning early dialogue across different actors. 

·      Revealed that structural gender analysis is a muscle. A diverse mix of capacities is needed for researchers and IDRC staff to practice. Structural gender analysis is key of course. Other important skills include: deliberative dialogue, brokering effective partnerships, systems analysis, conflict mediation, and positioning evidence for use. 

Transforming gender relations is not just an issue of inclusion or individual agency, not even of good structural gender analysis. Research can play a role in this process where it is a much deeper situating of evidence in ways that influence underlying norms and institutions, and power relations that perpetuate unequal access, agency and control. Gender-transformative research goes beyond access and technical issues, and even goes beyond research papers to embed and embody the change being sought through effectives processes. Even in a shorter time-frame this embeddedness, influence and engagement of key actors is not only a way to redress inequities through the research. It is an investment in longer term changes in perceptions, norms, as well as institutional changes and uptake in policies, laws, budgets and markets. 

Changing the Gender Narrative

Screen Shot 2019-03-28 at 12.56.13 PM.png

I worked with The Story Kitchen and the Coady Institute in Nepal to co-facilitate with an inspiring leader, Jaya ji Luintel. We connect on the power of the Rao/Kelleher framework to ground our work in analysis of gender and power dynamics. Power analysis is overlaid [and we'll be back with a longer piece on that topic]. 

The work of The Story Kitchen (TSK) demonstrates what is possible in using change in one area to leverage it in others. They are "driven by the passionate belief that upholding the stories of women can unravel systems of gender oppression and patriarchy that continue to exist in Nepal. The domination of stories from men's perspectives leaves women out of the history of Nepal and fails to recognize the extent to which women are currently contributing and have always contributed to the development of the country." Aside from supporting stories of survivors of armed conflict from women of different castes, ages, communities and cultures, they train justice reporters and influence media in changing gender narratives. The survivors have now formed a network that has managed to begin discussions with the Truth and Reconciliation Commission and the President around transitional justice. Storytelling, in this case, is healing, awareness raising, changing the gender narrative and an advocacy tool. All of these pieces are critical and reinforcing.   

Like SEWA Bank taught me years ago, StoryKitchen reinforces so many of the key lessons in addressing gender and power dynamics. All realms of change and influence are gendered and intersectional. We need healing and connecting not just "agency" because marginalization from the norms and the ensuing violence is so internalized. Formal strategies, particularly organizational, are not enough. Media, social media, artists, elders have a critical role to play in the informal work on gendered social norms and exclusions. Networks and movements have the potential to affect both formal and informal systemic change. 

http://thestorykitchen.org

Yours ever in the struggle to understand change and influence, 

Nanci

Researching women's economic empowerment

pen-idea-bulb-paper.jpg

Sisters Ink has just been awarded a contract to support IDRC with an internal learning evaluation on the research funded to support youth and women’s economic empowerment. Nanci led a Mid-Term Evaluation for part of this program, GrOW. There were 14 research partnerships between North-South, South-South research agencies and universities. 

Highly rated research partnerships (quality; building local capacity; positioning for policy influence) had strong coherence between qualitative and quantitative methods, clarity on policy uptake and good governance marked by complementarity of partners, a mix of scientific rigour and context knowledge with at least one partner having a track-record for evidence-based policy research. Smaller mixed North-South partnerships with two partner institutions both academe and practitioner as well as academic North-South partnerships demonstrated stronger early results than larger consortiums and research projects only based in the South. 

What is the role of research in redressing power dynamics? The program demonstrates that the three outcomes (quality research; building local capacity; policy influence) can be mutually reinforcing but cannot be assumed. 

Reducing vulnerability in HIV vulnerable youth, orphans and vulnerable children

pexels-photo-94990.jpeg

Sisters Ink did a contract for FHI360 to review evidence related to evaluations and research on financial education for HIV vulnerable youth, orphans and vulnerable children. Evidence was limited in both scope and strength. However, a combined effect was found on HIV-related outcomes particularly programs integrating financial education with sexual and reproductive health supports. 

Not nearly enough is known about the various program and contextual “levers” that led to improved economic or health outcomes. Sometimes they are related. For example, we know that peer and trusted adult counselling can help. However, we don’t know how community-based programs compare to schools, for example. We need more studies to better understand the role that “self-efficacy” or agency plays in linking economic strengthening supports with asset building. 

image1.png