March 1998 ADRonline Monthly

Technology and the Action-Evaluation Research Initiative Project
by
Jay Rothman

Over the last few decades, various forms of innovative conflict resolution has been practiced around the world to reduce the negative dynamics of conflicts and to create positive change. Yet, in order for conflict resolution theories and practices to become more widely accepted and utilized, the field needs to develop ways to accurately and systematically define and evaluate the success of its efforts. Just as no two conflicts are identical, no two interventions are either. Thus, success must be contextually defined. One way to do this is to get all stakeholders in an intervention (e.g. participants, interveners, funders) to separately and interactively articulate their goals for a specific intervention. Since 1993, with Funding from the Pew Charitable Trusts, the Action-Evaluation Research Initiative has been developing a process called "Action-Evaluation" to systematically and contextually define, promote and assess success.

Action-Evaluation is a participatory goal setting, monitoring and assessment process that entails collaboratively articulating goals and objectives among the groups involved in a conflict intervention, including those funding it, those organizing and convening the intervention, and the participants themselves. An "action-evaluator" closely associated with specific projects collects this information from the groups and summarizes it with the help of a computerized database designed for the project to systematize the process and organize data within and across different projects for ongoing monitoring and comparative analysis.

This goal articulation takes place at the outset of an intervention, allowing the action-evaluator to track how goals of various stakeholders evolve, and use these goals as a basis for both designing the intervention and evaluating it along the way and at its conclusion. The collaborative nature of the goal-setting ensures that expectations are clear, and that the goals are realistic and suitable for the project at hand. With such clarity of goals, the conveners can design the project to fulfill those goals of various stakeholders and to develop the tools and flexibility needed to modify the design as the project evolves.

Dr. Jay Rothman, who directs this Project, and also teaches in the conflict resolution master's program at Antioch University, answers some questions for ADRonline Monthly

Q: Tell us about the technology you have developed to identify and measure conflict resolution goals.

A: Over the course of the two-year project, a database system has been built around a Filemaker Pro platform and linked to the Web by Lasso. This enables researchers around the world to gather data about the goals of those programs and projects they're working with on a three-tiered basis:

What is the project trying to accomplish
Why do these goals matter and
How will these goals be accomplished.

This information is gathered from all the relevant stakeholders - participants, conveners, intervenors, funders of projects and others. The researchers, who are part of a cyber-based research and intervention team, gather data, enter it into the web, and analyze it, with the input and guidance by the project director, as necessary. The technology doesn't replace face-to-face interaction but it helps us prepare for and follow up from such interactions in ways that are efficient, focused and substantive.

Q: How would you have done this preparation before the use of technology?

A: In some ways this use of technology is simply a more efficient way of doing what we normally do in the field of conflict resolution. We bring together people, sometimes individually and sometimes in groups, to determine what they are trying to do. We ask them to list their goals and we put these on flip charts. We also ask why these goals matter and how they will be accomplished, and then the walls are filled with flip charts. Or we interview them and transcribe hours of tapes. Sometimes we only get small segments of those stakeholder groups who are able to participate in those meetings or who are able to have time to be interviewed. This not only helps us to prepare more deeply, and more leisurely, but the process is also democratizing because more folks are able to put in their goals. In short, this is a process that can be used to enhance wide participation and elicitive program design, while various stakeholders are establishing their own visions of and standards for success.

With this technology, we gather these goals by giving all stakeholders access to questionnaires designed for their project. Most people can access the technology in their homes, or public libraries, or the convening institution, and answer the questionnaire in 30-45 minutes. So, it's relatively easy.

But in some countries there is limited access to this technology or there may be compatibility problems. In that case, the participants can send us the questionnaire by e-mail. If e-mail is not accessible, we can send the questionnaire by hard copy and they can return by mail or fax. In oral cultures, we may only gather data by interviews, which is then entered into the database by the researcher. Essentially, everyone can be included in answering the initial questions and we can get a wide baseline as we set goals for these projects.

Q: How do you analyze your data?

A: The data analysis is still labor-intensive. We summarize the raw individual data, pulling out key ideas and making it brief and manageable. This is the same thing we do in face-to-face meetings when we record people's comments on flip-charts, always trying to remain faithful to the participants' words and ideas while we capture them in concise ways.

We then print out summaries, line them up and identify shared goals, unique goals and, sometimes most interestingly, what goals seem to contrast.

Then we bring members of the group together to a meeting and present a goals analysis, to determine did we get it right? How do you want to determine shared goals? This is a process of conversation and negotiation determined by the participants. They may use consensus, or maybe majority vote; in any event it is a powerful process for group-building and clarity. They also determine what to do about contrasting goals: are they really contrasting and if so how can they address them so their differences become opportunities instead of barriers to effective team-work?

Q: Can you envision technology doing more?

A: I can envision using some smart processing tools to organize and analyze the data - like picking out key words, summarizing and matching. Perhaps the computer could create categories of goals based on recurring responses. Of course, this raises concern about losing individuality, which would need to be balanced against efficiency. Still we would maintain individuality in the face-to-face contact.

Q: How much time and money does this technology save?

A: This process assures that a full gathering of goals from a full range of stakeholders takes place, as well as a careful analysis of those goals. I think all this happens on an ad-hoc basis usually, not systematically or scientifically at all. We're seeking to contribute to the systematization of this field of conflict resolution through our process of action-evaluation.

In terms of savings, we are creating teams and projects that are efficient and effective. We're assisting good people in their efforts to produce good work. And in terms of the focus of this project, it helps conflict resolution projects trying to intervene in conflict, be clear about what they are trying to do, explicit about their goals and, in that way, be more efficient in accomplishing their goals, as well as, be more dynamic. This system also helps us monitor progress as the project evolves.

Q: It seems as though you are trying to infuse more science into what has traditionally been an art.

A: If we talk of science as being a process of systematization and replication and to some extent quantification, then I suppose so, especially if we think about replicability and the ability to compare and contrast within and across projects.

But, we are also allowing opportunity for more inclusion and participation, and thus promoting the core values of this field, while linking it to science and greater rigor.

A lot of people are beginning to explore the link between conflict resolution and technology. So far, this has been done in the spirit of sharing our expertise and knowledge, which is a very important trend. In that spirit, I invite colleagues to learn more about what we're doing and to share their work. In July, the project will hold a conference to share insights and conclusions and if any readers are interested in joining the conference or presenting parallel work, let us hear from you. Send comments and questions to Jay Rothman at jrothman@mcgregor.antioch.edu, and please visit our website http://www.ariassociates.org.