Great Feedback! Online Engagement in New Zealand
From ParticipatioNZ
< Guide to Online Participation | 4. Resources
Tom wants feedback
Tom is retired, lives in Wellington, and is committed to improving his community. He's been doing it for years as a scout leader, a Lions Club member, a father and a scientist. He talks about demonstrating during the Springboks Tour of 1981 -- "I always stayed away from the people wearing helmets". All of this work was worthwhile, he says. But were there any that stood out from the others?
Thinking about it, Tom says that "the local things are the most satisfying". He explains with a story: one of the main things Tom's Lions Club does is raise funds for the local community. Recently, they helped send a young person overseas with a scholarship on the condition that, when they returned, they give a short talk about their experience. Tom says: “It’s very satisfying to hear them say, ‘I had a whale of a time, and I couldn’t have done it without you’.” Almost everything else amounts to writing cheques -- good intentions and resources that could and should have an impact, though it's not always obvious what kind.
Tom makes it clear that these sorts of efforts, particularly on national issues, are absolutely worth doing. It's just that they're not as satisfying as those things that are local -- where there's an immediacy to the results you get for the effort you put in; where there's a good feedback loop between what you do and the way the world changes around you.
Purpose of this paper
Tom's story is not unique. Ask others about what makes participation satisfying, and their sense of the importance of feedback -- either in terms of hearing what other people think of their ideas, or their contribution to an overall result -- is common.
But if agencies embarking on online participation projects wish to respond to this desire, how can they best go about it?
The purpose of this paper is to create solid working ideas for managing feedback as agencies look to engage New Zealanders in improving services, policies and outcomes.
This paper is an evidence base for the strategy and management sections of the Guide to Online Participation, and a response to a call from members of the Online Participation Community of Practice for New Zealand case studies in this area.
Its findings are based on:
- a literature review, called “Understanding Feedback”, found on the ParticipationNZ wiki here: Understanding Feedback;
- case studies on current agency practices in the area of online participation -- themselves based on study of relevant documents and websites as well as interviews with practitioners
- attempts to work with practitioners to spot opportunities to improve their project and processes.
What the paper says
- Feedback is a critical part of the experience of participation.
- Agency studies show some key features of good practice in improving participation and feedback, so that people and agencies can achieve their goals.
- It proposes a New Zealand model for online participation based on these good practices.
- It suggests strategies and prototypes for improving feedback loops.
- It suggests a future service agenda for State Services Commission to consider.
The problem with feedback
When people use the word ‘feedback’, what do they mean? Typically, they understand feedback as information communicated from one person in return to another -- as in a conversation. When they talk about feedback loops, people are usually describing the construction and design of the information exchange, or the ways and means that the interaction happens.
In short, feedback is what you get when you interact. Feedback loops describe how the interaction happens.
These ideas of feedback and feedback loops are useful for public participation practioners because they help to simplify and describe the various components of a participatory process.
- The purpose of almost any participatory process is to gather feedback of some kind from the public on policy and/or services.
- The overall process of participation can be understood as a feedback loop between citizens and government, initiated by either government or citizens.
- Citizens in discussion with one another generate another, more specific, set of feedback loops.
- The processes agencies need to undertake to respond to citizens' ideas, concerns or criticisms close the overall feedback loop.
Diagram to illustrate the overall feedback loop
Unfortunately, traditional government efforts to incorporate citizen participation struggle with how to get feedback and how to build effective feedback loops. Often, they are unsure about the following things.
- How to ask compelling questions of the public in a way that adds value for decision making (i.e. to start the feedback loop, from the agency to the public, or to clarify what the public is saying to them if the public initiates the conversation).
- Which members of the public to trust and/or involve, and how to start and maintain informed, deliberative discussions that people will want to be a part of (i.e. to be heard and to hear the right voices at the public's end of the loop).
- How to manage people's expectations about what will happen with their input (i.e. to establish how the return loop will impact on the agency).
- How to demonstrate the impact people have had on the decision making process and/or outcomes that result from decisions (i.e. how to close the loop back again with the public).
From the perspective of the people who government seeks to involve or consult, there are the following related problems.
- Issues that the government wants to invite public participation on have a low profile or no profile, so it is difficult to know where and when to get involved even if a person is willing (i.e. to start the feedback loop).
- Information supplied by government to inform people's contribution is full of jargon or far too long to consume given the time pressures of day-to-day life (i.e. to facilitate contributions from people at their end of the loop).
- People are interested in interacting with one another and learning new things by discussing and debating, but the 'submission' model used in traditional consultations and select committee processes works against this desire. It only allows one person or group to speak at a time in isolation from others (i.e. to facilitate contributions from people at their end of the loop).
- People don't know what will happen with their input if it is given. They provide input on an assumption that it will be acted on and are disappointed when they can't perceive any reaction (i.e. other than perhaps an acknowledgement).
Virtuous feedback loops are ones that satisfy these concerns. Creating virtuous feedback loops means aligning people, processes and systems to ensure the goals of agencies and the goals of people are fairly met.
Feedback in practice: case studies
But how to do this? New Zealand agencies are grappling with these problems. Those involved with the Online Participation Community of Practice's seven case studies of online engagement show the specific challenges and practices that already exist in New Zealand to create better feedback loops between people and government. The case studies are:
Innovations in improving communication between New Zealanders and executive government through hypertext, video and social media.
The processes and practices used to engage New Zealanders in deliberative dialogue about the complexities of the social, cultural and spiritual aspects of biotechnology.
A Ministry of Transport led pre-policy stakeholder engagement programme, and the evolving online community that was one of its results.
How an online panel of 3000 members challenges and changes an organisation.
How opportunities to participate are presented on one of the New Zealand Government’s most significant websites.
A website and ‘participation database’, that aims to help businesses and agencies across government connect on consultations.
How research, blogging, wikis and a community of practitioners developed the State Sector Guide to Online Participation.
The details of these case studies are available via this link. For the purposes of this paper, we will focus on what we can learn from these cases overall.
The findings fall into two broad categories:
- engagement patterns, that describe the approaches they use to attract and involve people in their participation process
- operational patterns, that describe the approaches agencies use internally.
Engagement patterns
Each case study signalled several good practices for engaging the public in discussing public policy and public services.
Focused questions, designed collaboratively
Questions people do not understand can never be answered. And asking questions that have no relevance to agency concerns and priorities will never have value.
The Couch team insists on clear questions, in plain language that Couch members can quickly understand and answer. The Safe As team spent weeks working through the key questions that they would ask the public with management, Ministers and members of the National Road Safety Committee. They settled on three, but had to quickly adjust when the road safety co-ordinators used to run the workshops felt that the public might think the engagement team was predetermining the outcome.
All teams use a testing system to ensure the clarity and quality of their questions.
The Bioethics Council sets its own agenda, and spends a good deal of time considering the sorts of issues on which it can usefully involve the public. It has developed some criteria for how it chooses.
Elements include:
- public consciousness
- global problems
- controversy
- effects
- Treaty of Waitangi
- research
- government advice
- tractability
- what others are doing
- legislative vacuum.
The Council also spends a good deal of time ‘framing’ issues for its processes to ensure the questions are the right ones and that the public can respond to them.
A process the Bioethics Council uses for producing good framing material includes:
- literature reviews and environmental scanning of academic and news sources
- interviews with experts or stakeholders
- draft road testing with a small sample of experts and non-experts
- consistent revisions in light of feedback.
In addition, as of July 2007, the Council is about to engage with members of the public to frame an issue (pre-birth testing of human embryos and foetuses) in public terms for further public deliberation.
Compelling content
Involvement in policy development or service design projects demands that particpants spend some of their precious personal time. Case study agencies are determined to ensure that people find compelling content that motivates them to engage.
The Bioethics Council makes every effort to ensure the content it uses, to engage and support people in discussing issues raised by biotechnology, is readable and interesting. It works to summarise issues, rather than explain complicated details. Its reports use a narrative style and quotes from dialogue participants to enliven the text.
The Beehive website is making use of video to get its message out. It uses film production techniques such as storyboarding and script writing to ensure its videos are professional and communicate clearly.
The Couch ensures its questionnaire results are easy on the eyes. Graphs are clear and easy to understand. They show meaningful differences between members. Colours are well chosen and attractive.
Building awareness
No one will get involved with anything if they never hear about it. Agencies in our case studies show that significant effort must be put into raising awareness of an initiative if it is to succeed.
The Couch has had a strong marketing focus, which its project manager believes has been a major contributor to the site’s strong membership numbers. The Couch team has been present at many community events around the country, and has engaged families through 'Fun Family Photos', where families have their picture taken with a fun background, and can download them from the Couch website. They also hand out 'goodies' so people can take away branded items like packets of jellybeans. Moreover, the Couch team has literally taken a red couch (the logo of the site) to various family events to attract interest and sign-ups.
The Bioethics Council procures services from advertising and communication companies to ensure their processes receive media coverage and public notice. The Council also works with community organisations and other stakeholders to help recruit participants for its processes.
Similarly, the Safe As project ran advertisements in local community newspapers, and paid road safety co-ordinators to run the public forums, because of their local knowledge and links with the community.
The Beehive website uses of the social video sharing website, YouTube, to promote its videos, having opened up a ‘Government Youtube Channel’ at http://www.youtube.com/nzgovt.
One struggle for the Business Consultation website is its visibility. Not many agencies use the site and it would like to be able to increase its number of sign-ups. On the government portal, the section on current consultation is buried beneath multiple ‘clicks’.
Brickbats before bouquets
Community takes time to build. People are not inclined to trust government agencies and will often take the opportunity to vent anger and hurt at government before they engage with an issue fully. The Safe As project and the Bioethics Council have used strategies to cope with such sentiments.
At first, the Safe As online forum was full of criticism of the National Road Safety Committee agencies. The Safe As team regarded this input as a useful expression of people’s discontent with current road safety policy. Complaints and negativity were, in short, useful data.
As people got their complaints off of their chests, and public servants proved themselves to be worth talking to, comments became more constructive, if not positive. The online forum has evolved its own norms, and now maintains a respectful -- though constructively critical -- relationship with the Ministry of Transport.
The Bioethics Council runs many processes on complicated and sensitive issues. It focuses on dialogue as a means of achieving productive discussion on difficult topics. The Council understands dialogue as a conversation -- rather than a debate -- in which people who have different beliefs and perspectives seek to develop mutual understanding. A typical result from a dialogue process of framing the issue, holding public discussion and deliberation, and issuing a final report, results in:
- softening of stereotypes
- development of more trusting relationships
- fresh perspectives
- new possibilities for interaction.
The Council makes every effort to ensure positions do not harden in the course of its dialogue processes. Online and face-to-face facilitation and moderation are a key strategy for encouraging people to work with points of view different from their own. The Council has hosted online discussions where positions were intractable, and despite the best efforts of a moderator, have failed to evolve.
Representativeness
Projects that generate significant public policy recommendations (The Couch, the Bioethics Council and Safe As) work to ensure they have an appropriate balance of men, women, and ethnic diversity in their participant groups. In terms of numbers, each agency has engaged hundreds of people in its processes. Key documents, such as final reports, make explicit numbers and kinds of participants so that reviewers can judge the possible bias of the groups involved.
Radical transparency
Being able to show clearly how the input gathered from the public is summarised, used and impacts on the decision making process, helps promote trust in online participation projects. Using the referencing power of the Web to make transparent where ideas have come from, or are further substantiated, also has an impressive effect.
The Safe As project was remarkably transparent in its approach. To start, it made the notes from national face-to-face sessions available online. When it had completed the summary of the entire process -- including the interaction in the online forum -- each individual who had left their address at face-to-face sessions (more than 1000) was sent a copy of the summary report and encouraged to respond. After responding to comments from the group, the revised report was delivered to ministers in exactly the form it was last presented to participants.
The Bioethics Council’s report on xenotransplantation is an excellent example of a 'high-fidelity', transparent discussion summary. It creates a narrative around what was heard, but also has participant quotations in a continuous right hand column that follows along with the general summary. Creative design helps make the relationship between 'what was heard' and 'what was said' clear.
Couch members have told the Families Commission that they want to know how information gathered from The Couch is being put to use. The Couch team is working to ensure that new polls make a direct contribution to the ongoing policy work of the Families Commission, so that Couch members can see a direct connection between their input and the output of the Commission.
The Beehive site uses hyperlinks within speeches to make transparent and credible the messages it tries to convey. This transparency invites scrutiny, suggests that the agency is confident in what it says, and ultimately builds trust because agencies can make clear they are prepared to back up their words with facts.
Action
The ultimate goal of any participation process is action that delivers tangible benefit to New Zealand. The following two outcomes came from the Safe As project.
- A Road Safety Policy Statement was launched by the Minister of Transport, Hon Annette King, and Transport Safety Minister Hon Harry Duynhoven, which reflected and incorporated many of the suggestions provided in the workshop and online (eg a new approach to the demerits point system).
- Road Safety Education Strategic Framework was developed in response to the 'stand out issue' of the engagement process -- the desire for more information and education about road safety issues. The framework "explains the current road safety education context and provides a model for an aligned, co-ordinated and collaborative effort in addressing road safety education and promotion, and deciding with and through whom, to achieve better road safety outcomes".
The Safe As team reported that participants were pleased with the special mention their input received in the ministers’ announcement of these policies. For other projects, the direct linkage from participation to action on the part of the Government is less clear cut. There is a correlation between a Couch-supported Families Commission consultation on out of school services, and an announcement of funding for those services by the Ministry of Social Development.
Operational patterns
Business processes change
We heard that when agencies open up to engage people online, the way they do their work changes. For example:
- At the Families Commission, a new in-house editorial team is being set up to ensure that polls on the Couch have clear objectives that match how the Commission will use the information for its various purposes.
- At the Ministry of Transport, engagement has been integrated with the road safety work programme:
- Various work streams inside the Ministry have been determined as a result of public input into the national road safety strategy. On a day-to-day basis, individual Ministry of Transport programme managers responsible for each work stream manage the corresponding work stream area on the comments board. For example, the project manager responsible for speed management is responsible for the speed management section of the comments board; the alcohol and drugs project manager is responsible for the alcohol and drugs section of the comments board. This approach promotes clear and effective channels for interaction.
- Some ministerial letters refer people asking questions about road safety to the online community.
- The Beehive is working with audio/visual media in its communication. It has adopted film production processes and technology to ensure that its outputs are timely, compelling and professional. A simple example of this is logistical: when feasible, press secretaries are now carrying tripods, video cameras and lapel microphones to speeches to capture video and audio of speeches. These videos must then be cut and reformatted for Web consumption, and promoted once they are launched.
Critical infrastructure
Web communications are evolving rapidly. Broadband access is a necessity for making use of the most dynamic applications that can enhance people’s participation in government. Audio/visual equipment including cameras, tripods, microphones, video editing and graphic production software are needed to make the best use of the Web.
In addition, the Government Portal team indicates that agencies are not making good use of automatic Web feeds (such as RSS or Atom). Well used Web feeds would allow agencies to automatically move content about public participation across the Web and into relevant contexts for people, such as their email inbox, Web feed readers like Bloglines or Google Reader, personalised homepages on iGoogle, MyYahoo or Netvibes, or onto other websites including blogs, Facebook or MySpace profiles.
This lack of content syndication is a major barrier to effective online participation. It significantly inhibits people’s ability to be made aware of the latest consultations, and prevents agencies from connecting their work with pre-existing Web communities. Agencies will not be able to make full use of the Web’s growing potential without making good use of Web feeds.
The need for a multidisciplinary team
All these case studies in online participation underscore the need for a mix of skills to be successful. These include:
- research and policy analysis
- marketing and public relations
- communications (including multimedia, graphic design and superb writing skills)
- community development
- facilitation
- evaluation.
The Bioethics Council has engaged writers, communications and advertising professionals, expert facilitators and community organisations to run its processes. The Families Commission has people with similar backgrounds on its team and combines their skills with its policy and research branch. The Ministry of Transport worked with its governmental partners on the National Road Safety Committee, community groups and others to engage the public in road safety issues. It also commissioned a formal evaluation of its stakeholder engagement process.
Ministers and senior management tend to focus on managing risk
We heard that online engagement projects are subject to usual risk assessments around legal, security, financial and project management concerns. Ministers and managers tend to have a special focus on risk to an agency’s reputation.
Key concerns for them include the following:
- Ensuring online engagement does not lead to direct challenges to public statements made by senior leadership.
The Couch did not run an online poll on Section 59, the “anti-smacking” bill, since the Families Commissioners had already issued public statements on their position. The State Services Commission (SSC) was in the early stages of creating policy on online participation, and used its Public Address post to solicit input into development. It was careful not to pre-empt SSC in any way.
- Keeping public servants safe from personal attacks.
The Safe As Road Safety engagement process included the development of a ‘rules of engagement’ for public servants in online forums. The essential advice was that public servants should not directly confront members of the online community. They should intervene only when non-factual ideas and information start becoming ‘accepted wisdom’ in the online community, or they are asked direct questions about facts and information. To date, this has largely kept them safe from attack and in a trusted position in the online forum.
The State Services Commission’s post to the Public Address blog empowered the post author to interact with Public Address readers in the comments. The Public Address community had already adopted norms of respect in its forums, and so was seen as a basically safe space for interaction.
- Preventing online engagement from turning into a wholly negative free-for-all, assaulting the agency’s reputation.
The SafeAs Road Safety online forum began with most contributors being negative. However, as their interactions with public servants have proved consistently positive and their understanding of the role of the site has grown, people’s contributions have become more positive.
The Bioethics Council spends a great deal of time ‘framing’ discussions on issues like pre-birth genetic testing, so that people can begin from a common understanding of the facts, values and ideas at play in the dialogue. The Council carefully scopes questions so that discussions on sensitive issues do not roam so widely as to be of no use. Moreover, it employs moderators and facilitators to keep discussions on track.
The Beehive website is considering the possibility of creating weblogs for Ministers. However, the blogs will not have an open comments thread. The team behind the site believes that too much moderation will be required to make blogs feasible.
The Business Consultation website works to empower businesses by encouraging them to let government know their interests. This helps agencies to target people who have said they have something to contribute, helping to promote constructive contributions.
- Ensuring that the role of Ministers is not undermined
There is some anxiety in the public service about whether engaging the public means that public servants are encroaching on the territory of elected representatives; and that Ministers are likely to see this work as undermining them.
On the contrary, reports from the Safe As team were that Ministers were impressed with the project, and grateful for the public input facilitated by the Public Service. For them, it actually lowered the risk of announcing new policy around a politically sensitive issue. It brought public servants out of their office towers, and helped them put forward more practical policy recommendations. Ministers were then positioned to confidently decide on policy direction, rather than having to road test nascent policy ideas through the usual process of 'retail politics'.
- Allowing a small (and perhaps biased) group of engaged people to determine policy
In no cases were policy decisions or recommendations based on public input alone. All used a balance of quantitative, qualitative and expert research to arrive at their recommendations or conclusions.
Guarding the interface between people and agencies
Managing the locus of interests between what agencies would like to know from the public and what the public is reasonably able to provide in response, is a significant role in online participation projects.
The Families Commission is forming its ‘editorial board’ to ensure Couch community members are treated in accordance with strong community engagement principles. The editorial board will insist that engaging, understandable questions using plain English only are asked of the community. They will also insist on clear connections between the questions that get asked of Couch members, and the decisions or recommendations that the Family Commission issues.
The Government Portal and the Business Consultation website also serve as an interface between the public and agencies seeking their input. The Business Consultation website, in particular, has taken steps to regulate how often and on what, agencies can interact with its sign-ups. The Government Portal is also striving to ensure that the content on its site is as understandable and useful as possible for the general public. It is also taking steps that are raising questions about how much it can regulate the content that agencies use to attract the public to consultations.
Supportive leadership
Many organisations involved in online participation have made a decision to engage from senior leadership to project implementers. Projects appear to stumble when senior management or ministerial levels do not have sufficient confidence or awareness of the project.
The Families Commission vets the Couch online polls and is making increasing use of The Couch in its community consultations. Support from senior management and the Commissioners ensures that its inputs are valued and used.
The Safe As Road Safety Engagement Project had the support of two Ministers, who signed off on policy recommendations generated, in large part, from stakeholder and public engagement.
The State Services Commission (SSC) signed off on the Public Address blog post before it went public in accordance with SSC communications protocols. SSC also gave permission for the post’s author (formally a consultant, but a default public servant in the eyes of the public) to interact with the public’s subsequent comments in a free and frank way, as long as the questions were directly related to his area of work. Any other questions outside his scope of work were to be referred to the relevant parties inside SSC. The blog post would not have been effective without this permission.
To date, none of the Bioethics Council’s reports has generated an official response from the Government and the Council has struggled to demonstrate any influence on government policy or programmes on biotechnology. Though the Council wishes to remain independent, a firmer connection with the Executive might lead to its work having more impact.
Evaluation
Evaluation for online participation is still emerging as a practice area. Of our case studies, The Couch and the Safe As project are the only ones to complete evaluations. Each used different approaches.
- The Couch completed an online poll of its membership asking them about their experience. The questionnaire asked about the technical design of the site, as well as what they would like to see in future. See the results here: http://www.thecouch.org.nz/member/report/15
- Safe As commissioned a professional evaluator to review its process. However, the evaluation was completed before the project was finished. See it here: http://www.safeas.govt.nz/SafeAs-Review-Nov-2006.pdf.
A key finding of both evaluations was that people want to see more information about the results of their input. This has encouraged both teams to think more strategically about how to integrate the results of their engagement into the workstreams of their organisations.
Evolution and change
Communities of all kinds evolve and change over time. Online communities are no different and can provide value to agencies beyond the scope of a specific engagement project.
The Safe As project finished in December 2006. However, the materials it generated, such as an information booklet on road safety, and the online community that came together to input into the process, have had enduring value. The information booklet has been used in other contexts in promoting road safety and the online community continues to input and ask questions of the Ministry of Transport using the forum.
There have been gains in efficiency as well. Ministry of Transport officials believe they would have received significantly more ministerial letters and Official Information Act requests regarding the stakeholder engagement process had it not been for the online community forum.
An emerging New Zealand model
If we had to summarise these points into a model for understanding how to create effective feedback loops between agencies and people, what would it look like? Can we create a picture of how people, skills, technology and processes combine to help online participation happen?
Conversation with practitioners, reviews of relevant literature and the above case studies lead us to suggesting the following elements of creating strong feedback loops.
The above diagram illustrates an overview of a virtuous feedback loop between people and government. In it, the government of the day will set policy objectives for agencies, which will use their skills of community engagement, user experience design as well as their knowledge of government policy development and operations to engage people, who will balance their time, interest, and feelings of trust in government, before deciding to contribute their individual skills and personal community networks, connected to usable technology infrastructure. Ministers may choose to get involved in the discussion, and will typically have final say on policy recommendations.
The interaction is driven by excellent use of content and data, positive experience of interactions with others, and an action oriented stance that signals achievement during the life of the project, and in its eventual output.
Out of this process will come policy and service improvement ideas that should, ideally, lead to better outcomes for people -- though, at this point, the evidence of the connection is inferred rather than empirical. They should also lead to changes in agency processes that will help them optimise their performance.
People will then use their heightened awareness and knowledge of policy issues to hold the government of the day more closely to account -- through the electoral cycle, letters to Ministers, public advocacy or through their Members of Parliament. They may also keep pressure on through online channels established by the engagement process.
New ideas may give cause for more discussion, at which point the process will begin again.
Importantly, engagement may be initiated by the players at the top, left or right of the diagram. For now, most of what we’ve seen has been the government of the day or agencies taking the initiative. However, as the recent example of the CYFSwatch* blog has shown, people are also prepared to use the tools to put pressure on Ministers and agencies. It will be important for government actors to be able to respond effectively should these instances become more frequent.
(*The "CYFSwatch" blog was created to "name and shame" Child, Youth and Family Social workers)
Key Elements
Moving from the agency side to the people side, here is more detail about the constituent parts of the model.
Players
The government of the day, agencies and people are the players in an online engagement process. Agencies and people, in particular, will have different views of how a participation project should proceed. Fagan et al nicely lay out the different interests at play.
Table 1. Two views of online participation compared: agency-centred and participant-centred
| Agency-centred | Participant-centred | |
| Focus | Consulter’s needs | Participants’ needs |
| Citizens' role | Sources of information | Partners in problem solving |
| Values | Efficiency, effectiveness, accountability | Experience, relationships, authenticity |
| Purpose | Aggregating preferences | Shaping preferences |
| Timescale | Short term | Long term |
Source: adapted from G. Honor Fagan et al (2006) pp. 37-47 See full article
Agencies will be motivated to engage the public because of statutory requirements, political direction or the need to build relationships to improve the effectiveness of policy proposals or services.
People will be motivated by their interest in a subject, their desire to influence government, to learn new things, build new relationships, or feel a sense of pride and efficacy in their lives. They tend to evaluate opportunities to participate based on their awareness of an initiative, its relevance to their lives or interests, the time commitment required and whether they feel their input would make a difference.
Ministers will want to enhance the reputation of their government and establish effective policy that can be operationalised and deliver results quickly.
Agency skill sets
User experience design is a relatively new design discipline. Born out of the overlap between the study of human-computer interfaces, Web design and industrial product design, this discipline focuses on enhancing how people interact with and relate to systems, products and services. It looks at how to best ensure that systems help people and organisations to achieve their goals through good functionality. It also works to ensure that the psychological and aesthetic experience of the system is enjoyable and positive, since that helps to encourage repeat use and satisfaction.
User experience design processes have two broad stages. First is researching and clarifying the priorities, goals and expectations of those who will use the system, as well as the goals of the people who administer and support it. The second stage involves looking for ‘touchpoints’ where the system and people interact (such as a website or a face-to-face meeting), and using the understanding of people’s expectations and goals to make sure the system and the people who support the system are well positioned to deliver.
Bridging these stages are multiple iterations and prototypes of the reconfigured system, refined by steady feedback from people who administer the system, and those who use it.
User experience design is relevant for online participation because of its emphasis on ensuring two sets of goals are understood and met -- the goals of the people who may become part of a participation process and those of the organisation that will receive input from the process. It then enacts these expectations by building a technical system to support the right outcome for both parties.
Community engagement has a long history as a means of extending opportunities to gather input into policy making from important socio-economic and ethnic groups. Its focus is mainly on the fair and just treatment of people in the course of their contribution to policy making. It uses a variety of techniques to structure information and facilitate discussion so that it sparks meaningful dialogue, deliberation and input from people.
Intermediaries like community, voluntary and professional organisations tend to play a strong role in community engagement projects, because they are well positioned as ‘connectors’ to the people they serve. Government agencies can also play the role of intermediary. And schools and educational institutions are well positioned in this regard.
Strong community engagement processes will take steps to inform people that the engagement is happening and ensure:
- their views are captured through events like public meetings or focus groups
- these are summarised properly -- often they are reviewed by the people who contributed
- the ideas are put in the hands of decision makers
- decisions are fed back through to the people involved.
Community engagement processes work to ensure the views they capture are representative of various points of view and value systems -- and representative in the sense of including the right cross section of ethnic and social strata. They work to build relationships and a sense of trust between participants and between participants and institutions. The overall result is usually a thoughtful and deliberative consideration of the areas of concern and opportunity around an issue, and recommendations about how to proceed.
Community engagement is relevant for online participation because it provides a foundation of practices for actively involving people in policy development.
Policy and operational knowledge of government are critical to any engagement process. Expertise on issues, machinery of government and service operations provide the knowledge used to:
- support the process
- ensure its quality in terms of facts and issues
- appropriately interface the process with the executive.
People skills and capacity
People involved in online participation need the following characteristics.
- Motivation -- people must have the time available, interest in a subject and a belief that agencies will put their input to work before they contribute.
- Literacy-- the ability to read and write in the appropriate language as well as basic computer literacy.
- Community networks -- of friends, neighbors and colleagues who can help contribute more to the discussion.
- Tacit and/or expert knowledge -- of issues. People wear many hats -- a lawyer may also be a Web developer; a stay-at-home father may also have a doctorate in economics; a landlord may also be a carving expert. Also, people have many different personal experiences of important policy areas like families, environmental sustainability and education. It is simply a question of finding ways to access untapped knowledge and experience for the benefit of policy development and service design.
- Confidence -- with technology (particularly the Internet), with sharing views, with interacting with people, with the idea that they have valuable information to share that will be recognised and appreciated by others.
- Technology infrastructure -- such as access to computers and the Internet is critical. Moreover, the opportunities for strong content will mean broadband is increasingly important for online engagement.
Driving the loop
The loop turns back based on four elements that overlap and support one another:
- Content that creates interest and understanding in a given topic, and that fairly summarises people’s views. This is content created by agencies as well as users.
- Data created by online interaction that is discoverable, meaningful and transportable to and from relevant contexts, such as agency websites, the blogosphere or social media sites. This data is the evidence generated by online participation for decision making and evaluation.
- Experience of the process that is positive and satisfying. People learn new things, have fun and feel they have made a difference. Agencies improve policy and their reputations.
- Action taken to move the process on to its next milestone, or action taken based on the results of the overall engagement process. This action is most meaningful when it makes tangible changes in the world that people can easily see.
Spotting opportunities to improve feedback
Part of this project’s work was to look for opportunities to improve feedback loops within each of the case studies. While experience and action were certainly a focus, standing on the outside of each project meant that getting one’s hands on the content and data side of things was often easiest.
The opportunities we saw were based on some key strategies.
- Writing in plain language with people’s motivations in mind so that individuals are well positioned and encouraged to participate.
- Seeking out meaningful information and channelling it so that it points people to relevant discussions or information as quickly as possible.
- Making complex information or long documents, simpler and interactive through online tools.
- Thinking about how to begin creating ‘social’ relationships online between people and government (eg when an agency should ‘friend’ a person online, as on MySpace, Facebook or YouTube).
- Thinking about how to provide better information about agency processes, so that people are clear when they have achieved goals or reached milestones.
We wanted to test and develop these strategies. While there were many ideas, three -- based on the first three strategies noted above -- most effectively put their ‘wheels on the tarmac’. The last two strategies deserve more thought and action, but were hard to develop because of outstanding questions about how to proceed.
Nevertheless, the project did produce several useful prototypes.
- Working with the Safe As project team, we produced two separate but related prototypes:
- We used free online data visualisation tools to give the Safe As summary report an interactive Web presence, using tag clouds and mind maps.
- Using a free online tool and the RSS data feed from the Safe As Road Safety Forum online discussion board, we created a tag cloud that helped people more quickly assess what is happening on the discussion board. The tag cloud tool surfaces commonly used key words that users can click. Each word links to another page that has all the additions to the discussion board using that word.
- Reacting to the difficulties of presenting consultation notifications, we revised two notifications using plain language and hyperlinks. These revisions had two main benefits:
- The revised versions were not obscured by technical language and paid specific attention to people’s motivations for joining a consultation. Adding language about what was at stake, the issue's importance, who would be an ideal responder and how much time and energy it would take to respond, helps provide the information people need to decide if they wish to contribute.
- A general template for consultation notifications was suggested. This template may be picked up by agencies to improve how they engage the public, and may be used by the State Services Commission to develop a consultation notification standard.
These prototypes were not implemented during the life of this project -- there are two main reasons why.
- Agency resources have been committed in other areas and so improvements to existing participation projects are a lower priority than more mainstream agency requirements.
- Agencies have improvements scheduled, but the prototypes here did not fall within those improvement time frames.
The intention to see them implemented still stands, however. Creating prototypes has shown itself to be an effective way of learning about the choices and challenges of online engagement, and a useful mode of leadership in developing agency capacity in online engagement.
Future services
What more can the State Services Commission do to help improve online participation in the future?
Strategy for the dataweb
Online interactions that matter to government services and public policy do not just happen inside the .govt.nz namespace. Social media and networking tools like blogs, Facebook, Myspace and YouTube offer ready made communities where people are producing and sharing content that expresses their thoughts, feelings, loves and hates. Government can meet people where they are.
The by-product of these interactions and expressions is data -- key words, tags, click and click paths that can be aggregated to produce fascinating pictures of the experience of millions of people who are using the Web. This data can be shaped, mashed and manipulated to illuminate paths to new and existing knowledge. It can be used to break down isolation between people and their spheres of knowledge, tightening feedback loops between people participating online.
Government will be challenged by this new capacity. What does it mean for policy making if millions of people are sharing their daily experiences and ideas? What sort of interface will make sense of this seemingly infinite, utterly chaotic resource? And if we can make sense of it, can it be trusted?
Here are a few key areas of focus.
- Social media analytics -- tracking data and influence online through ‘social media analytics’ is an emerging field. The tools have the potential to help agencies mine the Web for those who are talking about public policy and service related issues. Moreover, they help create an aggregate picture of trends. As they grow in sophistication, they could become important evidence gathering tools for policy and service improvement.
- Social media awareness -- social media can be a key organising tool for agencies seeking to involve people in public meetings or in online interactions. Agencies should be made aware of these sites and be given practical strategies for interacting within them.
- Data syndication -- using data syndication tools, like RSS or Atom, may make it easier to bring opportunities to participate across the Web into multiple contexts where it is most relevant for people. However, according to reports from SSC’s ICT Branch, agency uptake of data syndication is low. Without wider uptake of syndication, online participation is likely to fail.
- Government gadgets -- using data syndication functionality, application programming interfaces (sometimes known as widgets and gadgets) package information in highly relevant ways across the Web. This allows content stored in one place (such as a video on YouTube) to be syndicated in others (such as on a weblog). SSC should begin engaging the development community to promote packaging government information into APIs that could be released onto platforms such as Google Homepage, Facebook, TradeMe, MySpace or perhaps the Government Portal.
- Data visualisation -- companies like Stamen, artists like Jonathan Harris and Jess Bachman and not-for-profit organisations like Gap Minder, are creating stunning new ways of seeing and making sense of vast quantities of information. Visualisation of data should be explored as a core public management skill to foster collaborative analysis of important public issues and to help agencies make sense of the large amounts of data produced by online participation projects.
Capacity for the dataweb
The State Services Commission can provide valuable assistance to agency personnel, particularly those in the Participation Community of Practice, in fostering excellent use of the tools.
Areas of focus should include the following.
- Strategy services in exchange for knowledge -- SSC should resource more case study work that aims to prototype improvements to online participation. The work of this resource would be to network with agencies to identify good practice and transfer it around the State sector. Agencies would get an extra strategic resource for their team and SSC would gain up-to-date knowledge of the state of play in online participation.
- Consultation notification standard -- the consultation notification prototype provides strong evidence for change in how agencies notify the public of consultations. Web standards and the Government Portal should be used to influence how agencies create content in this area, using the template supplied in the prototype.
- Multimedia and design skills development -- the Web of the future goes beyond text. It includes audio, video and graphic design. Public servants are not typically trained in these areas. Building a skills programme for them to become leading edge Web communicators and experienced designers will improve the prospects for effective online participation.
Annex: Prototyping improved feedback loops
Taking what we’ve learned, how might this be applied in specific contexts to create improvements?
In the course of creating case studies, opportunities sprang up to prototype improvements, primarily by thinking about how the case study agencies were using content and data. It was our objective to see them tested, and report on their results.
To read about these prototypes, see File:Annex--Prototyping Improved Feedback Loops.pdf.
