For practitioners of public diplomacy, the ability to accurately measure the effectiveness of PD has often been described as “the holy grail.”

Yesterday at the Heritage Foundation, Under Secretary for Public Diplomacy and Public Affairs Tara Sonenshine spoke about the necessity of public diplomacy and the metrics being used and developed for its evaluation.

Poignantly, the Under Secretary got down to the basic need for public diplomacy, explaining why it is an effective long-term tool for increasing U.S. national security:

Public diplomacy does bring benefits to the American people. It does that by building positive environments with people everywhere – face-to-face, through educational and cultural exchanges, and through social media. When we help more people become healthy, productive, democratic, empowered, and prosperous, they become our economic, trade, social, political, and strategic partners. That spells security and prosperity for America.

The bottom line is this: Public diplomacy is a smart, strategic, – and cost-effective – chess move towards enhancing our national security and building prosperity. More and more of us, Republicans and Democrats alike, understand that.

Exploring the development of metrics, she highlighted the work of the Educational and Cultural Affairs (ECA) Bureau:

Through our Performance Measurement Initiative, ECA alone has surveyed 40,000 exchange participants – both foreign and American –in 35 major programs that involve hundreds of exchanges. In addition to this, ECA has conducted 60 multi-year, multi-country evaluations that survey participants and include fieldwork internationally.

Sonenshine also explored evaluation methods for programs that brought foreign journalists to the United States to give them the experience and knowledge of operating in a free media environment:

We wanted to know how effective these programs were. So we hired an independent firm with experience in program evaluation to evaluate participants in our International Visitor Leadership Program, the Edward R. Murrow Program, and our Citizen Exchange Programs.

They conducted surveys for quantitative data, ran focus groups and held interviews on three continents. And they evaluated the impact of these exchange programs on the opinions, career paths, and professional activities of all participants.

The results were compelling: Significant numbers of these exchange participants went on to advocate for greater freedom of information in their home countries. They worked to support new professional and ethical standards in their work.

And they encouraged their peers to demand greater access to public records.

Looking at digital engagement, Sonenshine said:

We are also evaluating our digital engagement via social media because we know how critical it is to reach key audiences. We examine retweets of our Tweets so we can see how our reach is increasing. We conduct satisfaction surveys on our embassy websites – and adjust our programming accordingly. And we measure the number of clicks our websites get, and how many followers and fans we are getting on our social media platforms.

Examining the re-tweets is extremely important, because it is, at its most fundamental level, a measure of effect. A re-tweet is a tangible action taken by a person to further spread the American message. The question is: who is actually doing the re-tweeting? Are they foreign re-tweeters, or domestic? Are they in or outside of the target audience? If the majority of people following, liking, or re-tweeting State Department material are actually outside of the target audience, what does that mean for effectiveness? This needs to be measured.

Additionally, Sonenshine covered some of the methods to evaluate public diplomacy as a whole, to see if it improves people’s overall views of the United States:

One such study is the biannual Public Diplomacy Impact, which has just completed its 2011 work in Bangladesh, Brazil, Cambodia, Indonesia, Kenya, Morocco, and Turkey.

The PDI, as we call it, is designed to measure the aggregate impact of public diplomacy in general. As with our ECA studies, we use independent evaluators who administer surveys and run focus groups among people who have participated in our programs – as well as demographically matched individuals who have not. With this set of measures, we can see if our programs are influencing opinions.

The 2011 results found that 79.1 percent of our program participants reported initiating some form of positive change in their communities, organizations, or institutions. They reported this as a result of their engagement with public diplomacy programs, tools, or personnel.

Compared to the non-program participants, we saw greater numbers reporting favorability towards the U.S. We also saw better results in their understanding of U.S. policies, society, and values. Those included the concepts of free speech and press, human rights and legal protections, free elections, educational opportunities and religious diversity.

Overall, Under Secretary Sonenshine presented a compelling case for the interest State has in measuring the effectiveness of PD. While so many PD programs require long term evaluation, it is reassuring to know that efforts are being made to do this now. Openness about this process is key in helping both policy makers and the American public understand how public diplomacy can be used to benefit our nation. However, particularly with methods of digital engagement, some scrutiny should be exercised so that we find out who’s in the conversation, and thus whether we are reaching the right people.

Share on Facebook

Tweet