Analysis of results of the national poll on the war on Iraq, 7th–23rd March 2003

Contents:

Summary
Publicity and voting patterns
Voting mechanism
Data archive

1. Summary

Although the low participation was disappointing, the results of the poll point at areas for development:

Publicity and credibility are critical: even among the limited number of people reached, a significant proportion did not believe it to be a true, honest exercise. Securing significant national publicity will be necessary for a future poll to be successful.
Text messaging as a voting mechanism has acknowledged flaws, but a surprising one was people’s unfamiliarity with sending empty messages and their consequent reticence to do so. Future polls should allow more than one way to participate.

As promised, the data on the individual votes is available in the data archive below, for those who would like to perform their own analyses.

2. Publicity and voting patterns

The original idea for the poll involved securing national advertising in the press and on TV; we failed to achieve this. In practice, individual contributors seeded a viral e-mail campaign by sending out a total of about 500 messages. In addition, we sent e-mails to a selection of charities (for which we could find suitable addresses) pointing out the potential benefit to them (see the original FAQ).

The resulting pattern of votes is unsurprising in shape, although smaller in both time and volume than we had hoped. Below is a graph of the number of votes received over time (in days) since the voting mechanism was set up:
Graph of votes by day (day,votes): (0,1); (1,0); (2,13); (3,32); (4,14); (5,9); (6,53); (7,5); (8,0); (9,0); (10,5); (11,4); (12,2); (13,1); (14,2); (15,0); (16,2)
There is a sharp peak building up on the day and day after the e-mails were sent out, followed by a gradual tail-off. Had the campaign been more successful, the times for the build-up and the tail might have been longer. Because the numbers are so low, noise is a factor. However, the slight reapparance of voting on day 10 coincides with publicity around the Bush/Blair ultimatum and the apparent start of hostilities.

Bucking the trend is the spike on day 6, which is the largest day of voting and coincides with the appearance of an article in Wired.

Interestingly, the piece in The Guardian had little visible effect. This is probably a combination of its being a comment piece and of the question, by that time, being considered moot by many.

Why was the e-mail campaign not more successful?

Anecdotally, the anonymity of the site — in particular, the lack of a known organisation sponsoring it — was off-putting. In fact, a significant number of people thought it was a cynical money-making exercise based on premium-rate mobile numbers. One recipient thought that the sender’s address had been faked.

Another reason appears to be “classical” apathy. Again anecdotally, some recipients said, verbally, “it wouldn’t make any difference.”

The prospect of a significant donation also did not appear to catch the enthusiasm of charities: we will follow this up with the charities concerned to get feedback.

3. Voting mechanism

We used text messaging as a voting mechanism because of its high penetration (>80% of British population) and low cost (average cost per message to the sender is less than 10p). The e-mails and the original home page explained the mechanism clearly — that is, sending a text message to one of three numbers would constitute a vote for the corresponding statement, and multiple votes from one number would be discarded.

A striking feature of the voting data (see the data archive below) is that around two-thirds of voters felt the need to put some content in the message, even though none was required:

Short, emphatic words or phrases ("Yes", "Agree", "No", "No war") were popular;
"X" appeared once — an echo of paper polling?
Most surprisingly, over 10% of voters felt the need to write longer messages, in some cases reproducing the entire statement for which they were voting.

In summary, while this voting mechanism is technically straightforward, there are some significant usability issues which are likely to block participation.

4. Data archive

The following are available:

The original home page and FAQ page for the Iraq poll.
The voting records in the form of a comma-separated file. The fields within this file are:
- Days, hours, minutes and seconds count the time after the voting system was started;
- Translated source number is an anonymised version of the mobile number that sent the message, to allow spotting of duplicate votes;
- Statement number is the statement for which the vote was made, numbered the same way as the Web site;
- Content is the content of the message.
In preparing the results, we discarded 2 votes that were from the same number, 2 that were apparently from Web-based SMS services (=> not from an individual’s phone), 4 SMS spam messages and 3 test messages.
The final results, which are in no way a properly representative sample, were: 9 votes for statement 1 (back the US without UN); 67 each for the other two (go to war with UN approval, don't go to war under any circumstances).