Don't Trust Pollara

From earlier this week, a cautionary tale for anyone who was, or ever might be, tempted to take any poll produced by Pollara seriously.

On March 17, Michael Geist wrote a post entitled, "CRIA's Own Study Counters P2P Claims". The study he referred to was a 144 page report (Appendix A) included as part of a CRIA (Canadian Recording Industry Association) submission to the CRTC (Canadian Radio-Television and Telecommunications Commission).

Geist lists a number of findings which seem to suggest that downloading isn't that harmful to the recording industry and concludes,

"In summary, CRIA's own research now concludes that P2P downloading constitutes less than one-third of the music on downloaders' computers, that P2P users frequently try music on P2P services before they buy, that the largest P2P downloader demographic is also the largest music buying demographic, and that reduced purchasing has little to do with the availability of music on P2P services. I've argued many of these same things, but now you don't have to take my word for it; you can take it from the record labels themselves."

Now, in a 144 page report, you're bound to be able to find stuff which supports your conclusion so I took Geist's analysis with a grain of salt (despite the fact that I consider him the most reliable source for intellectual property information). I also figured that, given Geist's high profile, he might get an answering response from the CRIA.

What I didn't expect was for the polling company, Pollara, to produce an 11 page rebuttal to Geist's claims. As Geist says in his follow-up post,

"Pollara suggests that my statements are "misleading, incorrect, and inconsistent." The company adds for good measure that my intervention is "impertinent and presumptuous" and that it hopes that it won't "distract us from the serious business at hand."

You should, as Geist suggests, read through it all for yourself, but what came through quite clearly for me, was that Pollara was not responding simply to clarify the interpretation of their results (which would have been fine by me), they were responding because the CRIA had paid them to do the survey and to the extent that the survey looked bad for the CRIA, Pollara would suffer.

The CRIA is an industry propaganda organization so I expect them to be biased and to twist the facts to support their position as much as they can. True, in an ideal world they wouldn't be so indifferent to the public interest, but I have to be realistic. My willingness to be realistic ends, however, when the company doing the polling is playing the same biased game.

And the 11 page Pollara memo wasn't written by some young partisan hack they only hired because he has an influential father, it was written by the President of the company, Duncan McKie.

Tracking back further into the questions used in the survey, it quickly becomes clear what kind of answers they were looking for. Consider Question 26, from page 127:

"Some people have said that the current copyright laws are sufficient to protect artists, and others suggest that Canadian laws should be up to international standards that have been drafted by the WIPO, the world organization responsible for copyright. Generally, do you think Canadian copyright laws should be up to international standards or is the present law sufficient?"

I could make up some other ways to write this question that might be equally objective, but CopyrightWatch.ca has already done that, some examples:

Do you agree that Canadian law should be amended so as to allow record companies to sue your teenage children for $150,000 in damages for each song they download onto your home computer?

Do you agree that Canadian copyright law should be identical to American copyright law?

Is it personally important to you that copyright law unduly favour existing technologies by suppressing technological innovation?

Is it personally important to you that copyright law unduly favour existing business models by suppressing marketplace innovation?

Do you agree or disagree with this statement: Recording companies or other media firms should be allowed to place potentially damaging software in my computer, without my consent, using their CDs, DVDs, or other digital entertainment products."

and they have more...


But getting back to my main point, a polling company should stick to polling and to doing that in a fair and objective manner. They shouldn't be going to great lengths to push the talking points of the companies who hire them, they shouldn't be making personal attacks against people for posting interpretations of their results and they shouldn't be writing survey questions which are designed to get a specific answer.

When it comes to credibility, Pollara is dead to me now, and if anyone wants me to take anything they say or any study they produce seriously, the burden of proof is on them to convince me that, for the case in question, they were paid to produce a survey and not paid to produce an answer.