Questions emerge about statistical sampling used on rejected anti

Questions emerge about statistical sampling used on rejected anti-coal petition

Statistics professor finds problems with model used by Elections Alberta

The failure of the Water Not Coal campaign to get the required number of signatures on its anti-coal mining petition is putting the focus on the process Elections Alberta uses to verify signatures.

Jeffrey Rosenthal, a professor of statistics at the University of Toronto, ran an analysis for CBC News using figures provided by Elections Alberta. He said his findings suggest to him that the petition exceeded the signature number threshold.

“It seems that their sampling method is seriously flawed and could vastly underestimate the percentage of valid signatories,” he said in an email to CBC News.

Proponent Corb Lund and his volunteers submitted 207,435 signatures to Elections Alberta on June 11. The petition asks the provincial government to adopt a policy prohibiting all new coal mining and exploration in the Eastern Slopes of the Rocky Mountains, including the Grassy Mountain and Blackstone projects. Alternatively, campaign organizers sought a referendum on the matter.

Corb Lund submits ‘Water Not Coal’ petition to Elections Alberta

Last week, Elections Alberta said its staff verified 172,088 signatures, below the 177,732 signature threshold required by legislation.

Lund said he is mulling his legal options, which could include a request for a judicial review. He said one of his lawyers was in the room for the Elections Alberta verification process.

He told CBC News that his lawyer has “quite a laundry list of things that she felt went on in there that were not fair and disenfranchised thousands of Albertans.”

“So I think we got screwed on this thing pretty hard, actually,” Lund said.

Elections Alberta says it verified the signatures on Lund’s petition using a two-part system. First, staff disqualified signatures that were duplicates or had incomplete information.

Then staff used a sample of 384 signatures on the list to get a result the agency said met the 95 per cent confidence level set out in the provincial Citizen Initiative Act.

People were contacted via phone, text and email over a three-day period. Staff left a message with a number to call if no one picked up.

How Elections Alberta verifies petition signatures

Staff did not try reaching people again after leaving those messages. A spokesperson for Elections Alberta said staff contacted 853 people to get the sample of 384.

Maia-Lys Hanrahan, a spokesperson for Elections Alberta, said in an email to CBC News that people with a phone number or email address that wasn’t in service were deemed to be “fails.” An individual who didn’t return a message sent to a valid email address or left with a valid phone number was not considered a fail.

Lund told CBC News he was told 47 of the 384 were considered “fails.”

In his analysis, Rosenthal said he ran through two scenarios that both found the petition exceeded the signature number threshold

He said it appears to him that Elections Alberta used the midpoint of the confidence interval of plus or minus five per cent in its analysis, instead of the lower bound, which he believes is more appropriate. He said using the lower bound would result in 181,891 valid signatures, well above the threshold of 177,732.

“Indeed, that is the only appropriate way to account for the uncertainties due to their small 384 sample size,” Rosenthal wrote.

Rosenthal said assuming 95 per cent of the people who didn’t respond to Elections Alberta were valid signatories and that the other five per cent were invalid, it would result in a rejection rate of nine per cent, resulting in 178,440 valid signatures — which again would be above the threshold.

Rosenthal said he is also concerned that Elections Alberta didn’t try to follow up with people they initially contacted.

“Because if you don't do some follow-up, then your sample might show fewer valid electors than there really were,” he said. “And that could very much skew the result.”

Elections Alberta defended its process when asked for comment on Rosenthal’s analysis.

Corb Lund says he won't give up fight against coal mining in Alberta's Eastern Slopes

“Respectfully, we’re not going to engage in a debate through the media with experts who have incomplete information,” Hanrahan said in an email.

“We are confident in our process, which was validated by government statistical experts in both its form and function, and welcome a judicial review.”

Lund said he is concerned that Elections Alberta didn’t come up as the caller on people’s phones when its staff reached out during the validation process.

He noted he believes people would be suspicious about verifying their information with someone calling from an unidentified phone number.

A spokesperson for Elections Alberta said that messages were left so people could call the agency directly.

Michelle Bellefontaine covers the Alberta legislature for CBC News in Edmonton. She has also worked as a reporter in the Maritimes and in northern Canada.