There's little doubt that Prime Minister Narendra Modi is the Indian politician with the best citizen-connect today, online or offline. Online, Modi has carved a new path, quickly becoming the most popular Indian politician on Facebook and Twitter. His mobile app has several million downloads on Android devices alone and was the most downloaded app on Apple's App Store this week.
So his attempt this week to gauge people's response to the government's move to demonetise Rs500 and Rs1,000 notes through his mobile app is yet another innovative step in connecting with citizens. Unfortunately, the outcome of the poll is going to be virtually worthless.
Here are four big reasons why.
1. No Sampling
For a survey to be of use, the sample it is conducted on needs to be representative of the universe it is drawn from. For this, it needs to closely mimic the characteristics of the universe--chiefly its demographics. So what pollsters conducting a nationally representative survey in India need to do is find a sample that is as diverse in geography, age, gender, caste and class, to name a few categories, as India is. If the sample isn't representative enough, they might adjust the weights of responses after they get their responses to correct for this. What no respectable pollster would do is not draw up a sample at all.
Unfortunately, this is what an online or app-based poll does--they don't select respondents but merely aggregate the responses of those who chose to take the survey. Some polls try to make up for this by asking personal questions of respondents; the Modi app requires you to register before taking the survey and enter details that include your location.
All this essentially does is describe your respondents--it doesn't make up for the lack of sampling. At most, Modi's team will be able to say something about the gender, geographical spread and professions of the survey's respondents. Even this is unverified information.
2. Sampling Bias
The reason that sampling is particularly important in this context is because the survey's respondents are likely to be atypical. My guess is that smartphone users are likely to be younger, more male, urban, upper caste and richer than the average Indian. More pertinent in this context are their political views. It is fair to assume that a majority of people who have downloaded Modi's app are his supporters. At any rate, internet users tend to be more supportive of the BJP than other population segments, data from the 2014 National Election Study conducted by the Centre for the Study of Developing Societies shows.
Similarly, Modi was consistently more popular with internet users than Rahul Gandhi.
So it isn't very helpful to know that people who largely support you support this move too.
3. Poorly framed questions
A combination of highly loaded and leading questions--"Do you believe some anti-corruption activists are now actually fighting in support of black money, corruption and terrorism" is one particularly amusing one, quite obviously targeted at Delhi CM Arvind Kejriwal. And the lack of "disagree" options to some questions make this a particularly poorly designed survey questionnaire.
4. Risks Loss of Credibility
Having declared the demonestisation exercise a success based on the response of 5 lakh people who he says responded to the survey, Modi runs the risk of the exercise's popularity being dismissed as a result of the poor quality of the survey. This would be unfortunate, because both credible news reporting and credible polls suggest that the move is, in fact, very popular.
As part of its routine tracking poll, CVoter interviewed 1212 randomly selected respondents across all demographics in 11 languages for the Huffington Post/Business World/CVoter nationwide snap poll on 21st November. It covered 252 parliamentary areas across 26 States, the data weighted to the known population profile and the margin of error was +/- 3% at national level and +/- 5% at regional level.
CVoter found that over 70% of people in urban areas and nearly 60% of people in rural areas thought it was a good step.
But there is nuance: a majority thought that the government could have implemented it better.
It seems highly probable that the government's demonetisation is indeed very popular. But by peddling an unscientific survey to bolster its point, the government is doing itself a disservice.