Why Prannoy Roy’s U-Turn on his Punjab forecast does no justice to his reputation as a master pollster

Sunday, February 05, 2017

Anybody who takes a keen interest in psephology must be familiar with Dr Prannoy Roy, who is credited as the first psephologist in Indian media - the man who created a bench mark for poll prediction. He burst into the scene during the 1984 Lok Sabha elections when together with and economist Ashok Lahiri, soon after Indira Gandhi's assassination, when as a Doordhasan anchor with Vinod Dua, he hit the bullseye predicting the actual seat number of the Congress scored.

From then onwards, Prannoy Roy has been at the helm in NDTV carrying out incisive reviews of exit polls and opinion polls of every general elections held since 1988. Many of us who tracked Prannoy Roy's predictions ended up getting lifelong bitten by the bug of psephology and still consider him the master psephologist in the country.

Imagine the shock many of us felt when in a span of just 3 days, Prannoy Roy made a radical inexplicable U-turn regarding his Punjab Assembly predictions as summarised in the table below

Yashwant Deshmukh, founder psephologist of CVoter perhaps gave vent to expression to the degree of shock and awe which all of us fans of Dr Roy felt by this singular tweet

“Dear Dr Roy, you inspired people like me to pick psephology as career. But this stuff is not helping. Probabilities need some kind of base.”

Yashwant Deshmukh hit the nail on its head  - the basis for probabilities is not made clear by Dr Roy. The dangers of using probabilities as a tool can be illustrated how Nate Silver of the website www.538.com overnight lost his iconic status within the psephology industry to be reduced as an international laughing stock. Trump won despite Silver giving him just 20% winning probability! This when unlike Dr Prannoy Roy, Nate Silver elaborated the basis of his probability calculations very comprehensively. 

The moot question is what has radically changed within a span of just 3 days for Prannoy Roy to compel him to make such a radical revision of probability distribution of contending political parties winning chances?? 

A Single FGD is a sample?

What is extremely shocking is that Dr Roy undertook a radical revision of the probabilities he dished out only three days earlier based on a single sample Focus Group Discussion (FGD) in one village. How can the robustness and representative character of this sample be defended in terms of psephology or research methodology? 

Misleading to Categorise Bassi Pathana as “Predictor’ 

The only detail Prannoy Roy shared about the single sample is that it was one among the several ‘predictors’. He explains that a ‘predictor’ constituency was one which consistently votes with the tide viz the winner. There are two problems come across. Punjab had been a bi-polar contest between SAD and Congress from inception. Only in 2014, AAP made it a tri-polar contest, where SAD-Punjab won 6 out of Punjab’s 13 Lok Sabha seats to AAP’s 4. Bassi Pathana was one of the constituency leads for AAP and accordingly does not qualify by Dr Roy’s own definition. So why was it misleading categorised a ‘predictor’?

Character of Bassi Pathana as a Sample

1. Bassi Pathana falls under Fatehgarh Sahib Lok Sabha constituency which is one of the four constituencies which AAP won in 2012 by nearly 6% margin.

2.   If we were to go to Assembly leads during Lok Sabha 2016, we see once again AAP sweeping Bassi Pathana constituency by around 12%

3.  The AAP candidate is Santokh Singh Salana, a controversial individual. The Indian Express exposed Salana being dismissed from service. Salana worked as a canal patwari in Sutlej-Yamuna Link Water Services Circle, Ambala, was “absent” from duty for almost four years, following which he was “dismissed”. Salana had earlier claimed that he had quit on his own own. This together that he was outsider prompted local AAP cadres to revolt against his getting a ticket. 

Character of FGD members

Coincidentally this happens to be a SC reserved constituency. AAP’s candidate Salona had been a former BSP leader and thus in a constituency where Dalits are numerically the largest single caste, this gives AAP a huge popularity advantage. In 2012, AAP canabilised most of the Dalit votes from both the BSP and Congress to chalk up nearly 40% lead in Bassi Pathana. What did the FGD members which the NDTV team met with look like? Dalits.

NDTV data also tells us that the Congress tends to have a relative advantage in urban areas while AAP in rural areas. Where did the NDTV team conduct their single FGD? Rural Area!!

The findings of NDTV do not meet the methodological vigour standards of serious research and on the face of it appear to be smacking heavily of deliberate sampling bias. Bad enough the sample is not representative but its selection favoured AAP as the constituency is one of the bastions of AAP. 

Even if the FDG were to be otherwise a representative sample of Bassi Pathana, it does not lend itself to extrapolate the results to fit the entire population. And why is this? According to NDTV’s own data, AAP is mainly an East Malwa phenomenon and relative to the Congress, a marginal player elsewhere in the state. 

AAP is most likely to win big in constituencies like Bassi Pathana where voter turnout recorded yesterday was a whopping 77%. At the macro level this would distort the perception of its vote share looking stronger than they are actually. This is because their vote share distribution is highly uneven. AAP will win constituencies like Bassi Pathana big – 50-60% but in the first past the post system, the extra votes are wasted votes. 

Calling wrong an odd poll or two doesn’t affect the status of icon in the polling industry. As long as you have a strike rate 6-8 out of 10, a person with a track record as Dr Prannoy Roy would continue to rule the roost as the leading pollster in the country. But this U-turn in Punjab predictions within a span of three days is definitely going to cost Prannoy Roy his credibility as a pollster. AAP may or may not Punjab. That is secondary. But Dr Roy needs to do is to explain the reasons for his surprising U-turn. At least Nate Silver since November have been trying to explain himself. The question is whether Dr Roy would find the courage and the need to do the same.



You Might Also Like


Like us on Facebook

Follow us on Twitter