Why Prannoy Roy’s U-Turn on his Punjab forecast does no justice to his reputation as a master pollster
Sunday, February 05, 2017
Anybody who takes a keen interest in psephology must be familiar with Dr Prannoy Roy, who is credited as the first psephologist in Indian media - the man who created a bench mark for poll prediction. He burst into the scene during the 1984 Lok Sabha elections when together with and economist Ashok Lahiri, soon after Indira Gandhi's assassination, when as a Doordhasan anchor with Vinod Dua, he hit the bullseye predicting the actual seat number of the Congress scored.
From then onwards,
Prannoy Roy has been at the helm in NDTV carrying out incisive reviews of exit
polls and opinion polls of every general elections held since 1988. Many of us
who tracked Prannoy Roy's predictions ended up getting lifelong bitten by the
bug of psephology and still consider him the master psephologist in the
country.
Imagine the shock
many of us felt when in a span of just 3 days, Prannoy Roy made a radical inexplicable
U-turn regarding his Punjab Assembly predictions as summarised in the table
below
Yashwant Deshmukh,
founder psephologist of CVoter perhaps gave vent to expression to the degree of
shock and awe which all of us fans of Dr Roy felt by this singular tweet
“Dear Dr Roy, you inspired people like me to pick psephology as career. But this stuff is not helping. Probabilities need some kind of base.”
Yashwant Deshmukh
hit the nail on its head - the basis for probabilities is not made clear by
Dr Roy. The dangers of using probabilities as a tool can be illustrated how
Nate Silver of the website www.538.com overnight lost his iconic status within
the psephology industry to be reduced as an international laughing stock. Trump
won despite Silver giving him just 20% winning probability! This when unlike Dr
Prannoy Roy, Nate Silver elaborated the basis of his probability calculations
very comprehensively.
The moot question is
what has radically changed within a span of just 3 days for Prannoy Roy to compel
him to make such a radical revision of probability distribution of contending
political parties winning chances??
A Single FGD is a
sample?
What is extremely
shocking is that Dr Roy undertook a radical revision of the probabilities he dished
out only three days earlier based on a single sample Focus Group Discussion
(FGD) in one village. How can the robustness and representative character of
this sample be defended in terms of psephology or research methodology?
Misleading to
Categorise Bassi Pathana as “Predictor’
The only detail
Prannoy Roy shared about the single sample is that it was one among the several
‘predictors’. He explains that a ‘predictor’ constituency was one which
consistently votes with the tide viz the winner. There are two problems come
across. Punjab had been a bi-polar contest between SAD and Congress from
inception. Only in 2014, AAP made it a tri-polar contest, where SAD-Punjab won
6 out of Punjab’s 13 Lok Sabha seats to AAP’s 4. Bassi
Pathana was one of the constituency leads for AAP and accordingly does not
qualify by Dr Roy’s own definition. So why was it misleading categorised a ‘predictor’?
Character of Bassi Pathana as a Sample
1. Bassi Pathana falls under Fatehgarh Sahib Lok Sabha
constituency which is one of the four constituencies which AAP won in 2012 by
nearly 6% margin.
2. If we were to go to Assembly leads during
Lok Sabha 2016, we see once again AAP sweeping Bassi
Pathana constituency by around 12%
3. The AAP candidate is Santokh Singh Salana, a
controversial individual. The Indian Express exposed Salana being dismissed
from service. Salana worked as a canal patwari in Sutlej-Yamuna Link Water
Services Circle, Ambala, was “absent” from duty for almost four years,
following which he was “dismissed”. Salana had earlier claimed that he had quit
on his own own. This together that he was outsider prompted local AAP cadres to
revolt against his getting a ticket.
Character of FGD
members
Coincidentally this happens to be a SC reserved constituency. AAP’s candidate Salona had been a former BSP leader and thus in a constituency where Dalits are numerically the largest single caste, this gives AAP a huge popularity advantage. In 2012, AAP canabilised most of the Dalit votes from both the BSP and Congress to chalk up nearly 40% lead in Bassi Pathana. What did the FGD members which the NDTV team met with look like? Dalits.
NDTV data also tells
us that the Congress tends to have a relative advantage in urban areas while
AAP in rural areas. Where did the NDTV team conduct their single FGD? Rural
Area!!
The findings of NDTV do not meet the methodological vigour standards of serious research and on the face of it appear to be smacking heavily of deliberate sampling bias. Bad enough the sample is not representative but its selection favoured AAP as the constituency is one of the bastions of AAP.
Even if the FDG were
to be otherwise a representative sample of Bassi Pathana,
it does not lend itself to extrapolate the results to fit the entire
population. And why is this? According to NDTV’s own data, AAP is mainly
an East Malwa phenomenon and relative to the Congress, a marginal player
elsewhere in the state.
AAP
is most likely to win big in constituencies like Bassi Pathana where voter
turnout recorded yesterday was a whopping 77%. At the macro level this would
distort the perception of its vote share looking stronger than they are
actually. This is because their vote share distribution is highly uneven. AAP
will win constituencies like Bassi Pathana big – 50-60% but in the first past
the post system, the extra votes are wasted votes.
Calling wrong an odd
poll or two doesn’t affect the status of icon in the polling industry. As long
as you have a strike rate 6-8 out of 10, a person with a track record as Dr
Prannoy Roy would continue to rule the roost as the leading pollster in the
country. But this U-turn in Punjab predictions within a span of three days is
definitely going to cost Prannoy Roy his credibility as a pollster. AAP may or
may not Punjab. That is secondary. But Dr Roy needs to do is to explain the reasons
for his surprising U-turn. At least Nate Silver since November have been trying
to explain himself. The question is whether Dr Roy would find the courage and
the need to do the same.
1 comments
Dr. Roy we trusted you so much!!
ReplyDelete