For part deux of this review I decided to go for a beer that is significantly less awful than Honey Brown. Given that everybody here loves pumpkin ales, I found a doozy.
This is my review of Grand Canyon Brewery Will o the Wisp Bourbon Barrel-Aged Imperial Pumpkin Ale.
Part 1 discussed some of the biases within the data that are making polls less reliable. Today I will touch on parts of the book that discuss methods professional pollsters use to account for these biases, and their drawbacks. This is the second part of the book. The first thing Wood points out is how the public at large misinterprets polls to begin with. Most people hear about polls through the media, who among other things, have a penchant for oversimplifying. It is because of the way the polls are presented that most people do not realize the confidence level the pollsters have in the poll results, or even how the margin of error works.
This factor makes it hard to definitively state that findings are wrong; more often, they are presented as simply imprecise. Probability samples using the standard 95% confidence level generally have a margin of error roughly equal to the inverse of the square root of the sample size.48 The confidence level tells us how sure we are that the true average lies within the margin of error.
If in our above example with a simple random sample of 836 registered voters wherein 45% state they will vote for candidate A, assuming there are only two candidates in the race and nobody claims not to know whom they support such that the other 55% state they will vote for candidate B, the real margin of error for that poll at a 95% confidence level is 6.8%. While this poll would report candidate B with a 10-point margin over candidate A, in reality this poll states with 95% confidence that candidate B’s lead over candidate A will be between about 3 and 17 points.
In closer races, this means election polls can claim to be accurate while having next to no predictive power. If in our example, the candidates instead polled at 48% and 52%, the candidate supported by 48% may actually have a lead of nearly 3 points.
This of course means the races could have been within the margin of error the entire time, within the confidence levels, and still come out with the “unexpected” result and the majority of the public would be none the wiser. Unexpected results as we have seen in recent events, have been met with shall we say, less than heroic reaction.
A way professional polls will account for some the self-selection and sampling biases is by weighing the results.
By its nature, weighting entails a lot of assumptions which do not necessarily hold up to scrutiny. It embeds the presumption that researchers somehow know the actual proportion of various groups within the study population such that deviations from those proportions within the sample can be detected. It also presupposes that each group behaves as a block with identical characteristics, rather than as individuals. This could be described as a form of scientific stereotyping, since it suggests that individuals have behaviors identical to the groups to which researchers have assigned them.
Let’s say the researchers who sampled ten people to determine their favorite colors ran the study again with a new random sample pulled from the same population. This time, a sample of 8 people are selected instead of 10. When this group is surveyed, the results are substantially different from the first group: four respondents pick green as a favorite color, one chooses red, and another three select blue. When looking at the sex of the respondents, the researchers notice that seven of the eight in this sample were male, with only one female.
The findings from this second study and those from the first are both equally considered valid. However, the first study made it appear that green and red were the only colors preferred by the population. Due to sampling and weighting errors, it completely missed the fact that blue is also a common preference for some. Additionally, in the second sample, a single data point was used to represent the entire population block of females. A larger sample would have reduced the magnitude of this error, but this example shows how weighting can misrepresent the data and increase error while theoretically accounting for sampling bias in both cases. Weighting trades precision in the hopes of increasing accuracy, yet it can actually detract from both.
All Mexicans with libertarian politics like beer. We can say that, because you all have me as that single data point.
Further into the book, a third section begins to present potential solutions towards obtaining more accurate, and precise poll results. The first seems obvious:
Questions are inherently subjective because they must be interpreted by the person answering the question. Researchers require objective, quantified data rather than a collection of individual interpretations. While some question formats may appear to make quantification possible, such as a “For/Against” question, no two responses are truly comparable when individuals are responding to questions with their own subjective interpretations of the question’s meaning. The same core concept, that different people must answer the same question for the data collected to be comparable, is behind the importance of using identical wording when asking the same question at different points in time to measure changes between the two periods.19 However, because every individual has a different understanding of any given word’s definition, the folly of asking questions at all becomes apparent.
Definitions are probably the most difficult thing to define between individuals with dissimilar viewpoints. Lets pick on the statement, “everyone has the right to free healthcare.” What is a right? What is free? Healthcare in of itself is not a right, but a commodity. We can probably give the pollster the benefit of the doubt and think perhaps they mean access to healthcare, which in of itself may not be a right either, but not something anybody is necessarily going to deny…..Wait, what in the hell do you mean for free!?
One can see how such obvious differences in opinions can lead to differences in how one answers qualitative questions like this.
A potential solution is by using social media algorithms to continuously take in information on the user, and by accounting for self selection biases by measuring sentiment amongst a like minded people. This is not without other issues.
In doing, so these platforms are collecting data that are inherently subject to multiple biases,including social desirability bias. Social media posts are often directly tied to one’s identity, so making taboo statements or posting certain perspectives may have ramifications. Therefore, individuals using social media have incentives to portray themselves in a specific way based on the perceived preferences of their chosen social circles.
As far as samples drawn from social media populations themselves go, there is a fair amount of self-selection bias at work which is closely related to the social desirability bias contained within the data. Many individuals, even those who nominally use a given public or semi-public platform, will react to perceptions of facing social disapproval for their beliefs or group membership by simply opting not to make statements at all, effectively withdrawing from such platforms. Some views are consequentially likely to be proportionally misrepresented. As Anne Halsall, co-founder and CPO of the company Winnie, noted, “Online representations of self must be carefully designed and maintained; a well- cultivated social media account has taken the place of the well-manicured lawn in signaling wealth, status, and general got-it-togetherness to peers.”39
Indeed. My social media accounts all include pictures of me wearing a fitted suit, and generally have little controversial content posted. Why? Perhaps I like to think of myself as an adult, and present myself as such.
The strength of large, properly collected datasets can allow for active proportional sampling. A large dataset may not itself be perfectly representative of a given population, but it is likely to contain representative samples of any given population. Once such a dataset has been assembled, the proper sample for a particular study need only be identified from within that broader dataset.
In cases where specific representation is necessary, researchers can repeatedly randomly sample an existing large set of data until certain parameters are met and study that sample. Since the data has already been collected and researchers in this scenario are only conducting sampling to determine which data to pull, rather than who to attempt to reach, results can be synthesized without worrying about sampling bias, self-selection bias, or the increased error associated with traditional sample adjustment methods. This sample can even be tagged and repeatedly referenced in the future to examine change over time. It can also be isolated from the larger dataset before again running the sampling process until the same parameters are met in a new sample without using the initial sample group, in order to conduct verification checks.
I am going have to go ahead and disagree here. While I accept that virtue signaling is a thing, and one that is not going away anytime soon. There are a number of ways I avoid giving information about myself on the internet. I use as few Google products as possible, along with other no brainers like using a pen-name. Plus when I go on Facebook I will screw with the ad-bot by listing any and all political ads as hate-speech, and saying that NowThis news is sexually explicit. I get ads like this now…
Whatever the results of the election in two weeks, because of this book I am more interested in seeing how the results of poll predictions play out. I have an itchy feeling in some places, they will be dead on, and in others….well…
This is not a beer for the faint of heart. It is a level of insanity that most of you will happily accept, given you are receiving it as a gift. This falls in the “overdone, gluten-free dunkel” category. The bourbon is rather overpowering, but it goes well with the pumpkin since most of us associate it with sweetness, except it does not taste at all like pumpkin pie. It is one to savor the complex palate for a long time. So if you show up to chug it, you are going to have bad time. Grand Canyon Brewery Will o the Wisp Bourbon Barrel aged imperial Pumpkin Ale 4.8/5
As for the book, it is now available on Kindle…unfortunately it may no longer available gratis, but I highly recommend it!