Workshop 2021 Case Study 1

--- title: Workshop 2021 Case Study 1 tags: workshop-2021 --- # Case study 1 - the JCVI debate - what can we learn about online science communication? _Or “a case study in degenerating science discourse”_ The last 48 hours of the JCVI debate forms an interesting case study for the micro dynamics of how things go wrong. We document this to spark ideas for building the information environment we need for effective scientific discourse and communication. :::danger N.b. DISCLAIMER --- it is not our focus whether or not the JCVI decision was or was not right in substance. The point of this case study is to help highlight salient features of online science discourse in its current form in order to help us think about building better tools. ::: [TOC] ## 1. The problem with collective attribution Quoted tweet: <blockquote class="twitter-tweet"><p lang="en" dir="ltr">JCVI minutes:<br>“There is an argument for allowing the virus to circulate amongst children which could provide broader immunity to the children and boost immunity in adults.”<br>That argument is an example of <a href="https://twitter.com/hashtag/childism?src=hash&ref_src=twsrc%5Etfw">#childism</a>: systemic injustice against children.<br>See: <a href="https://t.co/G2mO0emprQ">https://www.bmj.com/content/375/bmj.n2641</a> <a href="https://t.co/OcMgS4r3Jk">pic.twitter.com/OcMgS4r3Jk</a></p>— Tweeted <a href="https://twitter.com/dr2nisreenalwan/status/1454473522575777801">October 30, 2021</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script> ![](https://i.imgur.com/tAOh0M9.png) Retweeted with this comment: <blockquote class="twitter-tweet"><p lang="en" dir="ltr">So now we know JCVI considered deliberate infection of children to be helpful for adults. Just appalling. <br><br>And I suppose now we know why they’ve been so keen to drop all protective measures in schools, and even stop testing in primary schools. <br><br>They WANTED our children infected <a href="https://t.co/Pv1VT7NfpL">https://t.co/Pv1VT7NfpL</a></p>— <a href="https://twitter.com/theAliceRoberts/status/1454486984630710276?s=20">Tweeted October 30, 2021</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script> This tweet (one of the most viewed of the exchange) raises multiple issues regarding collectives: * In what sense does a group (in this case JCVI) “want” something (discussion point in minutes vs. decision criterion etc)? * How does one/should one deal with attributions across collectives (i.e., an individual tweets something :arrow_forward: “iSAGE is doing/believes x,y,z”)? * How can this be managed among informal collectives, i.e. the kind of collective intelligence we might be envisioning? * When should we take scientists as speaking for themselves and when as (at least partly) speaking for others? * What rules/responsibilities/safeguards can one put in place in a context that will see extensive probes for conflicts of interest and or bias? ## 2. The slide from actual words to attributed implication (a) Is a core feature of Twitter, e.g., <blockquote class="twitter-tweet"><p lang="en" dir="ltr">Let’s play a game of “Who said that?”<br>For each statement you have to guess whether this came from an anti-vax group or the minutes of the JCVI.<br>Here we go: <br>“Circulation of covid in children could periodically boost immunity in adults.”</p>— <a href="https://twitter.com/Kit_Yates_Maths/status/1454708818579968003?s=20">Tweeted October 31, 2021</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script> (b) leading to: <blockquote class="twitter-tweet"><p lang="en" dir="ltr">The belief that "the JCVI is an anti-vax group" strikes me as a possible pinnacle in CAnon conspiracy nonsense.</p>— Tweeted <a href="https://twitter.com/BallouxFrancois/status/1454775590339457024?s=20">October 31, 2021</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script> and <blockquote class="twitter-tweet"><p lang="en" dir="ltr">*logs in to twitter* <br><br> “Independent SAGE” members attacking (actually independent) JCVI and comparing them to anti-vaxxers<br><br>*logs off twitter*</p>— Tweeted <a href="https://twitter.com/petesivey/status/1454904612562014211?s=20">October 31, 2021</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script> and <blockquote class="twitter-tweet"><p lang="en" dir="ltr">'Zero Covid absolutists' accuse JCVI of being 'anti-vax' <a href="https://t.co/7uCkoPmkjK">https://t.co/7uCkoPmkjK</a> via <a href="https://twitter.com/MailOnline?ref_src=twsrc%5Etfw">@MailOnline</a></p>— Tweeted <a href="https://twitter.com/BarryJWoods/status/1455323233687941125?s=20">November 1, 2021</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script> * Do initial speakers bear some responsibility for this? * Should this/could this have been foreseen? * Is it ever possible to avoid this on a platform that mixes academics and lay people in the same discourse, given different levels of training and sensitivity to language? * Is it fuelled by the focus on individual speakers? Brevity? ## 3. Chinese whispers/elision… <blockquote class="twitter-tweet"><p lang="en" dir="ltr">Deliberate spread in schools -see jcvi minutes 😔</p>— Tweeted <a href="https://twitter.com/PenderynDic/status/1455603861633347586?s=20">November 2, 2021</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script> <blockquote class="twitter-tweet"><p lang="en" dir="ltr">A criminal biological experiment that caused illness and deaths, number of victims isn't even a factor for JCVI's decision-making process.</p>— Tweeted <a href="https://twitter.com/SmallBigWord/status/1455520665990156297?s=20">November 2, 2021</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script> <blockquote class="twitter-tweet"><p lang="en" dir="ltr">Released info when it thought Cop26 would blanket it, minutes of the JCVI which admits that children and young people have been used to advance herd immunity. This ignores the consequences on vulnerable groups in children and there have been deaths as a consequence. <a href="https://t.co/zAV8YxB8Fs">https://t.co/zAV8YxB8Fs</a></p>— Tweeted <a href="https://twitter.com/AndreBoulanger/status/1455518809578737666?s=20">November 2, 2021</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script> * Which design features of Twitter foster this? Character limits? Hashtags? * What other design features of Twitter promote this problem (e.g., character limit)? * Could this be stopped via (automated) argument/content aggregation that pointed (and, in particular, pointed back) to other instances of the claim? * Is NLP good enough to do at least some of this? Is it desirable? Other, related problems: * Taking statements out of context? * Will such issues persist even in the absence of bad faith actors---are there platform “solutions” (see linking/aggregation)? ## 4. Is it even humanly possible to remain neutral and calm? This space is placeholder for the hundreds of tweets a high profile figure will receive that are unbelievably abusive Is it humanly possible/desirable to carry on in such an environment? This will also almost inevitably lead to overreaction somewhere down the road, which will be hard for observers to contextualise as everyone is “seeing” different versions of the conversation. Abuse seems easy enough to fix in a platform with the right incentives, with control over entry. However, there are also interesting questions raised by this about the respective design features of Twitter vs. Reddit: should what you see in a debate be customizable (see also “muting”, “blocking”), specific to the individual? If yes why, if no, why not? ## 5. Lay person weighs in:  <blockquote class="twitter-tweet"><p lang="en" dir="ltr">Anyone want to set this guy straight?<br><br>Cost is not why we don't vx against pox in the UK.<br><br>Cc <a href="https://twitter.com/apsmunro?ref_src=twsrc%5Etfw">@apsmunro</a> <a href="https://t.co/cXYNAjMP9L">https://t.co/cXYNAjMP9L</a></p>— Tweeted <a href="https://twitter.com/leoniedelt/status/1455135068213719041?s=20">November 1, 2021</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script> ## 6. The amplification of academic aggression <blockquote class="twitter-tweet"><p lang="en" dir="ltr">One may disagree with the JCVI, but they're remarkably mainstream academics with expertise in vaccine development, inf dis epi and immunology. If you suspect they are space lizards intent on killing human babies, please consider there might have been a misunderstanding ...<br>4/</p>— Tweeted <a href="https://twitter.com/BallouxFrancois/status/1454980194956201986?s=20">November 1, 2021</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script> Stylistically, this plays well on Twitter, but it does not add to the argument. It seems problematic that it will be rewarded (in particular by non-scientists). The problems of Twitter specifically as an “outrage machine” are well-known/discussed, but don’t just concern claims, they also concern balance/visibility within argumentative threads. More generally, this raises the questions of the right incentives for promoting high-quality discourse, and shielding academics from perverse incentives that undermine science (seen at its most extreme elsewhere in the COVID debates of academics who have seemingly thrown away academic respectability in favour of “influencer” status….). It also raises the question of who should reward for what, if “likes” of some sort are to be a currency for shaping discourse. Would more fine-grained “like” options be of use? (content, clarity, style, novelty, cogency…) ## 7. General ramping up <blockquote class="twitter-tweet"><p lang="en" dir="ltr">The <a href="https://twitter.com/hashtag/JCVI?src=hash&ref_src=twsrc%5Etfw">#JCVI</a> must be immediately disbanded. <br>Their minutes show that decisions have been driven by misinformation, bad science and political motivation. <br>We need an inquiry. We need to mitigate schools today. <a href="https://twitter.com/hashtag/MakeSchoolsSafe?src=hash&ref_src=twsrc%5Etfw">#MakeSchoolsSafe</a> <a href="https://twitter.com/hashtag/JCVIminutes?src=hash&ref_src=twsrc%5Etfw">#JCVIminutes</a> <a href="https://twitter.com/hashtag/COVID19?src=hash&ref_src=twsrc%5Etfw">#COVID19</a></p>— Tweeted <a href="https://twitter.com/GalNooks/status/1454408874434482183?s=20">October 30, 2021</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script> ## 8. Friendly fire <blockquote class="twitter-tweet"><p lang="en" dir="ltr">There is nothing new about this idea at all<br><br>In fact, this is one of the reasons we don’t vaccinate children against chicken pox in the UK<br><br>It is a totally reasonable thing to include as a point of discussion<a href="https://t.co/7S0z0lEZuF">https://t.co/7S0z0lEZuF</a><br><br>2/ <a href="https://t.co/oCrf0nX5rc">pic.twitter.com/oCrf0nX5rc</a></p>— Tweeted <a href="https://twitter.com/apsmunro/status/1454792162000916481?s=20">October 31, 2021</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script> vs. <blockquote class="twitter-tweet"><p lang="en" dir="ltr">Today's covid drama ...<br><br>JCVI minutes fleetingly mention that infection during childhood likely increases immunisation in adults. Many misunderstood 'adults' as others in society rather than the children themselves as 'adults in the future'.<br>1/</p>— Tweeted <a href="https://twitter.com/BallouxFrancois/status/1454980187976871941?s=20">November 1, 2021</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script> ## 9. But Twitter still, amidst all of this, offers timely substantive argument - even in this debate. So, what can we build that preserves that and loses the problems? ---- Some first, summarizing, thoughts: <sub>(from [here]( https://twitter.com/SciBeh/status/1455458854637117440?s=20))</sub> The current JCVI minutes debate clearly illustrates the problems with Twitter and scientific debate: * meaning is glossed, hedges and distinctions left behind, claims about arguments are conflated with claims about people, giving way to ramped up, emotive soundbites and claims. * important nuance is lost through repeated transmission of messages via actors who do not understand the subtlety in the language and actors who intentionally ignore it * In no time, everyone is outraged, and discussion has degenerated to exchanges about "the other side", and away from the actual issues themselves that we should be debating. * it's not new, but it's depressing every time, and when the stakes are so high, we really need something better. So how can we build a platform that avoids this? Some suggested ingredients: * social norms and content promotion that reward carefully worded material. * calling out (and sanction?) of misrepresentation onsite training/support to help people appreciate the kinds of linguistic distinctions that matter to science * algorithms for content aggregation and visualisation that help link connected pieces across the unfolding debate (both to promote accuracy and undercut bad faith "flooding the zone") ---- ## Discussion Points (in no particular order) 1. One discourse for scientists and non-scientists or two separate (but interrelated) platforms (see e.g., SciBeh’s original set of reddits which had a [scientist only space](https://www.reddit.com/r/BehSciResearch) and a [scientist-public interaction space](https://www.reddit.com/r/BehSciResearch).) 2. Could such spaces be built such that integration/separation is dynamic and can be controlled by the user? (i.e., I can switch between seeing just scientists or scientist-public). This seems technically possible but is it desirable? 3. Should there be “entry requirements”? If yes, what? 4. How (and by whom?) is gatekeeping handled? 5. If we had a suitable eco-system, would we still need Twitter or need to engage on Twitter given its relationship to other systems? 6. Can we build desirable systems out of or on top of Twitter, or to interact with Twitter in other ways? (particularly if answer to preceding question is 5) 7. Twitter pandemic science dialogue mixes science and policy recommendations. Could/should these be separated? Is this a concern in times of normal science? 8. Twitter mixes personal and scientific. Is this helpful/harmful? 9. Tools for highlighting conflicts of interest/bias/etc? Good idea/bad idea? 10. Algorithmic rewiring, content promotion: what kind of content promotion does science discourse need? What is possible? What do we know? N.b, there will always be an organizing principle for how material appears- there is no ‘neutral’ here 11. Organisation by arguer vs. organisation by argument. Twitter organises material by “arguer” and only indirectly (and only very loosely) by “argument” (via #). Reddit, because it has persistent structure, gives slightly more prominence to a claim, but does not amalgamate identical claims across sources. Do we want/need something radically different, i.e., discourse organized (or organizable) by “argument” not by “source”. Is current NLP good enough for this?