2021-03-17 Polis Technical Office Hours

Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →
View all notes

Background: https://github.com/compdemocracy/polis/discussions/907

Who's here? metasoarous, lawrencefulton, patcon
Hosting? metasoarous / Computational Democracy

Notes

  • generally reading through /math/src/polismath/math/conversation.clj together
  • metadata comments = constant priority. higher than avg priority / selection prob.
  • metadata comments are important, so get prioritized even if not significant. supports moderators
    • problems: so much high than avg that if moderator asks lots of metadata Qs, then can get heavily front-loaded
    • lots of "survey" questions first, can kinda turn off participants
  • chris spent a bunch of time tuning parameters, but still a bit suboptimal
    • as convo grows, these make less sense and get a bit wonky
  • priority metric + importance metric
  • priority metric scales by a factor the more that the comment has been seen
    • in small convos, number (s) is big. when large convo, is small
    • as you get a bigger convo, might want a better curve to upweight new comments in system ("give new comments a chance to bubble up")
    • bad thing is that it's constant. should be more dynamic, as a function of total number of comments
  • p and a are ests of probability of passing or agreeing
    • can infer prob of disagreeing from these?
    • consensus disagree not shown as much as consensus disagree. favouring positivist framing.
    • if there's something NO ONE agrees with, why are we showing it to everyone.
  • pc: moderator view won't have vote associated with.
    • participants who are hyper engaged might want to tease out a nuance of a group that's not them, and if they know that submitted a statement involves an implied +1, they'll need to write an negative frame. but this will currently be punished, so best to "lie" about personal stance to get positivist up-weighting.
    • lawrence: i might phrase an antagonistic way to appeal to certain audiences. e.g. "i don't want refugees in the country". Even though not common, nor my own view, i might want ppl to respond to this provocation, e.g., if i think one other group would agree.
      • is this intentional? are you trying to encourage agreement? are we trying to be positivist?
    • chris: easier to analyse with positive statements. easier to say what a group is by what the stand for, not for the negative space of disagreement.
    • chris: constructivists. "not not A != A"
      • maybe can't just flip it. hard to do synthesis when there are a lot of negatives
    • pc: GPT-3 to flip statement framing ONLY for reports page, with a big asterisk to warn about constructivist implications (inverse not same).
      • would need to be optional, as GPT API is private
      • lawrence: more dragons around closed source of GPT-3
    • chris noted the above points as interesting and worth considering, as they hadn't been anticipated behaviors, but they make a lot of sense.
  • lawrence: A P S E = agree / pass / distance / extremity [line 305]
    • distance = where you would project comment in PCA space
    • p, a = probabilities
    • seen = number of votes total + 1 pseudo-count so defaults to dumb prior of 0.5 (prior disappears)
    • if not extreme at all (e.g. dead in center of convo) ends up being "1"
      • lawrence: looking for comments that are LESS centered
    • "a split that hasn't been seen before?" chris: maybe, but not necessarily
      • maybe first 2 principal component axes are capturing positions on abortion and gun control, and next one is authoritarian/liberalism. truly meta-axes. bad example, since not independant. e.g. 3 uncorrellated axes. if most activity in convo is about abortion and gun control,
      • this is a third axis in conversation that doesn't have same momentum or overall focus of those rep'd in existing convo
        • lawrence: and will get promoted? if they don't have xxx wouldn't get promoted. but maybe we should promote it to help
        • lawrence: would it be better to do distance before PCA? rather than PCA then distance. why reduce the information to 2 dimensions to calculate distance?
          • chris: wrestled with this early on. reason we didn't was because. literature on PCA is that it's a way of reducing noise. often real stuff and noise.
          • "scree plot" can show that at some point, you can lop of other dimensions, that's where the noise is. where to make that cut is a nontrivial problem.
          • why not just take all distances? bc potentially adding noise to mix.
          • pragmatically, focussing on what will help us place people in two dimensions.
          • BUT as lawrence is getting at, what if there are things happening and bubbling in 3rd or 4th dimensions, we wouldn't want to lose that.
          • chris: "not downweighted for having low-extremity, just not up-weighted"
          • lawrence: sounds like exploration/exploitation discussion
        • thought PCA3 (third axis) might get lost, some of those comments will be on PCA2 or PCA1, and so they're not lost. so if PCA3 was surfacing, it would start
        • pc: is this PCA3 thing like a blind spot? could a high level adversary manipulate convo to keep something in blind spot?
        • chris: think of star field. if you have a plane that you can shine in all directions, how do you shine it to "cast the biggest shadow"
          • odd, because in this, we're inside the starfield.
        • pc: can see why audrey wanted to work in 3D.
        • chris: we did experiments where 2D plots of top 4 PCA dimensions. Caught off-guard the sort of correlations that showed up in there.
      • chris: this has to do with factor analysis. trying to determine what axes mean.
  • CLOJURE HAS EXECUTABLE COMMENTS. code will get evaluated with repl, but gets ignored in compiler.
    • REPLs very important in clojure. REPL-driven development. common in LISP
  • deployment
    • chris: hopeful about switching deployment to docker-compose prod functionality
      • started down this path, got things kinda working. a few gotchas. pretty responsive on bugs. put on backburner for a bit
      • traditionally compose works in production, on one machine
      • recently added ability to deploy to AWS/azure. specify context of compose, and by switching context, can deploy to prod cluster env.
      • docker-compose => docker compose
      • can re-use docker-compose.yml stuff with overlay files
    • pc: helm + kubernetes
    • for now, working on getting it in heroku again
      • doesn't work great with monorepo
      • BUT can deploy from docker images
      • prod/preprod tangling. diff math workers. same database. frontend has different code paths for prod/preprod
      • math can run in multiple modes. if preprod, labels things differently in postgres database
    • branches: edge, main/stable
    • either of HEADs can be deployed
  • notebooks repo is getting merged in analysis repo
    • analysis repo has docker image that it uses to run things
    • can also use custom jupyter setup for writing own notebooks
Select a repo