owned this note
owned this note
Published
Linked with GitHub
# Hackathon: An authoring environment for academic web presence and reach
## Background
Dissemination of academic research can take many forms these days, in addition to the traditionally published papers and books: blogs, videos, software, notebooks, slide decks, tutorials, arXiv, Github repos, Slack workspaces, social media platforms, etc. There is however no easy solution to author and manage this variety of artefacts in a coherent way: each outlet requires its own input, leading to duplication of effort, stuff getting out of sync, errors creeping in, and so on.
## Aim
To develop a unified authoring platform that makes it easy (for reasonably computer-savvy users) to make academic content in all its forms available on the web in a coherent and easily maintained way, increasing both presence and reach of the work going on in an AI research group or CDT.
## Approach
1. We assume relevant content has already been produced and is stored somewhere with a publicly accessible URL (e.g., PDF with ShareLatex, videos, Jupyter notebooks, Reveal.js presentation, etc.).
2. Stuff is pushed out by completing a Markdown template with YAML metadata (some examples below).
3. Uploading the completed template on Github/Gitlab/Bitbucket triggers a **workflow** (e.g. [Github action](https://docs.github.com/en/actions), [Gitlab hook](https://docs.gitlab.com/ee/administration/server_hooks.html)) running plugins or bespoke scripts.
In this hackathon we want to make a start with developing such workflows, e.g. for
- announcing publication of a paper on social media (e.g., [Twitter](https://github.com/marketplace/actions/send-tweet-action));
- adding a video to a Youtube channel organised by tags;
- adding a blog post with links to all artefacts;
- your idea here!
## Markdown template ##
This is not yet fully fixed but should give an idea of the direction of travel.
```markdown
---
title: <TITLE>
tldr: <TLDR>
doi: <Digital Object Identifier>
author: <AUTHOR>
author-scholar: <Google Scholar profile fo the author>
Hero-image: <IMG>
paper-authors: <AUTHOR>
venue: <VENUE>
venue-url: <URL (not a pdf)>
manuscript: <URL (a pdf)>
supplement: <URL (a pdf)>
appendix: <URL (a pdf)>
arxiv: <URL (not a pdf)>
scholar: <URL (not a pdf)>
video: <URL>
poster: <URL>
slides: <URL>
media: <URL> <URL><URL> (Not required, can be many)
code: <e.g., GitHub URL>
binder: <URL>
theme: <THEME> (Required, just **one**)
tags: <TAG> <TAG> <TAG> (Required, can be many)
template: default
---
Content of the blog post.
```
### Completed template for PRG curves
```markdown
---
title: Precision-Recall-Gain Curves: PR Analysis Done Right
tldr: Don't use Precision-Recall curves -- use PRG curves instead! Selected for spotlight presentation at NIPS'15, has clocked up 200+ citations on Google Scholar in six years, comes with code (Python/R/Matlab), video, poster and slides.
author: cspaf
Hero-image: http://people.cs.bris.ac.uk/~flach/PRGcurves/curves.jpg
paper-authors: Peter A. Flach and Meelis Kull
venue: NIPS 2015
venue-url: http://papers.nips.cc/paper/5867-precision-recall-gain-curves-pr-analysis-done-right
manuscript: https://research-information.bris.ac.uk/ws/portalfiles/portal/72164009/5867_precision_recall_gain_curves_pr_analysis_done_right.pdf
supplement: http://people.cs.bris.ac.uk/~flach/PRGcurves/PRcurves_supplementary.pdf
scholar: https://scholar.google.com/citations?user=o9ggd4sAAAAJ&hl=en#d=gs_md_cita-d&u=%2Fcitations%3Fview_op%3Dview_citation%26hl%3Den%26user%3Do9ggd4sAAAAJ%26cstart%3D20%26pagesize%3D80%26citation_for_view%3Do9ggd4sAAAAJ%3ARc-B-9qnGaUC%26tzom%3D-60
video: http://people.cs.bris.ac.uk/~flach/PRGcurves/259653_clipped.mp4
poster: http://people.cs.bris.ac.uk/~flach/PRGcurves/2015_12_07_nips_prg_poster.pdf
slides: http://people.cs.bris.ac.uk/~flach/PRGcurves/PRcurves_ISL.pdf
code: https://github.com/meeliskull/prg
theme: ROC
tags: Fscore Precision-Recall
template: default
---
Precision-Recall analysis abounds in applications of binary classification where true negatives do not add value and hence should not affect assessment of the classifier's performance. Perhaps inspired by the many advantages of receiver operating characteristic (ROC) curves and the area under such curves for accuracy-based performance assessment, many researchers have taken to report Precision-Recall (PR) curves and associated areas as performance metric. We demonstrate in this paper that this practice is fraught with difficulties, mainly because of incoherent scale assumptions -- e.g., the area under a PR curve takes the arithmetic mean of precision values whereas the $F_{\beta}$ score applies the harmonic mean. We show how to fix this by plotting PR curves in a different coordinate system, and demonstrate that the new Precision-Recall-Gain curves inherit all key advantages of ROC curves. In particular, the area under Precision-Recall-Gain curves conveys an expected $F_1$ score on a harmonic scale, and the convex hull of a Precision-Recall-Gain curve allows us to calibrate the classifier's scores so as to determine, for each operating point on the convex hull, the interval of $\beta$ values for which the point optimises $F_{\beta}$. We demonstrate experimentally that the area under traditional PR curves can easily favour models with lower expected $F_1$ score than others, and so the use of Precision-Recall-Gain curves will result in better model selection.
```
### [Rendered version](https://intelligentsystemslaboratory.github.io/homepage-dev/roc/fscore%20precision-recall/2020/12/01/prg-curves/) using the [So Simple Jekyll theme](https://github.com/mmistakes/so-simple-theme)
## Useful tools and frameworks
- [Markdown](https://www.markdownguide.org)
- [YAML](https://yaml.org)
- [Jekyll](https://jekyllrb.com)
- [Binder](https://mybinder.org)
- [Reveal.js](https://github.com/hakimel/reveal.js/) and [RISE](https://rise.readthedocs.io/en/stable/)
- [JupyterBook](https://jupyterbook.org/intro.html)
- [Sphinx](https://www.sphinx-doc.org/en/master/) and [Kacper's Sphinx extensions for Simply Logical](https://pypi.org/project/sphinx-prolog/)
- [You Only Write Thrice](https://kcprbskl.com/you-only-write-thrice/src/yowt.html) has an [example Github workflow](https://github.com/So-Cool/you-only-write-thrice/blob/master/.github/workflows/build-and-deploy.yml) (runs are [here](https://github.com/So-Cool/you-only-write-thrice/actions/workflows/build-and-deploy.yml))
Sign in with Wallet
