# Moat
###### tags: `sketchdance`

Moat is a tool for securely uploading and downloading temporary files. The goal is to allow library patrons to create content (photos, animations, green-screens, stop-motions, games, etc.) and be able to get a short URL/code which can be expressed as a QR code where they can download their content.

Programs that could benefit by having a convenient way for kids (and adults) to get their content that we now mostly throw away:

* Code Clubs
* Pico-8 Workshop
* Photo booth pop-up
* Green screen pop-up
* Stop-motion animation workshop
* Sketch-based animation workshop
* School tours (special effects popups)
* Scratch workshops
* Sketchup workshops
* Etc.

Goals

1. Only legitimate library programs can store content
2. Content is kept on library server
3. Content is deleted after two weeks
4. Content is deleted after 3 downloads
5. There is a standard URL for content, followed by a code
6. Users can visit the base URL and enter the code, or enter a full URL with the code already.
7. A QR code can be generated for the full URL
8. We can hook up a thermal printer to print their code/URL/QR code plus an explanation of the terms
9. We can display the QR code after storage, so they can capture it immediately.

See also [Moving a service from Glitch to its own server](https://hackmd.io/AVQqFsqTQMaf7Soa_KbkCw)

What is in progress (Details)?

* [ ] Add Logo and other branding
* [ ] Thorough documentation
* [ ] UX: Show feedback for drag-and-drop uploading
* [ ] Backend: Implement `/admin/program` instead of sticking files on admin page
* [ ] Admin UI: Have better privacy for admin login
* [ ] Admin UI: Show reaped files
* [ ] Admin UI: Show archived programs
* [ ] Admin UI: Show disk usage / free space
* [ ] Code cleanup: use async and await
* [ ] Code cleanup: simplify, eliminate repetition
* [ ] ~~A reaper program to remove expired files daily~~
* [x] drag-and-drop to upload file (https://www.smashingmagazine.com/2018/01/drag-drop-file-uploader-vanilla-js/)
* [x] Moving to program page logs out admin, logs in only for that program (tighten up security generally)
* [x] Short code for programs
* [x] User UI: Let user jump to file or program from home page
* [x] User UI: Have a URL/QR code for Program pages, to upload files
* [x] Admin UI: Delete a file
* [x] Bug: Ajax form submit is always sending multipart rather than urlencoded form data. WHY WOULD IT DO THAT? (Never satisfactorily solved, but worked around).
* [x] Admin UI: Need to be able to archive programs
* [x] Backend: Implement archiving (requires database migration)
* [x] Backend: Implement deletion
* [x] Admin UI: Show file expired, ready to delete
* [x] Admin UI: Show filesystem usage per program
* [x] Admin UI: Clean up, make more readable
* [x] Admin UI: Link to program pages
* [x] UI: Do not show file download link when file is not available
* [x] UX: Add Program information to the add_file view
* [x] Do not autofill secret key, make it a password field
* [x] Severe Bug: Timestamp is wonky. Created a program in the afternoon and could not see it in Shimmy. Had to create a program for the next day to be usable. Check the conversions to/from ISO-8601 format.
* [x] UI: Show prettier dates, and localized. This may help with timestamp bug
* [x] Moving to server: Run server persistently
* [x] Moving to server: Link to static files
* [x] Moving to server: Test all paths
* [x] Moving to server: Test Shimmy integration
* [x] Functioning download
* [x] UX: absolute url for QR code library
* [x] UX: Add link to download page on download page
* [x] UX: Relative timestamps
* [x] UX: URL in Shimmy goes to download page
* [x] UX: use letters for ids
* [x] UX: save original file name
* [x] file page
* [x] Add a column for integration type (Shimmy, Unsupported, etc.)
* [x] GET path for file page
* [x] file page handler
* [x] Able to render views in Markdown
* [x] Separation of view and layout
* [x] Check for library token, show current programs if valid
* [x] Add program to database
* [x] Bug: Not storing timestamp for programs
* [x] Query programs from database
* [x] Add file to database
* [x] Upload, save file, server-side
* [x] program page
* [x] program path handler
* [x] file upload page
* [x] file upload handler

What is needed (BIG STUFF)?

* [x] Administrative approval
* [x] Express app (or equivalent) to save files
* [x] Public/private keys to prevent unauthorized uploads
* [x] Dedicated storage space
* [x] Database to track uploads, downloads, and expiry dates
* [x] Testing: page to dump database and files
* [x] Navigating to file page signs out admin/program, signs in only for that file (no real login currently, just passes around token via POST)
* [x] Function to generate short, but unique, codes (what is the base64 of a hash?)
* [x] Function to generate [QR codes](https://davidshimjs.github.io/qrcodejs/)
* [x] Decide the URL, for instance https://launchpad.yourlibrary.ca/download/[CODE]

### File page

* Associated program
* Back to program page if uploaded via that page
* Download URL
* File type
* QR Code for the page
* Number of downloads remaining
* Time remaining before file expires

Inputs/Outputs

For Sketchdance Suite: server-side integration. Key is kept on server, connects to Moat via https to push file and receive url/qrcode for file.

For non-Sketchdance programs, librarian will have to log into Moat. Generate a one-time login per program? Identify program and length? Then a file or files can be dragged to the page to upload and generate URL/QRCode.

When visiting URL, show how many downloads are left and when file will expire, ask if they want to download the file now.

Admin page, logged-in librarians only. Show usage by program, size of uploads, how many downloads, expiry dates, etc. Also allow library code to be invalidated/reset.

So far, these endpoints:

`/upload` - API level, used by Sketchdance Suite
`/` - Information about usage - needs design
`/program` - Librarian sign-in and create a program page - needs design
`/program/xxxxxxx` - Generated program page for upload(s), only accessible during program and from the same network as teacher signed in with - needs design
`/admin` - Accessible only to logged-in librarian - needs design
`/admin/xxxx` - Drilling down into data further - needs design

## Table Schemas

### programs

id INTEGER not null,
name TEXT not null,
start TEXT not null,
end TEXT not null

### files

id INTEGER not null,
filename TEXT not null,
filetype TEXT not null,
filesize INTEGER not null,
program INTEGER not null,
created TEXT not null,
downloads INTEGER DEFAULT 0,
reaped INTEGER DEFAULT 0

## Intended flow - Web Version

Need separate flows:

* Create program
* Upload page for unsupported program
* Integration flow for supported programs
* Administration

### Create a new program

This is the most common step for library staff who are running Moat-enabled programs.

digraph hierarchy {
  nodesep=1.0 // increases the separation between nodes

  node [color=black,fontname=Courier,shape=box] //All nodes will this shape and colour
  edge [color=blue, style=dashed] //All the lines look like this

  start [shape=oval]
  start -> index [label="GET /"]
  index -> list_programs [label="POST /"]
  list_programs -> add_program [label="POST /program/new"]
  add_program -> add_file [label="POST /program/create"]
}

### Get to the upload page for an unssupported program

Programs that aren't specifically integrated into Moat can still use it. The program page supports both a traditional form-based file upload and (soon) a drag-and-drop interface for files that works pretty much the same as the integrated version.

digraph hierarchy {
  nodesep=1.0 // increases the separation between nodes

  node [color=black,fontname=Courier,shape=box] //All nodes will this shape and colour
  edge [color=blue, style=dashed] //All the lines look like this

  start [shape=oval]
  start -> index [label="GET /"]
  index -> list_programs [label="POST /"]
  list_programs -> add_file [label="POST /program"]
  add_file -> show_file [label="POST /file/create"]
  show_file -> add_file [label="POST /program"]
}

### User download a file (with a code)

The user can visit the front page of Moat and enter a file code to get to that file's page, which lists how many downloads are remaining, how long until the file expires, and a link to download it.

digraph hierarchy {
  nodesep=1.0 // increases the separation between nodes

  node [color=black,fontname=Courier,shape=box] //All nodes will this shape and colour
  edge [color=blue, style=dashed] //All the lines look like this

  start [shape=oval]
  start -> index [label="GET /"]
  index -> show_file [color=red, label="GET file/[id]" ]
  show_file -> download [label="POST /download"]
}

### User download a file (by URL or QR Code)

The user can bypass the front page and go straight to the file's page by using the full URL (or QR Code representation of the URL).

digraph hierarchy {
  nodesep=1.0 // increases the separation between nodes

  node [color=black,fontname=Courier,shape=box] //All nodes will this shape and colour
  edge [color=blue, style=dashed] //All the lines look like this

  start [shape=oval]
  start -> show_file [color=red, label="GET file/[id]" ]
  show_file -> download [label="POST /download"]
}

### API: list available programs

Programs with Moat integrations use this to list potential programs to save to. In most cases there should only be one program for a given integration in a given time slot, but we allow multiple possible targets.

digraph hierarchy {
  nodesep=1.0 // increases the separation between nodes

  node [color=black,fontname=Courier,shape=box] //All nodes will this shape and colour
  edge [color=blue, style=dashed] //All the lines look like this

  start [shape=oval]
  list_available_programs [color=red]
  start -> list_available_programs [color=red,label="GET /programs (as JSON)"]
}

### API: Save a file to a program

Used when a program with Moat integrtation saves a file to retrieve later. Currently this is re-usable (a single user can save their work multiple times). If it gets abused we may want to implement limits or overwrite previous saves.

digraph hierarchy {
  nodesep=1.0 // increases the separation between nodes

  node [color=black,fontname=Courier,shape=box] //All nodes will this shape and colour
  edge [color=blue, style=dashed] //All the lines look like this

  start [shape=oval]
  start -> show_file [color=red, label="POST file/create (as JSON)" ]
}

### All pathways (for Dethe's reference, mainly)

This includes integration paths used by Shimmy and other integration targets, but does not yet include administrative flow.

digraph hierarchy {
  nodesep=1.0 // increases the separation between nodes

  node [color=black,fontname=Courier,shape=box] //All nodes will this shape and colour
  edge [color=blue, style=dashed] //All the lines look like this

  start [shape=oval]
  list_available_programs [color=red]
  start -> index [label="GET /"]
  start -> list_available_programs [color=red,label="GET /programs (as JSON)"]
  start -> show_file [color=red, label="POST file/create (as JSON)" ]
  index -> list_programs [label="POST /"]
  index -> show_file [color=red,label="GET /file/[id]"]
  list_programs -> add_file [label="POST /program"]
  list_programs -> add_program [label="POST /program/new"]
  add_program -> add_file [label="POST /program/create"]
  add_file -> show_file [label="POST /file/create"]
  show_file -> add_file [label="POST /program"]
  show_file -> download [label="POST /download"]
}