owned this note
owned this note
Published
Linked with GitHub
# c3Lingo shift assignment + Fahrplan translation tool
The goal of this tool is to:
- Create translator shifts from the Fahrplan
- Allow translators to express interest in a given shift
- Assign translators to shifts
- Generate the list of completed shifts to import into engelsystem
- Publicize which talks will be translated to which language via an API
- Offer a translation of the name of the talks that will be translated via an API
- The API can be used by social media or Fahrplan and apps that use it
- An ical for translators to make sure no one misses their shifts
Bonus:
- Auto-check of username validity when signing up by calling EngelSystem
- Public page to see translated talks with links to streams, proper credit to translators, etc.
- Internal cheatsheet view for talks (Intros, Links, Numbers, Files, etc.)
- Include files from file drop (or implement file drop functionality)
- Link with engelsystem automatically
- Auto tweets/toots for upcoming talks
- Stats view
- Filters (Past / Current / Future, Day, Booth, Language, Room, Needs translation for field/language)
- Translate bios of speakers
- Comments under talks
For context, and after discussing with Frab/Fahrplan developers, it has been decided that we would provide a web API to share our translations. Once that's ready, the Fahrplan API, which is already aggregating data from various sources, could integrate our content. This decision aligns with recent shift in mindset towards more decentralization of data, rather then centralization (all stored in Frab's db).
### Technologies and dependencies
Django has been decided because multiple participants in the Tech mumble work with it, and Python is overall easier to read for beginners.
No mobile app needed, as long as the web app is mobile friendly.
| | Backend | Frontend | Design | External |
|-|---------|----------|--------|----------|
| Main | Django | jQuery | LessCSS | |
| Extra | iCal lib (tbd), django-bootstrap-form | Bower | Bootstrap, FontAwesome | PoEditor |
### Design
We mentioned using the design chart of 31c3 as our baseline for the design of this tool and any other public communication about c3lingo (including social media).
https://gitea.see-base.de/C3SIGN/31c3-design/
Tests using "Power" font, TeX Gyre Adventor font, or handmade shapes:
[](https://i.imgur.com/wNXSwCb.png) [](https://i.imgur.com/npXangz.png) [](https://i.imgur.com/RfGa9Do.png) [](https://i.imgur.com/ybEYKy3.png) [](https://i.imgur.com/pZeFsyi.png) [](https://i.imgur.com/pJnOfdX.png) [](https://i.imgur.com/tg6XFQ2.png) [](https://i.imgur.com/AJJ7j8i.png)
- [x] Graphic identity has been designed
### Database
A lot of fields retrieved from the Fahrplan are purposefully permissive (can be blank/null, unlimited char space).
The idea is to keep it simple and only "cache" from Fahrplan what is useful to display to translators. The API should also only expose what is not in the Fahrplan + references (ids, codenames).
Naming should be consistent with [Fahrplan API](https://fahrplan.events.ccc.de/congress/2019/Fahrplan/schedule.json) and [Frab](https://github.com/frab/frab).
Tables:
- [Language](#Language)
- [Conference](#Conference)
- [Talk](#Talk)
- [Translation](#Translation)
- [Translator](#Translator)
- [Translator speaks](#Translator-speaks)
- [Booth](#Booth)
- [Shift](#Shift)
- [Shift assignment](#Shift-assignment)
##### Language
Lists the languages we support. Editable by staff only.
| Field | Details | Example | Notes |
|-------|---------|---------|-------|
| Code | CharField, 8 | de | Follow RFC 3066 (like Django does) or ISO 639-3 for signed languages |
| Name EN | CharField, 100 | "German" | Used mostly by our team |
| Name | CharField, 100 | "Deutsch" | For presentation to end users. Use a name users of the language will be most likely to search for. |
Where possible (spoken languages) we should follow the [language list provided by Django]([Choices](https://github.com/django/django/blob/master/django/conf/global_settings.py#L51)).
##### Conference
| Field | Details | Example | Notes |
|-------|---------|---------|-------|
| Acronym | CharField 100, unique, required | 36c3 | |
| Name | TextField, null *(defaults to acronym)* | `36th Chaos Communication Congress` | |
| Image URL | TextField, blank | | |
| Start date | DateField, null | | |
| End date | DateField, null | | |
| Fahrplan version | Textfield | | The latest version of the Fahrplan JSON we imported |
##### Talk
Called "Event" in frab, but everyone talks about "Talk" in c3lingo so I used that.
| Field | Details | Notes |
|-------|---------|-------|
| Conference | ForeignKey(Conference) | |
| Fahrplan ID | CharField, 100, unique | unique_together with Conference |
| Guid | TextField, blank | |
| Title | TextField, blank | From Fahrplan, not translated |
| Subtitle | TextField, blank | From Fahrplan, not translated |
| Abstract | TextField, blank | From Fahrplan, not translated |
| Description | TextField, blank | From Fahrplan, not translated |
| Logo URL | TextField, blank | |
| Language | ForeignKey(Language) | Original language |
| Room | TextField, blank | |
| Length | PositiveIntegerField, null, in seconds | |
| Start date | DateField, null | |
| End date | DateField, null | |
| Type | TextField, blank | |
| Speakers | TextField, blank, *comma-separated usernames from `persons`* | |
```python
@property
def slug(self):
return '{conference_name}-{fahrplan_id}-{name}'.format(
conference_name=self.conference.acronym,
fahrplan_id=self.fahrplan_id,
name=parameterize*(self.name),
)
@property
def watch_url(self):
return 'https://media.ccc.de/v/{slug}'.format(slug=self.slug)
@property
def slides_url(self):
return 'https://speakers.c3lingo.org/talks/{guid}/'.format(self.guid)
```
*[About parameterize](https://apidock.com/rails/String/parameterize) - [source](https://github.com/rails/rails/blob/08304732380773e907bc21e5023c0a32c1e9ea71/activesupport/lib/active_support/inflector/transliterate.rb#L121)
##### Translation
This represents the translation of a property of a talk (title, subtitle, abstract, description).
| Field | Details | Example |
|-------|---------|---------|
| User | ForeignKey(User), null |
| Talk | ForeignKey(Talk), related_name='translations' | |
| Field | CharField, 100, required | `name` |
| Language | ForeignKey(Language), required | `de` |
| Value | TextField, required, blank | "Die Bremsen kreischten fürchterlich" |
- unique_together = `(talk, field, language)`
##### Translator
Represents a translation angel.
Note: User is a model that already exists in Django and already contains username, email, hashed password, and more.
The "Contact Info" may duplicate the email address, but the Email field in User is only visible by staff, users should be prompted to add their email to Contact Info as well if comfortable during signup. DECT too.
| Field | Details | Notes |
|-------|---------|-------|
| User | OneToOneField(User) | |
| Confirmed by | ForeignKey(User), null | Can't login when not confirmed |
| Bio | TextField, blank | Allow Markdown |
| Contact info | TextField, blank | Visible to other translators. Example: your email address, phone number, Twitter @, Telegram, etc. |
| Secret token | CharField(128) | Used to authenticate requests for the iCal URL. User can regenerate a new random secret token at any time. |
```python
import hashlib
@property
def avatar_url(self):
return 'https://www.gravatar.com/avatar/{}'.format(
hashlib.md5(self.user.email.lower()).hexdigest(),
)
```
##### Translator speaks
What Languages a Translator can translate to/from. Editable by each translator for themselves.
| Field | Details |
|-------|---------|
| User | ForeignKey(User), required |
| Language | ForeignKey(Language), required |
##### Booth
Represents a translation booth that contains a console.
| Field | Details | Notes |
|-------|---------|-------|
| Room | TextField, required | |
| Name | TextField, required | |
| DECT | CharField(30), blank | |
| DesiredOccupancy | Integer, required | How many translators are needed to translate a talk in this booth |
| MaxOccupancy | Integer, required | How many translators can work in this booth maximum |
##### Shift
A Shift is an opportunity for a number of Translators to translate a given Talk in a given Booth.
| Field | Details | Notes |
|-------|---------|-------|
| Booth | ForeignKey(Booth) | |
| Talk | ForeignKey(Talk), required | |
| Language | ForeignKey(language), null | null means the shift's language is not defined yet (e.g. 2nd booth, it could become fr/es/... based on volunteers)|
##### Shift assignement
Represents a Translator volunteering for a Shift.
Any translator can waitlist themselves for a shift. Only staff can confirm (set waitlist to false). Only staff can set/unset the "freeloaded" status.
| Field | Details | Notes |
|-------|---------|-------|
| Shift | ForeignKey(Shift), required | |
| User | ForeignKey(User), required | |
| Is waitlist | BooleanField(default=True) | |
| Freeloaded | BooleanField(default=False) | In EngelSystem, "freeloaded" means the angel did not show up. Hours should not be counted. |
| Comment | TextField, blank | User-editable free comment, e.g. to express why they are a good fit for this talk |
### API
`api.c3lingo.org`
`GET /translations/{conference_acronym}/`
```json
{
"acronym": "36c3",
"updated": "2020-12-28T14:03:12+08:00",
"fahrplan_version": "mirror the version of the fahrplan last used to generate this",
"talks": [
{
"fahrplan_id": "11223",
"watch_url": "https://media.ccc.de/v/36c3-10991-science_for_future",
"live_translators": {
"en": [
{
"username": "john89",
"avatar": "https://www.gravatar.com/avatar/d64292036113c3e74f08fdf01055a8a7",
"bio": "I hope you enjoy my translations. Follow me on [Twitter](https://twitter.com/john89). Thank you!"
},
{ ... }
],
"fr": [ ... ]
},
"translations": {
"title": {
"en": {
"translation": "The brakes screeched terribly",
"translator": { ... }
},
"fr": { ... },
"it": { ... }
},
"subtitle": { ... },
"abstract": { ... },
"description": { ... }
}
},
{ ... }
],
}
```
API FAQ:
* Should this be searchable? E.g. filter by room, language, day/timerange?
* Probably no: it's easier to just just dump everything cached in a big json static file, updated once in a while, and let clients filter things themselves
* Documentation?
* With a single endpoint and by keeping fields simple, I don't think we would need to publish a full documentation, besides just saying it exists and link to it.
* User profiles get repeated a lot?
* We considered returning just usernames, and have profiles at the end of the JSON file, but we prefer duplicating the profiles because we think by having them closer to the rest of the data, it's more likely developers will see them and want to use them, which means a higher chance for our translators to be properly credited.
### Ical
`GET /user/{username}/schedule.ics?secret_token=xxxxxxx`
```
ical file of all the shifts a user is assigned to, or waitlisted for.
The event title is something like:
[en->de] [waitlisted] How to hack robots (Hall A)
Events include:
- start time xx minutes before the talk
- location (room and booth) with link to our wiki on how to navigate there (or c3nav)
- description & links to the talk info
- info about other translators, including contact info
```
### Views
- [x] Basics:
- [x] `/signup/` Signup (have to fill details in `Translator` table when signin up)
- [x] `/login/` Login (when confirmed)/Logout/Forgot password
- [x] `/conferences/` List of conferences
- [x] Index`/`:
- [x] If a conference is ongoing, starts soon, or ends soon:
- [x] Redirects to `/conferences/{acronym}/`
- Else:
- [x] Redirects to `/conferences/`
- [ ] List of talks per conferences: `/conferences/{acronym}/`
- [ ] See shifts per talk
- [ ] Insert self in talk's waitlist when interested
- [ ] Save translation of talk details in supported languages
- [ ] Talks view with staff privileges: `/conferences/{acronym}/`
- [ ] Set/unset translators as waitlisted
- [ ] Add translator to a shift (waitlisted or not, defaults to not)
- [ ] Set/unset translators as freeloaded
- [ ] Add/delete/edit shifts per talk
- [ ] User `/user/{username}/`
- [ ] View a user's profile (bio, shifts, languages, ContactInfo)
- [ ] Staff can view the registration email, freeloaded status for shifts
- [ ] Settings `/user/{username}/settings/`
- [ ] Set which language(s) you can speak
- [ ] Regenerate a new SecretToken
- [ ] Staff configuration: `/configurations/`
- [ ] Add/edit languages
- [ ] Add/edit list of booths
- [ ] Export shifts for Engelsystem
- [ ] Force refresh list of talks from Fahrplan
### To do
1. [x] Pre-development
- [x] Create repository
2. [ ] Development
- [x] Design database
- [ ] Write script to import list of talks from Fahrplan
- [ ] Add API routes to get translations details and translators
- [x] Basic authentication
- [ ] Permissions sytem
- [ ] Views
5. [ ] Post-development
- [ ] Deploy to instance
- [ ] Setup cron for auto update from Fahrplan
- [ ] Setup cron for auto caching of API
- [ ] Setup domain or sub-domain (c3lingo.org)
6. [ ] Bonus
# Appendix
### Questions
- Name of this project?
- "Portal"/"Hub"? "shifts.c3lingo.org"?
- c3lingo INSTANT - Interpreter's New Schedule Translation + Assignment Negotiation Tool
- Self assign, or only show interest then confirm by someone? Both options, configurable?
- Is there an OAuth integration in EngelSystem we could use to login to our own tool (to avoid having to double accept newcomers)? -- no there isn't :(
- We would need some layer on Javascript to refresh/keep the page updated when doing shift assignments. Look into websocket? Is it critical, maybe we could let people refresh the page manually?
- Consider the pros and cons of an extra table vs use a [custom User model](https://docs.djangoproject.com/fr/2.2/topics/auth/customizing/#specifying-a-custom-user-model)?
- Should we have per-booth rules so in booth2 anyone can confirm themselves?
Issues in this doc:
* Need to figure out how to deal with Fahrplan talk IDs (do we want the ID, the GUID, both?)
* Maybe need to create a model to represent Rooms
### Links
- Script to parse current pads: https://github.com/c3lingo/tootTranslations/tree/master/toottranslations
### People
- @db0company point of contact for this project
- @erdgeist Fahrplan / frab point of contact
- @bluewhobert is willing to help contribute to Android apps to help integrate with our API when ready
- @korfuri point of contact for Engelsystem integration
- @stb worked with web sockets with Django
- @oskar like to dev, moderate experience with python backends and native JS frontends
Feel free to add yourself if you're interested in joining the efforts!
Sign in with Wallet
