Echolalia Annotator Follow-Up

# Echolalia Annotator Follow-Up ## TODO ### 18/04/2023 (w/ Xuan Nga) - Lancer entretiens pour - Saad - Noe - Pablo - Commence à partir du premier juin - On les prend pour 2.5 mois/3 mois - AU plus tôt le 15 mai - Idéalement le 15 juins - 650 net en gratif environ ### Entretiens ### Pablo ## Video Script ### Interface Presentation 1. Playback interface - Basic Controls - Play/pause audio/loop - zoom in/zoom out - Validate annotation/remove annotation - Waveform/spectrogram - Waveform/spectrogram is also a segment selection interface 2. Annotations tracks - For each tier, an annotation track - Very similar to what is already present in Praat/Elan, but more interactive - A simplified waveform is underlayed under the annotations to contexualize them 3. Tiers selection - 3 parts: - Tiers "Arming" for annotations - Tiers display for the annotation tracks timelines - Tiers creation/edition 4. Misc: - Display selection for the audio - Keyboard bindings configuration In this video, i'll present the annotation tool we're currently working on. It's aimed at being a web-based replacement for Praat and/or elan. It's aimed at being simple, easy to use, and user-friendly. Note that this is a demo based on a figma mock-up: these are not real screen captures from the tool, but the final product should be as close as possible to what you'll be seeing. There are 4 main components that you can see in the interface: - the first part we'll focus on is the basic controls bar and the audio timeline, which allow users to - view and play or pause the audio - zoom in or out of the audio, to go through it with either speed or precision - create new annotations directly on the audio timeline, first by segmenting then writing down the annotation. We'll get back to that later on in this video - the second part, right underneath, are the annotation tracks - this is very similar to what you can find in praat/elan - this component is synchronized with the audio timeline - each track corresponds to a tier - each track is underlayed with a simplified view of the audio timeline, as to better contextualize the annotations it contains - annotations can also moved/edited on the annotation track - the third part we'll focus on is the left panel, dedicated to tiers. there are 3 different parts in this panel: - the upper one is for what we call "arming" a tier: it allows the user to select the tier with which you want to annotate on the audio timeline - the middle part is there to show you all the tiers in your annotation. It also allows you select which tiers you want to display on the annotation tracks - the lower and final part of this panel is to create tiers and edit tier properties - finally, the last part of this interface is dedicacted to miscelanneous configurations: - this part allows you to select the kind of view you want for your audio timeline: waveform, spectrogram, pitch, etc - the lower part of this panel is for keyboard shortcuts display and configuration: we'll get back to that later on in the videoS ### Actions demo Now, using the figma mockup, i'll try to demonstrate a couple of basic yet central features of the interface we're planning on building. 1. Timeline Interaction (use "Timeline Interaction") The audio timeline, which doubles as a segmentation timeline, can be navigated like any regular audio timeline, such as one found in audio editing tools like Audacity or Ableton. The vertical bar is the playback cursor. As with any such type of interface, it's possible to move it around. The annotation tiers, positionned below the main audio timeline, are fully synchronized with the audio timeline. Scrolling along this timeline will scroll through both the segmentation timeline and the tiers. It's obviously also possible to play the audio from the cursor. [Pause] Or, select part of the timeline, and play only a segment of the audio. Finally, as previously mentionned, we can zoom out of the timeline, to get more visual context on the annotations, as well as zoom in. 3. Segmentation + Annotation (Use "Nouvelle annotation") The first and most important feature is to be able to create new segments and to annotate them. Let's say that we want to annotate with this tier called PERSON-2. We'll first start by arming this tier in the lefthand side of the interface, by clicking it. We can see that it's armed, because there's a small pencil next to it, and it's displayed next to the segmentation timeline. We can now create a new segmentation right on the audio timeline. Then, we write down its corresponding annotation. Finally, we can validate this annotation, which saves it and adds it to the annotation tracks. 3. Coding (Use "Annotation") Another typical annotation task is "coding". It describes a type of annotation that is restricted to a small set of codes. In our case these codes are a, b c and d. Let's say that we want to change the code for this annotation. We first click it, then, on the timeline, we can change it to a new code (in this case, D). Codes can be predefined for each tier, as we'll see later on. 5. Fine segmentation (use "Segmentation Fine") In some cases, you might want to have a very fine control of the boundaries of an annotation segment, while still retaining some view of the general context for that segment. For this, we added what we call "fine segmentation". This will allow an annotator to easily edit a segment's boundaries down to the millisecond. For this, we enable the "fine segmention" audio timeline, which is a zoomed-in view of the currently selected segment's context. On this audio timeline, we can then precisely edit the boundaries of that segment, in the same way we would do it on the regular audio timeline. 7. Tiers display (Use "Naviguer dans la segmentation") In some cases, when there are multiple tiers in the annotation project or file, some of these tiers might be useless for the annotation task. They tend to clutter the annotation tracks and make the annotation process more frustrating. Our interface will allow annotators to select which tiers they want to show or hide, as to make the annotation tracks as unobstructed as possible, depending on the annotation task they want to perform. This can be done by simply clicking the view button on each tier. In this example, we show subtiers that were previously hidden 9. Tiers properties creation/edition (Use "Gérer les tiers") Our annotation interface will naturally allow the annotator to create or edit tiers. This part of the inteface will be dedicated to this task. There are several properties that can be configured for a tier. First, we can configure the type of annotation it can contain. This can be "free" (meaning, segments can be annotated with any text), or controled. Controled annotations can be codes, or restricted to a vocabulary, or even checked by some annotation-checking script. Second, it's also possible to define the hierarchical relationships of a tier: a tier can be a child of another tier, or a parent of a set of tiers. Hierarchical relationships are similar to what can be found in the Elan software. Finally, the type of segmentation a tier contains can also be configured. This includes, for instance, the possibility of segments overlapping in a tier, or to set a tier to be contiguous partition of the audio. 11. # Corrections - Trop de temps mort au début et au niveau des transitions - Modifier A,B, C - Curseur de lecture - Synchronisation des timelines - changer l'armement par un clic directement dans le listing des tiers - virer le décalage ## Questions ### Ergonomics * Do the media playback controls seem to be easy to to find and use (play/stop/forward/backward/zoomin/zoomout)? * Is the audio track visualization sufficiently informative * Does the segmentation tool seem easy and intuitive to use? * Do you think the segmentation tool gives you enough precision to finely annotate boundaries? * Does it look like the editing of the _content_ of annotations is easy and frictionless enough in the mockup? * Just judging by the mockup, do you have a clear mental picture of your annotation workflow on this platform? * What improvements in usability seem the most obvious, in comparison to either Praat/Elan? * Are there any controls in the mockup whose purpose you don't understand by just looking at them? ### Functionalities * Does the tier system presented in the mockup seem to cover all your needs in that regard? * Does the playback interface in the mockup seem to be complete enough to efficiently annotate an audio file? * Do you see any essential functionnality that seems to be missing from the mockup?