WAAAT! - HackMD

--- slideOptions: spotlight: enabled: true --- <style> .slides *:not(pre *) { font-family: "Comic Sans MS", sans-serif } .slides a { font-family: sans-serif } </style> :link: https://hackmd.io/@gabalafou/waaat # WAAAT! JupyterLab as a Lab for Web App Automated Accessibility Testing :clock230: 15:30 - 16:00 📍 Louis Armand 2 Note: - [➡️ Click here for these slides in presentation mode ⬅️](https://hackmd.io/@gabalafou/waaat?type=slide) Speaker: Hello, hello, welcome, welcome. Good afternoon. You made it! You made it to the end of JupyterCon. Give yourself a little stretch if it feels good. This, by the way, this whole thing here, is proof that you can't trust clichés like "they save the best for last." But, actually, this isn't quite the last part of JupyterCon. After this, there are lightning talks, and there are sprints this weekend, but as far as I'm aware, that's it. So we're going to get started in a bit. The title of this talk is Waaat! JupyterLab as a Lab for Web App Automated Accessibility Testing. waaat does that even mean? I'll explain later. Today is Friday, May 12. Gorgeous weather in Paris. We're scheduled from 3:30 to 4 in Louis Armand 2. If it helps you to follow along you can load my presentation in your browser at hackmd.io/@gabalafou/waaat. This will show you all of the content in one page, including my speaker notes. If you prefer to see the content in slide mode, just like it's shown up here, there is a link to the slide mode in the notes of the document view. Links: - [This same document in slide mode](https://hackmd.io/@gabalafou/waaat?type=slide) - Wondering [why this presentation is in Comic Sans font?](https://ericwbailey.website/published/comic-sans-is-a-good-typeface-actually/) --- :link: hackmd.io/@gabalafou/waaat ## Intro Note: Speaker: Before we get started, you should know that I'm not going to be doing an intro to accessibility, so if you're not familiar with accessibility, there will probably be some parts of the presentation that might not make sense, but hopefully most of it should be understandable. --- :link: hackmd.io/@gabalafou/waaat ### Talk outline <img alt="Stephannie's gray and white cat sitting on the purple roof of what could be a doll or dog house, looking very serious" src="https://hackmd.io/_uploads/HyTh3MPE3.jpg" style="float: right" width="300" /> 1. Story: Focus visibility 2. Conclusion 3. Q&A Note: Speaker: Here is a slide of our outline today. First, I am going to tell a story and walk you through one accessibility issue that I worked on. My hope is that by using a specific issue, it will help to make things concrete that might otherwise be a bit abstract. After that, I will do a little wrap up. And finally, I hope to leave some time for questions. Oh and there will be floofs along the way. A floof, in case you don't know, is just a cute word for pet. And on this slide we have one of my coworker's floofs. She's a black and white cat and in this photo she's sitting with a very serious face on the purple roofline of what I imagine is some kind of pet house? Stephannie assures me that Frachesca always has a very serious face. --- :link: hackmd.io/@gabalafou/waaat ### Who is Gabriel? ![@gabalafou's GitHub avatar, glitchy low-fi headshot, wearing sunglasses with white plastic frames, in the style of the deal with it meme](https://i.imgur.com/DgbKo7S.png) GitHub @gabalafou :rainbow: Front-end / Accessibility 🦙 Quansight Labs :unicorn_face: Note: Speaker: But first, who am I? My name is Gabriel Fouasnon. I work at Quansight Labs. My background is as a front-end engineer and my focus these days is in accessibility. One fun fact about me: I once had a 15 minute conversation with Marvin Minsky, sometimes called the father of artificial intelligence, about long division, and I didn't realize until weeks later who I had been talking with. --- :link: hackmd.io/@gabalafou/waaat ![selfie of the presenter and his sister's dog](https://i.imgur.com/zoRYrAC.jpg) Note: Speaker: Okay another floof. This is Moose. Nope he's not a moose. He's just a golden doodle who's sometimes a little confused about his identity, which, like, I can relate. --- :link: hackmd.io/@gabalafou/waaat ## Story: Focus Visible Note: Speaker: So now I'm going to tell you a story about solving an accessibility issue in JupyterLab. --- :link: hackmd.io/@gabalafou/waaat ### Backstory #9399 and WCAG 2.4.7 ![screenshot of JupyterLab GitHub issue #9399 showing where it lists the lack of a visual focus indicator as one of the many accessibility issues in JupyterLab](https://i.imgur.com/WSaYNj6.png) Note: Speaker: But first, a little backstory. So, working in open source is interesting. When I joined the folks working on accessibility in JupyterLab, I quickly learned about an issue on GitHub, number 9399. It was posted in 2020 by the user "manfromjupyter". I asked a coworker more about this issue, and they said they actually don't know who "manfromjupyter" is. Like the man in the iron mask, which, I felt obliged to make that reference since we are in France. So #9399 is a bit infamous, not just for the air of mystery, but also because it's a massive, extensive accessibility audit of JupyterLab. And I would like to draw your attention to just one small part of #9399, just one accessibility issue out of many reported in that audit. In this one area, manfromjupyter points out that the JupyterLab version 2.2.6 was not following the web content accessibility guideline number 2.4.7, which is about making focus visible. Links: - [jupyterlab #9399](https://github.com/jupyterlab/jupyterlab/pull/9399) - [WCAG 2.4.7](https://www.w3.org/WAI/WCAG21/Understanding/focus-visible.html) --- :link: hackmd.io/@gabalafou/waaat #### Focus indicator | on first | on last | | -- | -- | | ![web form with two fields, first and last name, the first field has focus, the second does not](https://hackmd.io/_uploads/Hyjn6w5E2.png) | ![web form with two fields, first and last name, the second field has focus, the first does not](https://hackmd.io/_uploads/H18kRvqEh.png) | Note: Speaker: what is focus visible? So focus visible, without getting too technical about it, is about putting a visual indicator around whichever part of the web page the user is interacting with. The typical example for this is, if you're a sigted user, you're on a web form and you move from one form field to another and you see an outline appear around the form field that you just entered, like in the example screenshot on this slide. On the left, the first name field has focus. On the right, the last name field has focus. But focus is not just for form fields. If you rely on the keyboard, and you press the tab key to move around web pages, then you also want focus indicators around links and buttons and such. In terms of accessibility, having a visible focus indicator is an important feature for anyone who relies on a keyboard to operate a page. It's also important for folks with attention limitations. --- :link: hackmd.io/@gabalafou/waaat ### Fixme: JupyterLab focus ![screenshot of JupyterLab 3.6 with no focus indicator on any element in the UI](https://hackmd.io/_uploads/r1oFuVPV3.png) Note: Speaker: So how does this show up in JupyterLab? Here we have a screenshot of the first screen of the JupyterLab UI, with the file browswer and the launcher tab open. Which part of the UI has focus right now? You might want to guess the new file button, this one here in the top left labeled with a "+" symbol. But you would be wrong. It's actually just impossible to look at this screenshot and see which element has focus. And that's the whole problem. --- :link: hackmd.io/@gabalafou/waaat #### Culprit: bad css 😭 Don't do this ❗️ ```css /* Disable default focus outline */ :focus { outline: unset; } ``` Note: Speaker: Digging into this, it was pretty easy to find the culprit. There was a rule in the CSS that I've put up here on the slide. It says: for any element on the page that has focus, unset the outline, which is also known as a focus ring. So this just turns off the default browser focus indicator. Which is bad for accessibility. Please don't do this. Some elements in JupyterLab created their own focus indicators, using things like the element border, so they were unaffected by this rule. But for everything else, this created an accessibility issue. --- :link: hackmd.io/@gabalafou/waaat #### Fixed in JupyterLab ![screenshot of JupyterLab 4 with focus indicator on an element in the UI](https://hackmd.io/_uploads/BkleKNPVh.png) Note: Speaker: So the fix was pretty easy. I just had to remove the CSS and then fix a few things that got broken in the UI. And here's the result. Now you know which element had focus in the previous JupyterLab screenshot: it was the first file in the file browser. So why is this important? There are lots of reasons, but let's just take one. Imagine you have a disability that makes it hard for you to use a mouse but you can use a keyboard. If you rely on the keyboard, you probably heavily rely on keyboard shortcuts, so you want to know if you press a keyboard shortcut, for example to delete a file or rename a file, then you want to know which file will be affected by that keyboard shortcut. And if you can see which file has focus, that can be difficult if not impossible. --- :link: hackmd.io/@gabalafou/waaat <img alt="Maysoon Zayid is smiling. She’s wearing green eyeshadow and her dark hair is in a bun. She is wearing a purple dress with white cross-stitched butterflies and is carrying a Scottish fold calico cat named Beyoncé." src="https://hackmd.io/_uploads/Hkbe8PUE3.png" width="400" /> Note: Speaker: Okay. It's floof time again. This is Maysoon Zayid with her late cat Beyoncé. Maysoon is an actress, comedian, writer, and disability advocate. She had the number one TED Talk of 2014, titled "I got 99 problems ... palsy is just one." It's really funny, I recommend it. This photo, by the way, is from the Disability Visability project, which you can visit by going to disabilityvisabilityproject.com. You can find links to all of these in the presentation document. Links: - [I got 99 problems ... palsy is just one](https://www.ted.com/talks/maysoon_zayid_i_got_99_problems_palsy_is_just_one) - [Disability Visability Project](https://disabilityvisibilityproject.com/) - Licensed under a [Creative Commons Attribution-NonCommercial 4.0 International License](https://creativecommons.org/licenses/by-nc/4.0/). --- :link: hackmd.io/@gabalafou/waaat ### Testing Note: Speaker: Okay? We fixed the accessibility bug, yay! We made the world a better place, Frodo, can we go back to the Shire now? This is where I would like to talk about the security analogy. By which I mean, a lot of times in the web accessibility space, an analogy is made to security. They're both cross-cutting concerns. They're both kind of asymptotic in the sense that you can never really say your app is 100% secure; similarly you can't ever really say your app is 100% accessibile. Such statements don't really mean anything. If this were a security bug, you probably wouldn't call the job done until you'd written a test to make sure that the bug doesn't reoccur. Because that would be embarrassing, if somebody found reported the same security bug some time down the road, after you'd already declared it fixed. It's the same with accessibility. Once you fix it, don't you want to make sure it stays fixed? --- :link: hackmd.io/@gabalafou/waaat ### Accessibility checkers? Pa11y? Lighthouse? Axe-core? <img alt="" src="https://hackmd.io/_uploads/Bk0ZBfw42.png" width="100" /> <img alt="" src="https://hackmd.io/_uploads/HyOgrMPN2.png" width="100" /> <img alt="" src="https://hackmd.io/_uploads/r1vEzXwVn.png" width="100" /> Note: Speaker: If you ask DuckDuckGo or some other more popular but less privacy-protecting search engines about accessibility testing, there are a number of different things out there that you will probably find, but three of the most popular accessibility checkers that you will find are Pa11y, Lighthouse and Axe-core. But in reality both Pa11y and Lighthouse are built on Axe-core, so it's really just Axe-core. Pa11y actually support one other engine called HTML CodeSniffer. But my point is that if you do web search it will look like there are lots of different checkers to choose from but there's really only a few, and the space is really dominated by Axe-score. So what happens if you run these checkers against JupyterLab? Links: - [Pa11y GitHub](https://github.com/pa11y/pa11y) - [Lighthouse GitHub](https://github.com/GoogleChrome/lighthouse) - [Axe-core GitHub](https://github.com/dequelabs/axe-core) --- ![Screenshot of JupyterLab on left and Lighthouse accessibility report on right](https://hackmd.io/_uploads/H1UdQEDVn.png) Note: Speaker: In this screenshot, you have JupyterLab on the left and Lighthouse on the right. Specifically, you have a Lighthouse accessibility report on the JupyterLab UI that you see on the left. And guess what? There are a few issues it identifies but none of them are about WCAG 2.4.7, focus visible. --- :link: hackmd.io/@gabalafou/waaat #### 100/100 does not mean accessible <img alt="screenshot of 100% accessible web page: plain, vanilla, black and white, default browser styles, semantic HTML" src="https://hackmd.io/_uploads/BkprLuUVh.png" width="350" /> <img alt="screenshot of the same page hacked to be accesible to no one: a blank white screen" src="https://hackmd.io/_uploads/Hyk8U_LN3.png" width="350" /> ![screenshot of Lighthouse report showing accessibility score 100](https://hackmd.io/_uploads/r1isLdUEn.png) Note: Speaker: I bring this up because I think it's really important for developers to understand what these checkers say when they have scanned your web site or app and found no accessibility issues. It's a little bit like using a spell checker. Just because you use a spell checker doesn't mean that your paper is free of writing errors. But I think that's also where these tools create confusion. Because if you run a spell checker against a document and you get no errors, then you probably don't have any spelling errors. You may have some malapropisms or something like that but you can be fairly confident that every word in your document is also in a dictionary. But if you run an accessibility checker against your web site or app, it does NOT mean that your site is free of accessibility bugs. And, at this point I should mention that I am really indebted to a developer named Manuel Matuzovic for my thinking on this subject. I saw a talk of his in which he went wild on a site and made it diabolically inaccessible, like down to the source code. This slide shows on the left the highly accessible website that he started with and on the right—no this isn't a bug in my slides—he actually made the site invisible, but still got a "perfect" score on Lighthouse. Okay, so, where does that leave us? For this particular acccessibility bug about a missing focus indicator, we can't use the off-the-shelf checkers. They don't catch the bug. Maybe we could write a manual test script. What would that look like? If I imagine myself, I would load JupyterLab in the browser, I would press the tab key, and each time I press the tab key, I would check the screen for changes. If at any moment, there's no focus indicator, then the test fails. If only we had a tool that would let us write a test like that ... Links: - Manuel Matuzović's talk: [Building the most inaccessible site possible with a perfect Lighthouse score](https://www.matuzo.at/blog/building-the-most-inaccessible-site-possible-with-a-perfect-lighthouse-score/) --- :link: hackmd.io/@gabalafou/waaat ### <img alt="Playwright logo, two masks, green top of red" src="https://hackmd.io/_uploads/rkUTyt8V2.png" width="80" />Playwright ![Deus ex machina in Euripides' Medea, performed in 2009 in Syracuse, Italy; the sun god sends a golden chariot to rescue Medea.](https://hackmd.io/_uploads/r1U4ddL43.jpg) Note: Speaker: This is an image I took from the Wikipedia page for Deux ex machina. It's of Euripedes' play Medea, performed in 2009 in Syracuse, Italy. The sun god sends a golden chariot to rescue Medea. Deux ex machina: because we've reached that point in our story. Turns out, there is a tool that does exactly what we want. It's called Playwright, it's a free and open source tool from Microsoft. Links: - [Playwright GitHub](https://github.com/microsoft/playwright) - Image license CC-BY-2.5, as seen on the [Deus ex machina Wikipedia image page](https://en.wikipedia.org/wiki/Deus_ex_machina#/media/File:Medea_rappresentation_(2009)_07.JPG) --- :link: hackmd.io/@gabalafou/waaat #### Go-between <img alt="scroll clip art" src="https://hackmd.io/_uploads/H1jflis43.png" width="100" /> ↔️ <img alt="Playwright logo, two masks, green top of red" src="https://hackmd.io/_uploads/rkUTyt8V2.png" width="300" /> ↔️ <img alt="Chromium logo" src="https://hackmd.io/_uploads/B1KFyjo4h.png" width="100" /> Note: Speaker: So what does Playwright do? Well, it acts as a kind of go-between. You can imagine your test code on one side, Playwright in the middle, and on the other side, a browser engine, such as WebKit (the engine behind Safari), or Gecko (the engine behind Firefox), or Chromium (the engine behind Chrome and just about every other browser on the market now). Your test code can tell Playwright to interact with the page in ways that are closer to how a user would interact with the page. So that means you can ask Playwright to click buttons or links on the page, to fill out form fields, to take screenshots, to navigate backwards or forwards, and so on. Links: - [Clip art source](https://openclipart.org/detail/4459/fwd__bubble_hand_drawn-by-rejon-177666) --- :link: hackmd.io/@gabalafou/waaat #### Contrast: A11y checkers vs. Playwright Web page ➡️ Axe-core Test script ➡️ Playwright Note: Speaker: I think probably the easiest way to contrast Playwright with an accessibility checker like Axe-core is to look at what each of them take as input. Axe-core takes a web page as input. It's almost like a Xerox. You give it one page at a time, it scans the page, and it tells you what it found. Playwright, on the other hand, takes as input a test script, like find this button, click it, go to this other page, check a value at the bottom, etc. --- :link: hackmd.io/@gabalafou/waaat #### Test: is focus visible? ```ts= [1-14|1|2|4-13|5-6|7|8|9|10-12] test("should have visible focus indicator", async ( { page } ) => { while (true) { const node = await page.evaluate(() =>/* browser */ document.activeElement); const focus = await node.screenshot(); await page.keyboard.press("Tab"); const noFocus = await node.screenshot(); expect( focus.equals(noFocus) ).toBe(false); } // not shown: breaking out of the loop }); ``` Note: Speaker: And here is an example of test script. Don't worry: I'm going to explain this code. This is basically the code version of the manual test script that I described earlier. Let's break it down. On the first line, we call a function called test. We pass it the name of the test as the first argument: "should have visible focus indicator". And we pass the test function as the second argument. On line 2, Playwright passes a variable called "page" to our test function, which allows us to manipulate and interact with the web page that Playwright has opened for our test. (Remember Playwright is a go-between.) The body of the test function is a loop that, well, loops over all of the tab-focusable elements on the page. Let's take a look at what that does. On lines 5 and 6 we call a function page.evaluate. This is a key functionality in Playwright and one that tripped me up a little bit at the beginning. We pass a function to page.evaluate. This function return document.activeElement, which is basically whatever element in the browser currently has focus. But here's the thing, what you need to understand is that Playwright takes this function that you pass to page.evaluate, and it actually runs the function in the browser's execution environment, waits for the return value, and then relays that return value from browser back into your test code. So that's how lines 5 and 6 get the currently focussed element and we store a reference to that browser element in a variable called node. Next we take a screenshot of the currently focussed element by calling node.screenshot(), which is another useful functionality that Playwright provides. It allows you to actually take screenshots of the page, and yes these are just normal screenshots like you would take. On line 8, we ask Playwright to press the tab key by passing the word "Tab" to the page.keyboard.press() method. And then on the next line we take a screenshot of the node while it doesn't have focus. And in the three lines 10-12, we compare the screenshots. And this is just a simple pixel-by-pixel comparison. If all of the pixels are equal, then we know the test has failed, because if there had been a visible focus indicator, then the two screenshots should be different because one was taken while the element had focus and one was taken while the element did not have focus. --- :link: hackmd.io/@gabalafou/waaat #### Fail 1: node.screenshot() <img alt="Playwright report showing two screenshots of an element from the page, one labeled focussed and the other unfocussed, but both screenshots are the same" src="https://hackmd.io/_uploads/r1M8yGvEh.png" width="300" /> Note: Speaker: And of course, that didn't work. The problem, as I discovered after some debugging was that when you call node.screenshot(), Playwright takes a screenshot of what's called the bounding box, and unfortunately, the bounding box does not include the focus ring, which is what this image shows. This is a screenshot from a section of a Playwright test report onto which I attached the node screenshots. Sorry this can be maybe a little confusing, because I'm showing you a screenshot I took of screenshots that Playwright took. And what this image shows is that both screenshots are identical, but in reality when I tested the app manually, I could see that there was a visible focus indicator. So I was a little disappointed. I thought I had found a clever little way to write a test for the focus visible issue, but it didn't work. By the way, I should just point out that the screenshots that Playwright took are not actually blurry, as they appear in this image. It's just that these are screenshots of two small elements on the page, so the PNG files are actually quite small themselves, but then they get blown up for the report page, and also for these slides. --- :link: hackmd.io/@gabalafou/waaat #### Fix: code ```ts= [1|2|3-8] const { x, y, width, height } = await node.boundingBox(); await page.screenshot({ clip: { x: x - 4, y: y - 4, width: width + 8, height: height + 8 } }); ``` Note: Speaker: But do we give up? No we do not. We read the Playwright API docs and we discover—aha!—Playwright provides functionality for you to take a screenshot of a section of the page. Let's take a quick look at this code. On the first line, we call node.boundingBox(), and this gets us the x, y coordinates of the top-left pixel of the node, as well as its width and height. Next instead of calling node.screenshot() we call page.screenshot() and we pass an option called "clip". And this is basically what it sounds like. It tells Playwright to take a screenshot of the whole page, but then clip away all but this one area. So it's like we add a little bit of cushion to what Playwright was screenshotting, we expand that area just a bit, so that we can capture the focus ring in the screenshot. Alright, so, drum roll please... --- :link: hackmd.io/@gabalafou/waaat #### Fix: screenshot <img alt="Playwright report showing two screenshots of an element from the page. In the top screenshot, the focussed element has a visible black outline. In the second screenshot, the element is unfocussed and does not have the black outline." src="https://hackmd.io/_uploads/HyEaYCDV2.png" width="300" /> Note: Speaker: Yes... yes! On this slide the two screenshots are different, the focussed state did indeed have a focus indicator and the unfocussed state did not. Okay so now does our test work? --- :link: hackmd.io/@gabalafou/waaat #### More fail? Note: Speaker: Well, if computers didn't suck and the world weren't a giant a dumpster fire, maybe. So of course, I did find some more issues, but this was actually good, and really good reason to write tests. I discovered a fairly subtle focus issue with the menubar in Jupyter and I also discovered that the skip link was broken. So the failing test was actually working, as in, it was detecting more focus visible bugs, which is great. I've also read that there can be issues with comparing screenshots in this way, so I might have to tweak that a bit down the road as well. --- :link: hackmd.io/@gabalafou/waaat ### Docs PRs  Issues  Accessibility Guidelines  Limitations  Note: Speaker: Okay, so we've written the test. We opened one pull request to fix the menubar focus issue and another to fix the skip link. Now what? It's documentation time. So, why documentation? I can think of at least two good reasons. Have any of you ever been worked in a library, you get an error, and then the error message has a link to a page that goes into a lot more detail about the error? Has that ever saved you a boat load of time? I know it has for me. And so I wanted to make sure that if this accessibility test fails, then whoever encounters the failure will have some clues to help pinpoint the problem. The other reason I can think of doing this kind of documentation is that it can also help others make accessibility assessments; it could be an input into an accessibility statement. Another reason: accessibility, as a knowledge domain, is currently a bit niche in the world of programming, a bit like security. As a field we want to increase accessibility literacy in the community, just as folks in security want to increase security literacy. But we're not there yet, and so I think that it can be helpful to other developers to have documentation about the accessibility test. So with those goals in mind, what do we want to put on our doc. Well, I can think of a few things right away. [NEXT SLIDE] If you were keeping count, it took me 3 separate PRs to fix the focus visible issue. I think the test doc should definitely link to those PRs. [NEXT SLIDE] I also think the test doc should link to all of the related, relevant GitHub issues. [NEXT SLIDE] It should also link to any accessibility guidelines that explain why this is truly an accessibility bug. [NEXT SLIDE] And I think there should also be a section that explains the limitations of the test. Can it give you a false negative? That kind of stuff. --- :link: hackmd.io/@gabalafou/waaat #### Success! (almost) ![draft PR screenshot for focus visble test](https://hackmd.io/_uploads/B1V0fsjVh.png) Note: Speaker: That's why on this screen you see a draft PR for my test because I still need to write those docs! Links: - [Focus visible test PR](https://github.com/Quansight-Labs/jupyter-a11y-testing/pull/33) --- <img alt="A black and white photo of Hannah Soyer, leaning back in her wheelchair, from her shoulders up. Her cat is laying across her chest, looking at the camera. She is smiling." src="https://hackmd.io/_uploads/SylNnwL43.png" width="400" /> Note: Speaker: We have another floof! That means we're getting close to the end. This is an image of Hannah Soyer and her cat, um, whose name I don't know. Hannah Soyer is a queer, disabled writer interested in perceptions and representations of ‘othered’ bodies. She is the founder of This Body is Worthy, a project designed to celebrate bodies outside of societal ideals. And I once again, got this photo from the Disability Visibility Project, so check them out. Links: - Photo is from the [Disability Visibility project](https://disabilityvisibilityproject.com/2020/05/18/reasons-to-be-angry-during-a-pandemic/) - Used with permission via [Creative Commons Attribution-NonCommercial 4.0 International License](https://creativecommons.org/licenses/by-nc/4.0/). --- :link: hackmd.io/@gabalafou/waaat ## Wrap up Note: Speaker: Alright let's wrap up. --- :link: hackmd.io/@gabalafou/waaat ### Scope and orientation ```graphviz digraph { graph [bgcolor="#191919", fontcolor="#FFFFFF"] node [color="#FFFFFF", fontcolor="#FFFFFF"] edge [color="#FFFFFF"] rankdir=LR; "Accessibility" -> "Digital"; "Accessibility" -> "Physical"; } ``` Note: Speaker: Okay so now that I've told you my story, I want to situate it within a larger context. I'm hoping that this will also unpack the title of my talk, which, uh, just in case you forgot is: waaat! JupyterLab as a Lab for Web App Automated Accessibility Testing. If we zoom way, we start with accessibility. Accessibility is a big space. It encompasses both the built and tangible world, as well as the intangible, digital world. --- :link: hackmd.io/@gabalafou/waaat ```graphviz digraph { graph [bgcolor="#191919", fontcolor="#FFFFFF"] node [color="#FFFFFF", fontcolor="#FFFFFF"] edge [color="#FFFFFF"] rankdir=LR; "Accessibility" -> "Digital"; "Digital" -> "Web a11y"; "Digital" -> "Phone a11y"; "Digital" -> "Desktop a11y"; "Digital" -> "etc." subgraph gray { node [color="#666666", fontcolor="#666666"] edge [color="#666666"] "Accessibility" -> "Physical"; } } ``` Note: Speaker: Next, the digital space of accessibility can be further subdivided into things like phone apps, desktop apps, and the web. --- :link: hackmd.io/@gabalafou/waaat ```graphviz digraph { graph [bgcolor="#191919", fontcolor="#FFFFFF"] node [color="#FFFFFF", fontcolor="#FFFFFF"] edge [color="#FFFFFF"] rankdir=LR; "Accessibility" -> "Digital"; "Digital" -> "Web a11y"; "Web a11y" -> "Web apps"; "Web a11y" -> "Static sites"; subgraph gray { node [color="#666666", fontcolor="#666666"] edge [color="#666666"] "Accessibility" -> "Physical"; "Digital" -> "Phone a11y"; "Digital" -> "Desktop a11y"; } } ``` Note: Speaker: We can then think of dividing the space of web accessibility into static web sites meant primarily for consumption versus web apps, such as JupyterLab which are highly interactive. --- :link: hackmd.io/@gabalafou/waaat ```graphviz digraph { graph [bgcolor="#191919", fontcolor="#FFFFFF"] node [color="#FFFFFF", fontcolor="#FFFFFF"] edge [color="#FFFFFF"] rankdir=LR; "Accessibility" -> "Digital"; "Digital" -> "Web a11y"; "Web a11y" -> "Web apps"; "Web apps" -> "Testing"; "Web apps" -> "ARIA"; "Web apps" -> "etc." subgraph gray { node [color="#666666", fontcolor="#666666"] edge [color="#666666"] "Accessibility" -> "Physical"; "Digital" -> "Phone a11y"; "Digital" -> "Desktop a11y"; "Web a11y" -> "Static sites"; } } ``` Note: Speaker: Because of their greater complexity, web apps have different needs than static websites when it comes to accessibility tooling, which I hope the focus visibility story made clear. In order to write a test, we needed to use Playwright; we couldn't rely on Axe-core or other checkers. So while web apps and static websites have similar accessibility needs, they are also different enough that I treat them separately. Taking that as a given, we can talk about web *app* accessibility as a distinct thing. And beneath web *app* accessibility, we have things like testing, ARIA, the accessibility of underlying frameworks, and so forth. --- :link: hackmd.io/@gabalafou/waaat ```graphviz digraph { graph [bgcolor="#191919", fontcolor="#FFFFFF"] node [color="#FFFFFF", fontcolor="#FFFFFF"] edge [color="#FFFFFF"] rankdir=LR; "Accessibility" -> "Digital"; "Digital" -> "Web a11y"; "Web a11y" -> "Web apps"; "Web apps" -> "Testing"; "Testing" -> "Automated"; "Testing" -> "Manual"; subgraph gray { node [color="#666666", fontcolor="#666666"] edge [color="#666666"] "Accessibility" -> "Physical"; "Digital" -> "Phone a11y"; "Digital" -> "Desktop a11y"; "Web a11y" -> "Static sites"; "Web apps" -> "ARIA"; "Web apps" -> "etc." } } ``` Note: Speaker: Finally, we divide the space of web app accessibility testing: automated versus manual. And now you understand the title of my talk: web app, automated, accessibility testing. Or automated accessibility testing for web apps. --- :link: hackmd.io/@gabalafou/waaat ![Sir Ernest Rutherfords laboratory, early 20th century](https://hackmd.io/_uploads/B1o2QiDNh.png) Note: Speaker: My point here is this is a niche of a niche a niche, which gets to another keyword in my talk title: "lab." Because this stuff is so niche, I often don't know what the tried and true solution is. Hence the lab analogy: we're really learning by doing, trying things out, seeing what works, what doesn't. It's all a work in progress. Links: - [Image source and license info](https://commons.wikimedia.org/w/index.php?curid=28024311) --- :link: hackmd.io/@gabalafou/waaat ### Playwright as compromise <img alt="Chalkboard style drawing showing Playwright as compromise between Axe and disabled testers, in terms of cost and coverage. Two labeled arrows indicate cost and coverage increasing from left to right. And sandwiched between the arrows, left to right: Axe, Playwright, Disabled testers." src="https://hackmd.io/_uploads/SyXlamvV2.png" /> Note: Speaker: But here's what I've learned so far. Accessibility checkers like Axe-core can be great as a first-pass. It's low cost but also low coverage. The best of all worlds is to have disabled people using and testing your app, but hiring human testers is more costly. Playwright sits somewhere between these two. It's more costly than Axe-core but also has more coverage, but is definitely not a substitute for disabled testers. --- :link: hackmd.io/@gabalafou/waaat ### Zoom Out, Shift Left <img alt="Shift Left is about doing things earlier in the development cycle." src="https://hackmd.io/_uploads/HkRXsoP43.png" style="background-color: white" /> Note: Speaker: A lot of this talk has been about how to write regression tests so that when you fix an accessibility issue it doesn't break again. This however is symptomatic of a broken model. Again, let's turn to the analogy with security. A lot of software gets developed without too much thought put into security. Then security bugs are found. Then everybody scrambles to fix the security bug. Repeat, repeat, repeat. Similarly, a lot of software gets developed without too much thought put into accessibility. Then an accessibility audit is done, maybe leading to a lawsuit. Then fixes. In this talk, I focussed on adding accessibility to testing. But you can think about adding accessibility to earlier phases. Think about linting, think about documentation for contributors, think about education and workshops. If you can pay disabled people to test your app before major releases, that's great, but even better if you can get disabled people in the early planning and design stages doing UX research. And even better if you can hire disabled designers and coders to co-create and co-design. There's a gradient from access to participation, and I think all of us in open source don't want to put up barriers to participation, but we do it all the time. Links: - [Shift left image source and credit](https://devopedia.org/shift-left#van-der-Cruijsen-2017) --- :link: hackmd.io/@gabalafou/waaat ### Goals and the Future <img alt="one metal cookie next to an Escher-like tesselated arrangement of cookies" src="https://live.staticflickr.com/7312/11455373525_4cc58363ec_b.jpg" width="400" /> ![JupyterLab logo](https://hackmd.io/_uploads/rkZavYLVh.png) Note: Speaker: Removing barriers to access is the long term mission, but for this testing work, we also have more near-term goals. The main thing that we're hoping to get out of this is a proof of concept that other web application in open source can use. That's why we have such a heavy emphasis on documentation. Eventually, once we've proved out this concept a bit more, we would like to integrate these tests into the default JupyterLab testing infrastructure and continuous integration. Links: - ["Yet another type of Escher cookie cutter"](https://www.flickr.com/photos/21649179@N00/11455373525) by [fdecomite](https://www.flickr.com/photos/21649179@N00) is licensed under [CC BY 2.0](https://creativecommons.org/licenses/by/2.0/?ref=openverse). --- :link: hackmd.io/@gabalafou/waaat ### Acknowledgements <img alt="@trallard GitHub avatar" src="https://hackmd.io/_uploads/rkw7OcUV2.png" width=200 /> <img alt="@isabela-pf GitHub avatar" src="https://hackmd.io/_uploads/rJmMuqLE2.png" width=200 /> <img alt="@steff456 GitHub avatar" src="https://hackmd.io/_uploads/ryiEu9INn.png" width=200 /> <img alt="@tonyfast GitHub avatar" src="https://hackmd.io/_uploads/ByYddcU4h.png" width=200 /> Note: Speaker: In my last slide, I used the word "we" a few times and while I don't actually speak for the people I work with or anyone else, these are some of the fantastic people I had in mind: Tania, Isabela, Stephannie, and Tony. These are their avatars on GitHub. I literally wouldn't be here doing this work right now if it weren't for Tony, Isabela, and Tania because they did the work to write a grant proposal for the Chan Zuckberg Iniative, which allowed them to bring me on. There are so many trailblazers who came before me who have made my day-to-day possible, and I want to acknowledge that. You all also have Tony to thank for the floof pics. I once heard him tell another colleague that their slides needed more dog pics, and well, I took that advice to heart. --- :link: hackmd.io/@gabalafou/waaat #### Let's go! <img alt="Maisey, Tania's Belgian Malinois, just being a model" src="https://hackmd.io/_uploads/S1M-OljN2.png" width="400" style="float: right" /> <img alt="GitHub" src="https://hackmd.io/_uploads/Sy-cCqi42.png" width="50" /><img alt="GitHub" src="https://hackmd.io/_uploads/BJ7aA9s43.png" width="50" />@gabalafou <img alt="Mastodon" src="https://hackmd.io/_uploads/SyZKh9sNh.png" width="50" />@gabalafou@lgbt.tech Note: Speaker: Speaking of which... This is Tania's floof Maisey, looking like the Belgian Malinois runway model that she is. Thank you all for coming. You made it to the end of JupyterCon, or at least to the end of the scheduled speakers. Links: - [Our GitHub repo for this testing work](https://github.com/Quansight-Labs/jupyter-a11y-testing)

Read more

JupyterLab Keyboard UX Discussion and Brainstorm

How to use the Playwright Python API with Axe-core

My takeaways from the Web Directions accessibility conference