# TestJS Summit 2021 心得 (WIP)

summary: 大多專注 e2e testing, visual regression testing, 工商大會 :money_with_wings: :money_with_wings: :money_with_wings:

因為是線上活動，議程皆為預錄，但議後QA或者小組討論會是現場線上語音，受限於與談人硬體限制，可能會聽不清楚。英文不是我的強項，可能某些對話會錯意或者聽不懂導致翻譯錯誤，請多多指教糾正 :bow:

非常感謝 Stanley 贊助門票 ~~(不愧是台GG 2XX 大學長)~~，以及 TBD 們審稿

以下摘要皆為講者提供

---

## [Your Tests Lack Vision: Adding Eyes to your Automation Framework](https://www.youtube.com/watch?v=iqy7zBS1DL8)

### 講者
[Angie Jones](https://angiejones.tech/)

### 摘要
In this talk, you’ll learn how visual validation works, see a live integration into an existing test code base, and discuss the pros and cons of using various visual validation techniques.

### 心得

講者首先分享了 [Whodunnit](https://www.youtube.com/watch?v=ubNF9QNEQLA) 這部影片，影片原意是提醒大眾駕駛時容易有「**不注意視盲(inattentional blindness)**」，可能會有致命危險，但也可套用在 visual regression testing，尤其是在**跨平台的網頁應用**，因為各家硬體及瀏覽器的實作不同，可能會在特定的裝置上顯示不如預期。

> 這是一個很明顯的 visual bug，但只發生在某個特定瀏覽器上，因為這個錯誤讓用戶感到不安心，不知因此流失了多少用戶？
> 圖片來源: 講者簡報

但 visual bug 又不一定非常明顯，有時候只是某個按鈕位置跑掉，也不能保證測試人員或開發人員能夠把畫面上每個 UI 元件都確認過(人類的專注力有限)，所以講者任職的 [Applitools](https://applitools.com/) 提供了一個服務，使用 AI 去判斷 UI 是否有不預期的變動，並不是傳統的 pixel-by-pixel 比對(因為容易有誤差)，能夠準確地標示出「人眼比對出來的差異」(官方號稱 99.5% 準確率)。

以下範例(圖片來源皆來自講者簡報):


> 左右圖有些微小的差異


> 但因為 snapshot 的誤差，如果使用 pixel-by-pixel 的比對方式，結果把所有地方都標示起來了(紅點皆為被標示的差異)


> 使用 AI 比對結果，能夠準確地標示出預期的差異，避免掉不必要的標示

因為 visual regression testing 成本很高(尤其是建置所花的時間及人力成本)，通常能做到 e2e testing 就已經很好了，但 e2e testing 通常只能偵測「某個元件是否有顯示在畫面上」，但無法偵測「**元件是否正常顯示**」，Applitools 提供的服務的確大幅減少了 visual regression testing 所需的建置的成本。


### Q&A

Q: visual regression testing 非常必要嗎？
A: 講者認為動態頁面可能不適合，但不是做不到，需要根據 API 回傳的內容來決定測試結果，或者直接略過會時常變動的區塊，推薦使用在靜態(或不常變動的內容)頁面。

Q: 要怎麼把 false positive 最小化？
A: 透過 CI 去判斷改動是否如預期，使用表情符號 :thumbsup: 或 :thumbsdown: 回饋結果。

Q: 要怎麼測試動態內容
A: 使用可控 query (例如: 日期 timestamp)，固定的資料內容去比較。講者提到如果是在行動裝置上，畫面最上方的狀態列永遠是變動的，可以讓每次測試在程式上(programmably)忽略它。

Q: Applitools 的服務是怎麼做到如此準確？
A: 機器學習。

Q: Applitools 的服務帶來的利益是？(與 snapshot testing 相比)
A: snapshot testing 還是有其價值，如果不是畫面樣式的改動(單純修改程式面的邏輯)，那 snapshot testing 就足夠了。

Q: a11y testing 如何？
A: 很棒呀，Applitools 也有提供此服務喔 :wink:

Q: Applitools 有自架伺服器(on-premise)的方案嗎？
A: 有，但建議還是使用官方雲端方案。

---

## Achieving A11y Automation Testing – Ava Wroten

Accessibility testing in has come a long way in recent years. We'll dive into how EmberJS prioritized A11y with meaningful RFC's, Addons, tooling and docs. Most importantly, we'll discuss how these successes can be applied to your very own apps be they Vue, React, Angular or anything else!

---

## Shipping High Quality JS Apps with Confidence – Tomasz Łakomy

Shipping bug-less code to production is (obviously) impossible, but still - our users deserve the best experience we can give them. Not only that - if we gain confidence in the way we build our software, we can sleep better at night knowing that it won’t explode in the middle of the night.

In this talk we're going to cover something I call "The Testing Spectrum" - a set of tools, practices and mindset of shipping high quality code to production. From prettier all the way to monitoring, let's avoid your next production incident together!

---

## Deploy with Speed and Confidence Using Contract Testing and Pact – Matt Fellows

It’s almost 2021 and we still rely on integrated environments and large end-to-end test suites to release complex, distributed applications called "software". In this talk, Matt breaks down the arguments for such nonsense and shows how a better, faster, safer alternative.

---

## Testing for the Modern Web with Playwright – Arjun Attam

The modern web platform is continuously evolving. Today's web apps are more sophisticated than ever before and testing for the modern web requires modern primitives. In this talk, we will cover how Playwright is uniquely enabling web developers to ship faster and more confidently.

---

## Play it Right with CodeceptJS: An Introduction to Supercharged E2E Testing – Michael Bodnarchuk

CodeceptJS is BDD-style end-to-end testing framework which plays nicely with all popular testing engines. CodeceptJS has built-in support of PageObjects, DataObjects, plays nicely with TypeScript, and even has its own UI app! In this session, we will take a look how CodeceptJS can be paired with Playwright to make the testing simple and effective.

---

## Core Web Vitals - What, Why and How? – Martin Splitt

Performance can make or break a website, but how can you quantify that? In this session we will look at the Core Web Vitals as a way to measure performance on the web. Specifically, we'll go through the history of web performance measurements, where the new metrics come from and how they are measured.

---

## Don’t Make These Testing Mistakes – Gleb Bahmutov

In this talk, I will discuss the common mistakes developers make when writing Cypress tests and how to avoid them. We will discuss tests that are too short, tests that hardcode data, tests that race against the application, and other mistakes. I believe this presentation will be useful to anyone writing E2E tests using JavaScript.

---

## Writing Testable Serverless Apps Using Hexagonal Architecture – Slobodan Stojanović

According to many polls, testing serverless applications and fear of the cloud vendor lock-in are among the top five challenges organizations face when adopting serverless. We often hear that using serverless effectively requires a mind shift. But what does that mean? Do we need new tools and strategies for testing serverless applications, or can we use existing tools we already use for our non-serverless applications? And what about cloud vendor lock-in? Is that a real thing or just a fictional story that scares people away from serverless? Can we decrease a risk of vendor lock-in using a well-known architecture, such as hexagonal architecture?

---

## The Evolution of Browser Automation – Christian Bromann

In this session, we’ll take a look at what has happened behind the scenes in browser automation throughout the years and what the future will have in stock for us. We will examine how web testing will develop and what challenges this will bring for conventional frameworks like Selenium or WebdriverIO, as well as new frameworks such as Cypress, Puppeteer and Playwright. Lastly, we will experiment with some new automation capabilities these frameworks provide to test some of the new web features.

---

## Detox: The Unobtainable Test Stability (or is it?) – Rotem Mizrachi-Meidan

In this talk, we'll discuss how Wix is using Detox internally, how we manage configuration, how we fight flakiness, and some best practices we've developed over the ~3 years of building and using Detox in our CI process. We'll also discuss our endless striving for "0 manual QA", which always seems in reach, if we only overcome that one last technical hurdle.

---

## Visual Regression Under the Hood – Dmitriy Kovalenko

Visual regression is one of the hardest part in UI testing. And you will likely agree that it is extremely powerful. But how it works? What the problem it is solving under the hood? Why people choose visual regression services and how we build the fastest visual regression tool in the world :)

---

## It’s not about your Assertion Library – Mark Wubben

I’ll be the first to admit: writing tests? Not all that much fun! And that’s coming from somebody who maintains a test runner in their spare time.

Once you have some tests though, you can have confidence. And once you have confidence, you can make changes. And changes are what’s needed to build awesome products.

So let’s not talk about API details, let’s talk about getting testing done. About being better engineers. About building awesome products.

---

## Beyond API Mocking – Artem Zakharchenko

Mocking is one of the best techniques to separate concerns during testing. When it comes to API mocking, we tend to either stub a request client or replace it with a mocked counterpart entirely. What we’re doing is altering the tested system so it makes requests to a different source, or doesn’t make them at all. That’s mainly because there was no better option. Until now.

In this talk, we’ll go through how to efficiently use API mocking that retains the integrity of your JavaScript application and results in more confident tests. On top of that, I’ll illustrate how to reuse the same mocks on different testing levels, as well as during development and debugging. All that with a single tool in your arsenal.

---

## The Life-Changing Magic of Tidying Up your Test Warnings – Victor Cordova

Even though we write tests for our web applications, the reality is that bugs still happen. Fortunately, many of these are easily preventable by paying more attention to the warnings from our apps. However, it's often so easy to put them under the rug and never come back until we find a bug in production, which leads to hundreds if now thousands of warnings appearing in our test output. This talk is about how to prevent this situation and how to get out of it.

---

## Testing React: A Convert’s Journey from Enzyme to Testing Library – Bonnie Schulkin

Testing Library's advantages over Enzyme for testing React:

- Opinionated framework enforces best testing practices (test behavior, not integration)
- Opinions on how to find elements (by role) encourages accessibility
- jest-dom assertions lead to simple, readable tests
- ESLint plugins help encourage best practices in real time
- test output helps locate elusive elements

There will be a few code examples, but this is largely a discussion (rather than a how-to).

---

## Get Testing out of your Tech Debt – Carly Litchfield

Technical debt and testing have a long and entangled history. Across many organizations, teams struggle to define “technical debt” and what should fall into the “tech debt” bucket. Testing commonly suffers the fate of being categorized as tech debt, and consequently isn’t prioritized. Defining tech debt, and even rebranding it, can help your team to prioritize testing and reduce the negative stigma around tech debt.

---

## Practical Web App Performance Problem Solving – Yonatan Kra

In this talk we will learn how to solve performance issues. We will learn how the JS engine works, see use cases from production and come out with practical tips that can help you to boost your app's performance by 90%!

---

## Testing React Hooks with Confidence – Radoslav Stankov

The talk will be presented as a refactoring story - will start from the messy untestable component, cover it with a brittle smoke test, and then show how to move all our react component logic into a custom hook and test this hook. Will present patterns to test things like - useState, effects, and Apollo.

---

## Security Testing for JS Apps – Ryan Severns

With StackHawk, engineering teams can run security tests against JS applications and the backing APIs to find and fix vulnerabilities before they hit production. With automated testing on every PR, you can be confident that your app is secure. Join StackHawk co-founder Ryan Severns for a quick overview of JS application security testing with StackHawk.

---

## Panel Discussion – Future of Web Testing

Panelists:

Jason Palmer (Jest Core Contributor),
Kent C. Dodds (Creator of TestingJavaScript.com),
Nancy Du (Solutions Architect @Rangle.io),
Oren Rubin (CEO @Testim).

Moderator: Yoni Goldberg (Independent Testing Consultant, the Author of Node's Largest Best Practices Collection).

---

## Panel Discussion – AppSec Testing

Panelists:

Scott Gerlach (CSO & Co-Founder @ StackHawk),
Sam Stepanyan (OWASP London Chapter Leader & Independent Application Security Consultant),
Liran Tal (Developer Advocate @ Snyk & Node.js Foundation Security Working Group),
Bar Hofesh (Co-Founder, CTO & Security Researcher @ NeuraLegion).

Moderator: Vandana Verma (Global Board of Directors - OWASP).