Automation Disambiguation 🧐

Most accessibility tooling vendors are telling the truth in their marketing. That does not mean it’s straightforward to compare apples to apples when talking about different approaches and solutions, especially when it comes to automated tools. Let’s clarify!

The Assist is the official newsletter of Assistiv Labs. We build tools to help the people who make the web make it more inclusive. 💙

The Assist publishes 3 or 4 times per year and includes our perspective on trends in web accessibility, company and product updates, as well as tips & tricks and other news relevant to accessibility teams, developers, and QA professionals.

Why is it so confusing to understand what automated accessibility tools can do?

When I (Andrew) worked in Marketing Communications at Apple, I learned that even the most far out sounding claims we made in our marketing materials were conservative and backed by hard data. If we claimed the new Macbook Pro was 10% faster, it was usually 12% faster. We’d essentially round down to what was certain to avoid any appearance of overstepping.

I believe that’s true for most accessibility vendors as well. The main exceptions are certain overlay vendors who claim one line of JavaScript will fix all of your accessibility issues in 24 hours! That’s hogwash, obviously.

Most vendors, though, are telling the truth. That does not mean it’s straightforward to compare apples to apples when talking about different approaches and solutions, especially when it comes to automated tools.

Nick describes the problem as follows in his recent article on our website, How far can automated tools go towards ensuring accessibility?

Some accessibility tooling vendors estimate that their products can catch anywhere from 30 to 80% of “accessibility issues”. But what is defined as an “accessibility issue”? How are these numbers calculated? Are they based on the percentage of WCAG Success Criteria (SCs) that the tool fully covers or do they include partial coverage? Is it simply the percentage of issues typically caught during manual audits? Do they count failures once or do they count every instance of the same failure wherever it shows up on the page?

Vendors, us included, tend to describe the impact of our solutions in terms that paint us in the best light. My pledge to our customers is that we will take an Apple-like approach to how we talk about our products: we will only make claims we can back up.

So, when we say our accessibility-first end-to-end testing service validates your flows against 58% of WCAG 2.2, A and AA (& we’re working towards 80%)? You can count on it.

Humans will always be the best judges of what is usable by humans. And, we believe the model of relying on once-per-year comprehensive audits to track the accessibility of digital products is fundamentally flawed.

We take the following approach:

1. Collaborate with you on a test plan to identify which are your highest-priority user flows
2. Conduct a human audit of those flows to find existing accessibility issues
3. Write tests that faithfully replicate how human users complete your flows
4. Exercise your tests up to 100× per day to ensure your flows are still working as expected
5. When our system identifies issues, work how you work by taking issue creation and triage off your plate as much as possible

It’s not hyperbole (because, remember, I said we wouldn’t do that) to say we think our solution is transformative. Get in touch if you’re interested in learning more!

Shining a light on people living at the intersection of Disability and other identities

In our last issue, we linked to Dr. Michele Williams’ article about Black History Month + Disability. We’re excited to continue our focus on people with disabilities who count other aspects of themselves as crucial to their identities with the following new articles on our site:

• AAPI Month + Disability
• Women’s History Month + Disability

The World Health Organization (WHO) estimates there are 1.3 billion people worldwide living with disabilities. People with disabilities are as diverse as, well, people. As Tiff (who we introduce towards the end of this newsletter) reminds us in their article on Disability Pride Month,  “Disability is the one identity that intersects with every other human demographic.”

More excerpts from their must-read article:

Intersectionality is crucial when talking about the Disabled community as it highlights how various forms of discrimination (such as racism, sexism, and classism) intersect and compound for individuals with disabilities.

Prioritizing accessibility and inclusivity in design leads to outcomes that enhance usability for everyone, and not doing so is simply bad design.

As we reflect on the conclusion of Disability Pride Month, it is the responsibility of each of us to recognize the intersectional nature of discrimination and actively work to dismantle all systems that perpetuate exclusion, ableism and marginalization.

Can we get an amen? 🙌

How Continuous Accessibility can help an organization’s accessibility maturity

Andrew presented a talk at Accessibility Camp Bay Area in May on how Continuous Accessibility can help improve a company’s accessibility maturity. He’s since published an article based on the talk and will be presenting a similar talk at the A11yNYC Meetup in November.

The gist of it is that there are a handful of proof points within the maturity models that can be bolstered by smarter use of automated testing, specifically our end-to-end accessibility-first testing solution.

At the same time, we shouldn’t just throw technology at problems. Understanding the strengths and limitations of our options helps us make the best use of limited software budgets so we can have the maximum positive impact for our users with disabilities.

Meet Tiff!

Hi 👋 I’m Tiffany Le-Nguyen (they/them), a Vietnamese/Canadian queer and disabled Senior Software Engineer at Assistiv Labs from Montreal, Canada 🇨🇦🇻🇳. I joined Assistiv Labs at the end of May 2024!

Career journey

My software engineering journey has been focused on a desire to create great, inclusive and intersectional digital experiences to enable a diverse wealth of users, including the devs building those experiences. I’ve worked at various sized companies from 1-15 person startups, to mid-sized startups such as Netlify to multi-thousand employee enterprises like ExpediaGroup.

Motivation to work on accessibility

My motivation to work on accessibility stems from my personal experiences and the challenges I’ve faced as a woman-attributed POC, queer, nonbinary, autistic, and disabled individual and of my community. I understand firsthand the importance of accessible technology and how it empowers everyone.

If you could wave a magic wand, what in the world would you change related to Disability/accessibility?

If I could wave a magic wand, I would ensure that accessibility for the most vulnerable groups (LGBTQIA+, BIPOC, etc.) is always at the forefront of design in all aspects of life, not just technology. Where innovation goes beyond simply having an accessible option; and should focus on what type of solution best meets the needs of those with disabilities. 

Fun fact / favorite foods / favorite travel destination(s)

I love Japan 🇯🇵 as a travel destination! I travel a lot with my partners and/or caregivers and Japan has been the most accessible destination (as an ambulatory wheelchair user and traveller) by far. I also have a lot of food sensitivities and Japanese food is generally always safe for me, not to mention delicious!

We hope you enjoyed this issue of The Assist! Let us know what you think by leaving a comment or sending us an email at contact@assistivlabs.com.