[Richard]: Thank you so much for inviting me here and for giving me some of your time. A little bit about me. I've been involved in the user-experience field for a little over 16 years or so. I spent time on big companies' internal teams like Dennis just mentioned.
About the last 4 or 5 years, I've been doing independent work, freelancing, contract work as well. Worked at a number of different settings, number of different companies. And I had an opportunity to work in all the different kinds of verticals with all different kinds of situations. When I'm not doing user-experience work, you'll find me outdoors.
That's me and my oldest son. We got to spend 5 days on the Appalachian trails couple of years ago in Tennessee, North Carolina area. We're on top of Big Bald Mountain there, which is a pretty cool place. On a clear day, you can see most of the southern Appalachian mountain range, which is pretty cool. If I'm not doing this, I'm on a canoe or in a tent, or some sort of place like that out with my kids.
So, with user-experience work, I find myself in a number of different settings. Up at the top there, I do a lot of strategy work. I do a lot of work with helping companies create user personas and customer journey maps. Relating this to, not only their business, but their user trying to connect those together so we can craft effective solutions together.
And then I also do a lot of wireframing work, interaction design as well. And I also do quite a bit of user research. That's me on the far right at a project recently where I was helping to design a concept for helping railway workers that are working on a railroad line be safe. So it was iPad based, set up your own network...My co-workers were jealous. I got to spend the entire day riding a train next to an engineer wearing my high vis vest, of course. I got to spend the second day out with the work crew, with the foreman, checking out the lines and the crews and the flags they were using. But the kind of work that we do in user experience, it takes you to a lot of different places and makes it kind of fun.
But what I want to talk about tonight is not just user experience, but accessibility. And what I keep finding, really, probably over the last 10 years or so, accessibility keeps coming up, it keeps becoming a part of what I do. I was telling someone earlier tonight, I've been through couple of these wars, if you will, or battles where a company is under threat of litigation. And they now realize we now have to do something about this. And I'm learning the jargon and learning how this all goes, and so it's a little bit about what I want to share.
And, so, why accessibility is important...All sorts of companies are finding themselves under litigation. I'm just going to cite one recent example. Peapod settled with the Department of Justice around A.D.A. violations, which were essentially, they were found in violation of not providing full accessibility for people with disabilities. I'm not going to read the entire settlement agreement to you here tonight. Some things I did want to highlight though: they were required to make their sites and their apps conform to WCAG 2.0 AA compliance. And they were also required, any third-party that was providing service to them, they also had to be WCAG 2.0 AA compliant as well. They needed to solicit any feedback from their customers that had disabilities. They had to contract and perform third-party evaluation and consultation. They need to perform regular automated conformance testing. They needed to train other staff, including new hires on accessibility. And any issues that were identified, they had to fix them within 30 days.
So, from my experience, this is fairly standard. It's pretty par for the course when a company is approached about accessibility issues. What they're required to do is very similar to this type of scenario. And I just have couple of companies here that have been cited over the last 15 years or so. It's a very long list. Oftentimes, it's an organization that really just wants the company to fix what's wrong. They're really not interested in money. They just want to know that you're going to take care of it. There are groups that do want money. And there are profiting from this. Unfortunately, Target was one of the best known companies that decided to fight their citation for being inaccessible and it really gave them a black eye. It was all over the news and New York Times and, so, on and so forth. And they had to settle.It didn't go over really well for them.
So, out of all this, some companies that I've worked for, I started to learn some of the jargon and lingo. Those of you who are with Dennis's accessibility meetup here, I'm just going to bore you for a second, forgive me, for everybody else in the room. Some of these acronyms and terms are really important.
The Americans with Disabilities Act was passed in 1990. And that really, from my point of view, that's sort of like the bedrock of how we dealt with accessibility issues and technology ever since then. And it essentially guarantees people with disabilities equal access. Section 508, kind of built upon that in the late '90s. It was another piece of legislation written by Congress that defined, in particular, around technology and accessibility as well. And WCAG is a series of guidelines. It's Web Content Accessibility Guidelines. That's sort of the watermark that everyone needs to hit.
And, so, anybody that does consulting in this area, WCAG is always what they point to and the guidelines were revised recently few years ago and now we're up to WCAG 2.0. And there's couple of grades they give out. Single A, double A (AA), and triple A (AAA). Most folks want you to aim for AA compliance. From what I understand, AAA is almost impossible to meet. But AA is what people are held up to.
So, with all of this going on, how are we supposed to respond to this? Well, with the clients that I've worked with, I've noticed a few patterns in terms of typical engagement. There are software programs that will scan your code and will highlight areas where your code is in clear violation. For example, not having an ALT tag for an image. Not having field labels called out. So on and so forth. That's usually sort of the ground level, to make sure that's done. Second thing that's typically done is a human evaluation. It's also a referred to as accessibility audit. That's usually done by someone that has a development background, is pretty well-versed in accessibility and the WCAG standards, and they try to draw a line between where your code exists and where you're in violation. A lot of times, they will use screen reading software.
The picture there is someone using JAWS which is the one of the popular software packages that people who are blind use. I find in many cases, the person doing the human evaluation is sighted. So they're not exactly seeing it, or hearing it, if you will, from a blind person's perspective. But typically, this is done as just sort of table stakes. And it moves on from there as prioritization exercise and what's going to be tackled.
When I have been part of these evaluations, we typically have had people doing the human evaluations deliver something like this, it's a spreadsheet. And it will call out particular issues, by standard, what the description of the issue is, what the requirement is, and how to mitigate against that. And so that becomes a conversation that's had between the consultant doing human evaluation and the IT architects and the developers about what can be done, when it can be done, in light of everything else that's going on inside of the IT Department.
So, for me, I guess, when I've been involved in these kinds of projects, where we're trying to, in many cases, in a panic, because we have a letter that's been sent out. Just a little bit of oh, my gosh, we're under the threat of litigation and we need to get this going. The question comes up, "is that all we can do?" And I'm sure for a lot of folks that I worked with on the client side, they're happy just to be compliant. So let's ...let's stop the bad news. Let's get out of trouble. I can tell the higher-ups its good, we're cool.
So what I've been a part of is conducting usability testing with people who have disabilities. So as an extra step on top of what I just described, you actually bring people in that have various disabilities. You have them use their hardware, you have them use their software, and actually step through the application. Whether it's on the web, whether it's on the mobile device, or whatever it might be. And from that, there's a great deal of value.
To give you an example, this is one I worked on not too long ago. It's supposed to find a financial services advisor from the homepage. This woman that I had here in my usability lab, she actually taught JAWS at a state school for the blind. And she was an expert, okay? So she's on the homepage of our website here. And she types in her zip code. She gets ... this is just part of the page that's returned to her. So the map there on the left is read as an image, but an image that's not labeled properly. So she's inferring it's a map. So a code scan would have said, hey, right there. Make that empty image tag, fill that out and make it a map. And you might feel you've taken care of the matter. On the right, it's an invitation to narrow your research to make get better results. So the way she was reading it with JAWs was empty image, which was probably the map, and then an invitation to narrow her search. So she automatically blamed herself. I must have fat fingered my zip code. So she went ahead and took that invitation to retype her zip code, and try again. When in actuality, this table is nested right underneath the map. And there were three folks that were financial service representatives that were within three miles of her zip code.
So, the reason why I bring this up is, if you had done a code scan, like I said, it would have caught the fact that there was an empty image tag for the map. If you had done a human evaluation with a sighted person, they might have missed this. A good person doing that would have caught it. In many cases, I call them interaction design problems or layout problems.It can only be uncovered by bringing people in and they're actually using the technology. And I found that over and over again to be the case. It's just a tremendous amount of value in adding in that extra layer of bringing in those sorts of folks to run these types of studies with.
So, if I piqued your interest in this, and you said, hey, maybe that Richard guy is onto something? Maybe I might want to try that? Well, let's talk a little bit about logistics. Because there's a little bit to think about before you start doing this kind of thing. First of all, are you going to do this testing at someone's house? Or where they work? There's advantages and disadvantages to that. Obviously, if you're going to work with people with disabilities in their natural environment, from an ethnographic point of view, that's a real win, right? You're in their natural environment. They're using their own technology. You're in their element which is great.
Some of the challenges involved with that is you're going to have to do a lot of driving around or getting from place to place. I find typically when I'm doing ethnographic research, it gets a little bit on the creepy side if there's more than two of us. It gets a little bit on the Spanish inquisition side of things when there's three or more of us in someone's cubicle or someone's home. Like who are all these people? So you're limited in how many people can tag along. You're also limited in terms of recording. A lot of folks aren't really interested in having you install software on their computers so you can record your screen activities. So you're somewhat limited to video at that point. Which isn't a major obstacle. But it's something to consider. And also liability is a concern as well. Some companies just won't let you do that. Because you're crossing the line. You're now getting into someone else's house. Insurance. What happens if the dog bites you? You trip over their couch or...Sometimes it's just a non-starter.
Another possibility is to test in a formal lab setting, a usability lab of some sort, whether that's in your building or rented facility. There's advantages and disadvantages there. Advantages, you're providing the hardware and software so you have a lot more control over it and you're able to set up the recording a little bit easier that in case. So that's a win. Some of these labs I worked in, there's mirrored glass. So you can have a whole host of people observing and the participant isn't mindful of that which is great to have as many people as possible checking these out as they're going on. Some of the negatives are getting the person to your facility. Now, if you're on public transportation line, that's great. If you're, depending on where you're situated, it could be better in one case or the other.
So, for those of you that have done user-experience work, if you've been part of a usability test, the question can come up how is this different from a typical usability test when you're running a usability with sighted people with certain application. I have found through experience, you really have to reduce your expectations drastically in terms of what you're going to be able to cover. Most often, you have a much smaller scope. You're only going to be able to get through a few tasks. And it's a much slower pace. It's much more conversational. And depending on how buggy or inaccessible your software is, it could take quite awhile. So, what I've found is when I recruit 5 or 6 people for a day and I have 10 tasks to cover, I might have to split those tasks up amongst maybe two people at a time. I only get to do those particular tasks. Which I know violates every usability paradigm out there to only run couple of task with two people. But you do start seeing the same thing over and over again very quickly. Moving from A to B. So getting the person around. I've learned from having work with people in this community who are very, for the most part, very sweet and very forgiving of my mistakes.
There's sort of a protocol that you have to develop. So, if you're going to meet the person, for example, out in from of your office, and they're blind, you offer your elbow, and say "would you like me to take you where we're going?" Please grab my elbow. And they'll grab the back of your elbow. And as you are walking, you have to describe where you are going. Say, "we're going to go about twenty feet, we're going to take a hard right, And I am going to open a door." So you are just describing the territory in front of them and what you're doing. So, you're not grabbing their hand, you're offering your elbow, and you're guiding them. That usually works out really well. You do want to make sure the doors of your building are wide enough to handle a wheelchair if you need to. Find out ahead if time if there are service animals that are involved. If you're like me, and you're allergic to dogs, that can become an issue. For me, if I just load up on medication, I'm good. I can make it through. I have had a service dog put his nose on my lap for tests. Which was a little weird, but we got through it. There is also another protocol on how you deal with the animal. You're really not supposed to scratch them between the ears and pet them like a regular dog. That's a work animal and they're specifically trained to serve the person they're assigned to and when you interact with them that way, you're pulling them out of that mode, if you will. So you have to kind of keep your distance on the service animals. Even though a lot of times, they're really awesome dogs and they're doing really amazing things.
So just a few other things to consider. Also, if you have a facility in your building, it's best to clear the tenant thing with your security people or whatnot. Hey, there's at least half dozen people coming throughout the day. Some of them have service animals. Keep an eye out for them.
In terms of finding these participants. There are local and national Federation for the Blind chapters all across the United States. Those are a great place to start. There's a lot of special interest groups that use assistive technology. Universities, particularly the larger ones are a great place to go. Most the universities and the governments have been under much stricter rules around accessibility for a much longer period of time. So most larger universities have an office for accessibility with staff that helps train the different departments on how to make their websites and mobile apps accessible. Most of these universities also have computer labs, on-site with all the technology they need. And, so, they can definitely point you in the right direction as far as finding folks.
Unfortunately, I found the unemployment rate amongst the people with disabilities is extremely high. So usually having them come and help you is usually not a huge barrier of them not being interested. And there's also among the community, I found out there also a lot of positive energy. They want to come in and they want to help you fix your stuff. They really appreciate the fact that you're reaching out to them as well. Which is really, really great.
Here's a project I was on a few years ago where we partnered with the local University. It was great, because I brought in some middle-aged folks to my lab that had accessibility issues. And they were using screen reader technology for the most part. They were also using screen magnifying software as well. And I partnered with this local University. University was really well-known for being open to people with accessibilities in the student body. And we had several other students step through the same website as well. What I wasn't anticipating, it almost wasn't a fair fight. Because the blind students, when they came to that University, they had an introductory JAWS course and they came out of that screen reader course with just like super powers. They knew all sorts of power moves, and different power user techniques. And there wasn't anything we could throw at them that they couldn't master. Whereas, a lot of the middle age folks that I was dealing with, they were more self-taught.
And what I also discovered through that and other projects is, a lot of times, folks, hardware and software isn't cheap. So if it works, that's good. I don't care if I'm couple of versions behind sort of thing. Whereas, the students in that lab, they were on the latest stuff and they knew all the stuff. And what actually came out of that was an interesting discussion I had with the professor. I got hooked up with her. And this student worked on a separate exercise, where we were essentially evaluating someone's proficiency with the JAWS screen reader we were placing them as beginning, medium, and advanced and trying to see what issues were uncovered through that, which was sort of a separate effort. Which is a whole 'nother discussion.
[Chuckles]
And I'm not sure how that relates to anything involving compliance, but it was just an interesting take on all that.
So what I did want to do though, I did want to show you an example video. This is a spoiler alert. It's a 5-minute video but I'm going to tell you how the story ends. So there's a friend of mine. And I'm having him step through a newly redesigned website at the Ohio State University on seeking financial aid. And they recently had a vendor come in. And the vendor was very well-versed in accessibility and they redesigned that with that in mind and it works great. And at a particular point, he's interested in a particular scholarship. And he goes to another website and that's not so great. Because that site has not been given that sort of treatment.
So let me queue that up. Let me see if I can get this to go.
[Video]
[Screen reader reading extremely fast on video]
Student Financial Aid Ohio State University Mozilla Firefox
Student Financial Aid Ohio State University document header tab control
Securely submit forms tabs two of three
Earn money while in school tab three of three
Navigation landmark skip to content link
The Ohio State University visited ...
List with six items Help link
Buckeye
Map
Find People link
Web Mail link
Search Ohio State link
Out of list banner land
Mark main landmark
The Ohio State University visited link
Student Financial Aid visited link
Search grouping
Search Student Financial Aid, edit
Blank
Navigation Landmark Search Landmark Search button
Incoming Freshmen submenu one of seven
[Richard]: So, a little bit more background here.
He's using a screen reader. It's highlighting it red where his area of focus is. He's a power user. He's a developer as well. And he purposely slowed down the speed of the slow reader so I can understand it.
[Laughter]
[Richard]: A lot of these folks that are really practiced on the screen reader, I don't know if it's because one sense shuts down, another sense becomes heightened. It's reading it so fast it sounds like mumbled. But he's purposely slowing it down for me.
[Dennis]: And to give you an idea, that's about 40 to 50 words per minute and many users run at about 300 to 400 words per minute.
[Richard]: Yeah, so he's being cool with me at this point.
[Laughter]
[Video]
[Screen reader reading extremely fast on video]
menu one of seven
exchange students submenu two of seven
transfer students submenu three of seven
[Voice of test user on video]: And I want to look under that.
[Screen reader on video]: Transfer students with four sub-items one of four
File the FAFSA one of four
Cost of attendance two of four
Types of aid three of four
Resources four of four
[Voice of test user on video]: And I'm looking for resources, so I can very easily ...
[Screen reader reading extremely fast on video]
Heading apply for a transfer student with four subitems two of four
[Voice of test user on video]: navigate through all of ...
[Screen reader reading extremely fast on video]
Resources four of four
Resources transfer students The Ohio ...
[Voice of test user on video]: Yep, so I'm just using the arrow keys to navigate through that menu once I've landed on that kind of large mega-menu control.
[Richard during user test on video]: So you're going to go ahead and search for the different types of aid available.
[Screen reader reading extremely fast on video]
Banner landmark main landmark resources transfer heading ...
Contact Us and Forms submenu seven of seven
Heading
Forms one of two
Forms
Heading
Co ... Forms one of ...
[Ding]
Search grouping
Search student financial aid
Navigation landmark search landmark search
Incoming freshman submenu
Heading
File the FA... Resource ... heading ... important date ... step ... apply ... heading
Accept aid incoming heading
Apply for aid heading
About ... cost of attendance three of five
Types of aid four of five
Types of student financial aid incoming freshmen the ohio state university
Navigate
[Voice of test user on video]: So I'm on incoming freshman
[Screen reader reading extremely fast on video]
Content info landmark ... Navigation ... Banner landmark ... Main landmark
Incoming ... Incoming freshmen students
Step number two ... incoming freshmen sub-menu one of seven
Current and campus change students ... transfer students sub-menu three of seven
Heading
About aid transfer students with ... file ... type of aid three of four
Types of student financial aid transfer students the ohio state university
Navigation landmark heading level two ohio state nav bar banner landmark main landmark
types of aid transfer students heading level one
Heading level one transfer students
link transfer
link students
link students
step number one
link about aid
main landmark navigation landmark scholarships heading level four link
link a scholarship is financial aid that does not have to be paid back.
grants headline level four a grant is financial aid
loans heading level four loans are financial aid that
private loans heading level four private loans are designed to assist students and their
federal work study heading level four federal work-study is a need-based aid that must
content info landmark the ohio state university
[Richard during user test on video]: *So where would you go from here?
[Voice of test user on video]: So, say if I wanted to learn about one of these different types of aid.. so if I was interested in...
[Screen reader reading extremely fast on video]
Main private loans heading level
loans heading four link
grants heading ... scholarships heading level four link
scholarships ... transfers ... students ... the ohio state university
banner landmark ... main landmark ... scholarships ... transfers ... students ... heading level one
Heading level one transfer students ... main landmark special scholarships heading level two
[Richard]: So he's looking for a particular scholarship at this point he wants to apply for.
[Screen reader reading extremely fast on video]
The university offers hundreds of special-eligibility scholarships with a wide variety of eligibility criteria.
To apply each year ... by the priority date of February 1
Regional campuses heading level two
Regional campus students can explore scholarship opportunities at link Lima, link Mansfield, link Marion
Choose Ohio First Scholarship Program heading level two
[Richard]: I know this is a controversial topic, but I'm going to bring it up anyways. This is one of the problems with embedded links. Because he can't use a key stroke to pull up the links on the page. But otherwise, he has to read through all that paragraph to get to the different links. He's just reading through it straight up.
[Screen reader reading extremely fast on video]
Ohio First Scholarship Program
universities are our business
partners that have developed innovative academic programs to recruit and ...
External scholarships heading level two
Many churches, professional, civic or service organizations,
list with four items bigfuture link bullet link College Scholarships bullet Ohio State does not post individual
[Richard] So now he's heading out to an external site.
[Screen reader reading extremely fast on video]
Choose Ohio First Scholarship Program heading level two
The Choose Ohio First Scholar...
Choose Ohio First Ohio Higher Ed Mozilla Firefox
out of list search form heading level two
main landmark you are here heading level two
link home link initiatives
heading level one Choose Ohio First
heading level one link Facebook
heading level one link Tweet
heading level one link Cuff
heading level one link Cuff
heading level one link clickable
background
link Annual Reports
link Why STEM
Participating Programs link
Program link
Resources link
list with four items Choose Ohio First Academic Major link
Choose Ohio First 2016 RFP Directions link
Choose Ohio First 2016 RFP Scoring Rubric link
Choose Ohio First 2016 RFP Webinar Recording link
while buffering and downloading
out of list heading four Choose Ohio First Proposals for...
eight million new jobs will be...
list with one items Choose Ohio First
Participating Programs link heading level two
[Voice of test user on video]: So I don't know...
[Screen reader reading extremely fast on video]
Choose Ohio First out of list heading level two Agency Initiatives
list with twenty items white bullet 3-Year
[Richard during user test on video]: So do you know how you would go ahead and apply?
[Voice of test user on video]: It's probably in one of these pages somewhere, but yeah that front page doesn't really make it super clear.
[Screen reader reading extremely fast on video]
out of list quick links heading level two
list with five items
[Richard]: So there's probably around 40 or 50 links on this page and it's just overwhelming. So this is sort of the bad example. For the most part, the first website was fairly straightforward and it went and announced 1 in 7 of the tab, and 2 of 7. This one is just a giant link farm and all he wants to do is find the application for it. So, he could get through this. It just wasn't ideal per se.
So in conclusion, it's my belief is that usability testing with people with disabilities, highlights issues that are difficult to uncover any other way. They highlight unique issues that other methods just won't address necessarily. And I'm not dismissing code scans. I'm not dismissing human evaluations. I believe all three can work together and give you the best solution altogether.
I'm also under the belief that when you focus on accessibility, ultimately it improves the user experience. You saw the difference between those last two websites we just walk through on the video. The first one had a much simpler UI to it. And any website or application or mobile app that I've been part of that's under gone accessibility review, it tends to simplify the interface. It tends to strip out many of the options. And all of those are tenets of user-experience, whether you're a content writer, an interaction designer, or user researchers, those are all the things we can get behind.
So from my standpoint, accessibility really serves user experience very well for all people. And ultimately, I believe that where this comes down to is whether or not the client wants to be compliant or whether or not they're interested in delighting the end-users no matter who they are. And I understand the reason to try to get out from under the throng of the litigation, to try to get that off to the side, but ultimately, we do serve a larger population. And what I've also found is that it really helps with client relationships.
One last story. I was working for a financial services company. And they held the public retirement funds for a very large state in the country. They were the custodian for that. And for that individual state, there was a woman who reported straight to the governor and she was in charge of accessibility. And she caught wind of the website that my client had for the employees of that state, the government employees. And she was not happy. She was very upset. And it was doing everything wrong you could imagine. Not only for people who were blind, but everybody else in between. It was just not a good website.
And when they came on-site, and they saw the usability lab and saw what they were doing, to try to help mitigate what was wrong, it really made a good impression on her from a client perspective. That we were going to that extent to try to make it right. So I'm not saying that's the only reason you wanted to do this, to try to win over a client. But it really does show you're all in when you do something like that, that you're really committed to making something as useable as possible for as many people as possible. And with that, I will open the floor up for any questions you might have.
Yes...
[Attendee]: Hi, I really liked your presentation. I had a question about in testing. Have you done remote testing?
[Richard]: Right, remote usability testing.
[Dennis]: Richard, could you repeat the question?
[Richard]: So the question was have I ever done remote use ability testing with people with disabilities? Actually my friend that was in video tape was going to do that with me here tonight. He told me that most of the web conference software, Skype and the like are not accessible. So essentially, he needs to have someone with him. His wife is sighted and he's friends are sighted. But they would need to be with him in order to make that go. He told me he'd be happy to do that, but then it all fell apart. And I'm going to bring it over to his office this weekend and do that. Because actually, the videos I've done for clients are all under confidentiality agreement so I can't show you. But it is possible. I just think it would take a sighted person on the other end to help them out if things didn't work out. I'm a big fan of remote usability testing in general. Because I find that I get better show rates and people are much less likely to bail. They're using their own hardware and software which is great. I can see what their screen resolution is set at. If there's noise in the background, that's great. That's the way you're trying to get things done with whatever it is you're using. But, yeah, potentially in that case, for someone with visual handicap, they might need a visual person as well. That's a great question.
Yes?
[Attendee]: [Away from mic]
[Richard]: Yes, so the question was how many participants do you need?
In general, there's a rule of thumb in our field, that goes back many years. You need a minimum of 4 or 5 people to see the majority of issues coming up again and, again. This is somewhat anecdotal, but I haven't found I need quite that many. Typically, if something is broken, for the first person, it's broken for the second and third as well.
It kind of gets to that issue I brought up a moment ago about the proficiency that the people have with the tool and why they're not able to overcome that. Which I don't think any of the legislation deals with at all whether you're making it accessible for people ... proficiency or not. But that's another whole can of worms for another night. I have found that usually after about 2 or 3 folks, we can usually see the same patterns.
Yes? I'm sorry, back there.
[Attendee]: [Away from mic]
[Richard]: The question was, when did I begin accessibility testing?
I was working for a software division within IBM. And my user experience manager at the time realized we had not done much with that and said, here you go. Go do it. And I met this amazing woman. She was blind. And she worked in one of our offices. And there, at that point in time, they pretty much said have her look at it. Have her look at it. So she showed up to work everyday and they just had her trying to use whatever software they were working on and she would tell them what worked and what didn't work. So it was very informal, but it was great that they were doing that. And that's how I got introduced to it. I met some really amazing people.
IBM has an accessibility center. They have spun out a whole consulting group out of that as well that does consulting with all sorts of companies working around accessibility as well and they're really committed to it. Actually, I went with that woman to the CSUN conference. I was just talking to Dennis about this. There's this really great disability conference out near the L.A.X. airport. And for many years, they had it in the same two hotels. And the participants wouldn't let them move, because some of the participants were blind and they knew how to get everywhere. I go down to the elevator, I go 15 feet forward to the right and there's the bar.
[Laughter]
[Richard]: Yeah, I got go with her to that conference. And you get to meet Stevie Wonder there too because he lives in L.A. And they have an amazing vendor area, you see some amazing vendors coming up with these crazy futuristic pieces of hardware and software for accessibility that are just mind-blowing. And that's what Stevie comes for.
[Dennis]: This past year at CSUN, Stevie was checking out the device that was similar to Echo. So he was talking in to it and listening to what it was returning.
[Richard]: And his body guard was 6-foot 6 and about 400 pounds?
[Dennis]: I thought it was his bass player.
[Richard]: Yes?
[Attendee]: So you're talking about usability testing and you would go at slow pace?Do your metrics change? Like is it more informal or how do your metrics change learnability?
[Richard]: The question was around usability testing and what metrics apply around learnability and how those change. In other settings, I have used time-on-task and error rates. And I find those don't apply quite as well. Sort of the slower pace piece I was talking about as well, you need to allow time for them to get comfortable. So if they're coming into your lab, and they have to adjust the speed setting. They have to figure out how to use your mouse and there's a lot of things that aren't quite the way that they prefer it. Sometimes you're using a different version of the software. So in general, with that, plus the general speed of it, I feel like there's a lot of things that don't apply that I might use normally in terms of evaluating something. And definitely it tends to be much more conversational and much more informal. But as they're going through, trying to complete different tasks, you're usually starting to see the same issues come up again and, again, with the participants.
I don't know if I answered your question. I tried...
[Laughter]
[Richard]: No, that was a great question though.
Did you have a question?
[Attendee]: I had a question. When you're doing usability testing, you talked a lot about users who are blind, but there are people with variety of types of disabilities, handicaps, and different challenges. Do you ever do usability testing for people with physical disabilities? Or hearing impairments? Or kind of a wider range of disabilities?
[Richard]: That's a great question. The question is, with usability testing, do you test with people that just have visual handicap or others as well? That's a great question. To date, the clients that I worked with made the decision whether it was a good decision or not, but they could master the vast majority of issues if they focus on people with visual handicap. Whether that's right or not, I think that's open for debate. And it would probably take a whole 'nother hour presentation on that. But to date, that's been the focus. I have been working with some different clients of late that are taking a different tack. And I'm looking forward to try to bring in people who have motor issues as well. I think there's definitely a whole host of disabilities that we could bring in that I think with this current client I have, we're definitely going to need to address that. And what's great about having a relationship with the university is, the students, they span all those disabilities It's a really easy way to quickly find people that have a variety of disabilities that you can tap into.
That's a great question.
Yes?
[Attendee]: I have a follow-up question to that. So when you're testing with people, like for visual, you also have, like, four or five people use a screen reader and four or five people on magnifier?
[Away from mic]
[Richard]: Right, So the question was in terms of recruiting participants, do you split them up 4 or 5 with a screen reader versus screen magnifier? I think a lot of it depends on your budget and the time to go get through. Obviously, the more participants, the more money you have to spend and longer it takes. What I have found in general is if I have code on a prototype, if it's already been released, this is post production testing and we're close to a production release, that's when it makes sense to bring in people that have screen readers. When I have folks probably more in a validation stage where we have the finished visual design, that's when it makes sense to bring in the folks to have the screen magnifying type issues. So I wouldn't necessarily bring 10 people in altogether and half of each in one setting. I would try to space it where it makes sense. Because I'm trying to feed information to my designer and my developers to change things as rapidly as possible. If I have something that's finished visually, but doesn't have code behind it, we can address that before we jump to the next stage.
I don't know if that made sense or not?
[Attendee]: Yeah. The last part is another question I had.
[Richard]: That's a great question.
Yes?
[Attendee]: My question is just some observations ...
[Away from mic]
[Attendee]: Carousel for an example.
[Away from mic]
[Richard]: Patterns that create obstacles for people with disabilities?What I've noticed, and something that someone shared with me from IBM and it made a lot of sense. And he shared this with me about 10 years ago and I think it's still true. Because I just had a conversation with somebody this week about the developers on the team are falling in love with this framework from Facebook called React. And they feel like they have to develop that in order to get the most talented developers to work on it and they get the "cool factor" with what we're developing. But when I bring in my accessibility consultant and we're working together, it just forces the code that comes out of it as inaccessible. And, so, what I've seen play out over and over again is the technology leads, the accessibility catches up. And by the time you figure it out, we've already moved on a generation or two ahead. For example, when Flash came out, that was the deal. You could demand $100 an hour if you can develop cool things in Flash. And, so, tons of hours and money were spent to make Flash accessible. Does anybody care about that today? No. We are couple of generations past that now. We've gone to HTML 5. Twitter, bootstrap. We're in React. We've moved past that. And, so, it's this constant catch up where the technology gets out in front, and then finally we figure out the accessibility and we've already -- that's more of what I've seen. It's just really hard, if you want to develop the latest and greatest technology to make that accessible.
Yes?
[Attendee]: [Away from mic]
[Richard]: I'm not a coder. So I'm probably not the best person to ask, but I do know that if you are going to make something,just lockdown type accessible, typically you're giving up a lot for the sighted community. There's always a tradeoff.
[Dennis]: There's been articles, recently, if you Google articles on frameworks and accessibility, they have actually done a rundown. Each framework has its own issues. But the general problem is, especially, when you're dealing with custom code, custom elements and such, you know, if you have a form button, it has everything you need to do what you need to do. If you create your own button using custom elements and such, you have to bake that stuff in. You have to bake in what it communicates to assistive technology. How it acts when you use a keyboard. So there's nothing wrong with frameworks. But just know that depending, there's an overhead for accessibility.
[Richard]: Yeah, and I think Facebook, for whatever it's worth, they're probably ahead of everyone else. I'm making React accessible, so my client is now making calls to California to try to figure it out. Because they have accessibility people on staff and they're obviously doing something beyond what the run-of-the-mill folks are doing already.
Yes?
[Attendee]: What advice do you have for a young designer who may not be able to work for a company that values accessibility but, able to implement -- what are the ways to implement from the bottom up? If the company isn't valuing accessibility outright?
[Richard]: So the question is how can someone work with a companythat is not ready to take on accessibility? And in a true forthright way? I've definitely been in that situation. I was telling someone earlier, I did some work for an insurance company. And the breadwinner there was auto insurance. And myself and some other people were trying to get things done right with accessibility, we were laughed at, because they said blind people don't buy car insurance. That's fine. But that's where all the investment was. And yet, this company still sold renter's insurance. So that code migrated down to renter's insurance. So if you're here, why not fix the mother ship while you're fixing everything else, in my mind. So essentially, we did a lot of work around making presentations, showing what was accessible and not accessible. Particularly around the things that are publicly available. That's how we got a chance to go work with that University. We were able to have the students run against the website, the screen readers, because it was already outside the firewall, they didn't need VPN access, so we used that along with highlight reels to try and build awareness. And it wasn't until they got in trouble with violation compliance letters and clients themselves being less than happy that we finally started getting traction. So you're kind of laying the ground work, if you will, for the bomb to drop.
[Laughter]
[Dennis]: We've got time for one last question.
[Richard]: One last question. Yes?
[Attendee]: So my question is, well, when does it make sense to bring in usersinto accessibility usability evaluation? Because during the auto-checks, like the human evaluations and you catch these human mistakes. So does it make sense to bring in someone else? And when you do run a session, do you bring in one or two key... And just have them in that one test?
[Richard]: Ok, so the question was...
[Dennis]: Go for it.
[Laughter]
[Richard]: When does it make sense to bring in people who are visually handicapped for usability test and does it make sense to bring them in with in sighted users as well? Yeah, that's a great question. If you are, obviously if you are in a compliance situation, I think it makes sense if you are going to do that human evaluation, the code scan, trying to get yourself up to compliance as quickly as possible. I would wait until they try to fix the majority of those issues before we then try to test with people who have disabilities. In development efforts, I think it makes the most sense to bring in people with screen magnifiers as early as possible when you have some visual comps. You've got a visual prototype to show them, particular around color contrast issues As well as once you have functioning code that a screen reader can read, bring them in as soon as possible.
In terms of bringing them in with sighted users as well, another technique that I've worked with with some of my co-workers in the past, is something we called garage band testing. And that is set up for an agile methodology. Because one of the challenges of agile for a UX person is you're constantly spitting out small bits of work. And you never have enough to build a whole usability test around, until it's too late. And so with garage band testing, we would take different components of that that's coming out of the agile work and we'll run one participant against three small projects over an hour, so you're getting the feedback sooner.
And I have experimented that particularly if there's finished code, and bringing in people who are blind at that point as well to try to get that input as quickly as possible. With Agile, hopefully you can throw your issue in sooner to get it addressed.
[Dennis]: Thank you very much!
[Applause]