WEBVTT 00:00:00.000 --> 00:00:02.410 align:center line:84% BRIANNA BLASER: Stefik, I'm turning it over to you. 00:00:02.410 --> 00:00:04.810 align:center line:90% ANDREAS STEFIK: Sounds good. 00:00:04.810 --> 00:00:07.320 align:center line:84% OK, so I have the pleasure of this morning, 00:00:07.320 --> 00:00:10.410 align:center line:84% as people are still kind of logging on a bit, 00:00:10.410 --> 00:00:13.290 align:center line:90% of introducing Mr. Neil Brown. 00:00:13.290 --> 00:00:17.160 align:center line:84% Neil lives in the UK, and has been 00:00:17.160 --> 00:00:19.080 align:center line:90% doing some really fantastic-- 00:00:19.080 --> 00:00:21.150 align:center line:84% I would actually argue, seminal work 00:00:21.150 --> 00:00:26.622 align:center line:84% on understanding at scale how people code. 00:00:26.622 --> 00:00:28.080 align:center line:84% A lot of the work that they've done 00:00:28.080 --> 00:00:30.390 align:center line:84% is on Black Box, which some of you may know. 00:00:30.390 --> 00:00:33.300 align:center line:84% But they've also done some really amazing work 00:00:33.300 --> 00:00:36.283 align:center line:84% on these crazy little tools called Stride, and Greenfoot, 00:00:36.283 --> 00:00:37.950 align:center line:84% and all sorts of things that he probably 00:00:37.950 --> 00:00:39.870 align:center line:90% knows better than I do anyway. 00:00:39.870 --> 00:00:43.140 align:center line:84% So I find Neil's work pretty inspiring, broadly, 00:00:43.140 --> 00:00:46.240 align:center line:84% and I'm really glad that he was willing to do this talk for us. 00:00:46.240 --> 00:00:48.690 align:center line:84% So at that point, I will be quiet 00:00:48.690 --> 00:00:50.890 align:center line:84% and, I will allow Neil to do the talking. 00:00:50.890 --> 00:00:53.490 align:center line:84% So thank you again for joining us, Neil. 00:00:53.490 --> 00:00:53.990 align:center line:90% OK. 00:00:53.990 --> 00:00:55.323 align:center line:90% NEIL BROWN: Thank you very much. 00:00:55.323 --> 00:00:58.355 align:center line:84% OK, so I just need a moment to share my screen. 00:00:58.355 --> 00:01:02.250 align:center line:90% 00:01:02.250 --> 00:01:05.142 align:center line:84% OK, so hopefully you're looking at my slides. 00:01:05.142 --> 00:01:06.600 align:center line:84% BRIANNA BLASER: It looks beautiful. 00:01:06.600 --> 00:01:08.267 align:center line:84% NEIL BROWN: Someone give me a thumbs up. 00:01:08.267 --> 00:01:09.310 align:center line:90% OK. 00:01:09.310 --> 00:01:09.810 align:center line:90% All right. 00:01:09.810 --> 00:01:11.700 align:center line:90% OK, so my name is Neil Brown. 00:01:11.700 --> 00:01:13.710 align:center line:84% I'm from King's College London in the UK, 00:01:13.710 --> 00:01:15.960 align:center line:84% and I'm going to be talking to you about, essentially, 00:01:15.960 --> 00:01:18.810 align:center line:84% block based programming, how at the moment 00:01:18.810 --> 00:01:22.750 align:center line:84% it's kind of inaccessible and what we could do about it, 00:01:22.750 --> 00:01:26.662 align:center line:84% and some of the work that we've been doing towards that. 00:01:26.662 --> 00:01:28.870 align:center line:84% So I'm just going to start with some acknowledgments. 00:01:28.870 --> 00:01:31.870 align:center line:84% So a lot of the background research for this talk 00:01:31.870 --> 00:01:35.093 align:center line:90% was done by Vaibhav Patel. 00:01:35.093 --> 00:01:36.760 align:center line:84% The design of the tools was done jointly 00:01:36.760 --> 00:01:39.130 align:center line:84% with Michael Kolling and Pierre Weill-Tessier. 00:01:39.130 --> 00:01:43.020 align:center line:84% Some of those might be on the call at some stage. 00:01:43.020 --> 00:01:44.770 align:center line:84% And some of the development work was 00:01:44.770 --> 00:01:47.500 align:center line:84% done by a few undergraduate students of ours. 00:01:47.500 --> 00:01:49.675 align:center line:84% And I just wanted to acknowledge and Andy 00:01:49.675 --> 00:01:53.260 align:center line:84% Stefik, who we just pat each other on the back all day long. 00:01:53.260 --> 00:01:56.050 align:center line:84% He's been very helpful and encouraging for us 00:01:56.050 --> 00:01:59.220 align:center line:90% to take on this work. 00:01:59.220 --> 00:02:02.030 align:center line:84% So what I'm going to be talking about today is accessibility, 00:02:02.030 --> 00:02:03.590 align:center line:84% but of course accessibility is huge. 00:02:03.590 --> 00:02:06.240 align:center line:84% There's all kinds of different parts of accessibility. 00:02:06.240 --> 00:02:08.900 align:center line:84% So I'm going to focus in on vision impaired users who 00:02:08.900 --> 00:02:10.130 align:center line:90% require a screen reader. 00:02:10.130 --> 00:02:11.660 align:center line:84% And most of the talk is just going 00:02:11.660 --> 00:02:14.420 align:center line:84% to be about, essentially, screen readers 00:02:14.420 --> 00:02:17.138 align:center line:84% and how we get them working with blocks. 00:02:17.138 --> 00:02:18.680 align:center line:84% So for those of you who were unaware, 00:02:18.680 --> 00:02:21.380 align:center line:84% and I wasn't really when I started out on this, 00:02:21.380 --> 00:02:23.780 align:center line:84% there's a variety of different levels and types 00:02:23.780 --> 00:02:25.293 align:center line:90% of vision impairment. 00:02:25.293 --> 00:02:26.960 align:center line:84% So to give an example I'm familiar with. 00:02:26.960 --> 00:02:30.810 align:center line:84% My neighbor has age-related macular degeneration. 00:02:30.810 --> 00:02:33.320 align:center line:84% So he's legally classified as blind, 00:02:33.320 --> 00:02:35.128 align:center line:84% but if I take him to the supermarket, 00:02:35.128 --> 00:02:36.920 align:center line:84% he can push a trolley round without hitting 00:02:36.920 --> 00:02:38.210 align:center line:90% anyone or anything. 00:02:38.210 --> 00:02:40.340 align:center line:84% So he has that level of vision, but he 00:02:40.340 --> 00:02:41.600 align:center line:90% can't read any of the signs. 00:02:41.600 --> 00:02:43.123 align:center line:84% He can't read any of the packaging. 00:02:43.123 --> 00:02:45.290 align:center line:84% So there's all kinds of different vision impairments 00:02:45.290 --> 00:02:48.583 align:center line:84% that have different effects for different people. 00:02:48.583 --> 00:02:50.000 align:center line:84% So I'm going to particularly focus 00:02:50.000 --> 00:02:52.340 align:center line:84% on the sort of actual rarer edge of the spectrum, which 00:02:52.340 --> 00:02:55.490 align:center line:84% is people with no vision, just because that's sort 00:02:55.490 --> 00:02:56.880 align:center line:90% of the most difficult problem. 00:02:56.880 --> 00:02:59.930 align:center line:84% And if we can make progress on that area, then generally, 00:02:59.930 --> 00:03:03.890 align:center line:84% we'll have benefits for people with other levels of vision 00:03:03.890 --> 00:03:06.630 align:center line:90% impairment. 00:03:06.630 --> 00:03:08.780 align:center line:84% So let's talk about the current state 00:03:08.780 --> 00:03:10.640 align:center line:90% of things and accessibility. 00:03:10.640 --> 00:03:12.470 align:center line:84% So I've got various demo videos that I'm 00:03:12.470 --> 00:03:16.560 align:center line:84% going to show you involving screen readers. 00:03:16.560 --> 00:03:17.285 align:center line:90% So I set out-- 00:03:17.285 --> 00:03:19.160 align:center line:84% I thought, OK, well, I'll start by making one 00:03:19.160 --> 00:03:20.720 align:center line:84% to show you what text programming is 00:03:20.720 --> 00:03:22.130 align:center line:90% like with a screen reader. 00:03:22.130 --> 00:03:24.350 align:center line:84% And already that turned out to be a lot more 00:03:24.350 --> 00:03:25.670 align:center line:90% difficult than I thought. 00:03:25.670 --> 00:03:29.630 align:center line:84% So I tried IntelliJ, which is the IDEA I usually use. 00:03:29.630 --> 00:03:30.590 align:center line:90% And it was silent. 00:03:30.590 --> 00:03:31.970 align:center line:84% I couldn't get it to say anything 00:03:31.970 --> 00:03:33.050 align:center line:90% with the screen reader. 00:03:33.050 --> 00:03:35.570 align:center line:84% And I think it's something to do with the Java access 00:03:35.570 --> 00:03:37.850 align:center line:90% bridge, but technical problems. 00:03:37.850 --> 00:03:40.310 align:center line:84% So I thought, OK, I'll try Visual Studio Code. 00:03:40.310 --> 00:03:42.650 align:center line:84% It just said blank on every single line, 00:03:42.650 --> 00:03:45.020 align:center line:84% like actually read out blank, blank, blank, 00:03:45.020 --> 00:03:46.150 align:center line:90% as I went up and down. 00:03:46.150 --> 00:03:46.790 align:center line:90% So I went OK. 00:03:46.790 --> 00:03:48.380 align:center line:90% I can't use that. 00:03:48.380 --> 00:03:51.110 align:center line:84% I tried Visual Studio, sort of, full Visual Studio. 00:03:51.110 --> 00:03:55.010 align:center line:84% It gave several errors initially once I had NVDA set up, 00:03:55.010 --> 00:03:56.445 align:center line:84% but eventually an update fixed it. 00:03:56.445 --> 00:03:58.070 align:center line:84% So this just kind of gives you a flavor 00:03:58.070 --> 00:03:59.778 align:center line:84% of what it's like to try and actually get 00:03:59.778 --> 00:04:00.920 align:center line:90% screen readers working. 00:04:00.920 --> 00:04:02.810 align:center line:90% Like I'm a programmer. 00:04:02.810 --> 00:04:05.882 align:center line:84% I have full vision, and I'm sort of running into all 00:04:05.882 --> 00:04:06.840 align:center line:90% these kind of problems. 00:04:06.840 --> 00:04:08.257 align:center line:84% You could imagine how difficult it 00:04:08.257 --> 00:04:11.090 align:center line:84% would be for somebody who's maybe new to programming 00:04:11.090 --> 00:04:13.880 align:center line:84% and not sort of very technical to actually even just get 00:04:13.880 --> 00:04:15.710 align:center line:90% these things going. 00:04:15.710 --> 00:04:17.563 align:center line:84% But I did manage to make you a video, 00:04:17.563 --> 00:04:18.980 align:center line:84% so I'm just going to show you what 00:04:18.980 --> 00:04:22.940 align:center line:84% it sounds like as you move the cursor around within a text 00:04:22.940 --> 00:04:23.510 align:center line:90% based editor. 00:04:23.510 --> 00:04:27.065 align:center line:90% 00:04:27.065 --> 00:04:27.732 align:center line:90% [VIDEO PLAYBACK] 00:04:27.732 --> 00:04:33.950 align:center line:84% SCREEN READER: Slash slash program uidof 689. 00:04:33.950 --> 00:04:35.520 align:center line:90% Begin data scope. 00:04:35.520 --> 00:04:37.270 align:center line:90% Blank. 00:04:37.270 --> 00:04:40.830 align:center line:84% Slash slash attach shader shader 687. 00:04:40.830 --> 00:04:44.760 align:center line:84% NV exec GL GL attach shader program uid 689. 00:04:44.760 --> 00:04:47.630 align:center line:90% shader uid 687. 00:04:47.630 --> 00:04:55.500 align:center line:84% Space space space N, V. Line E, X, E, C, line, G, L, 00:04:55.500 --> 00:04:59.324 align:center line:90% left paren, G, L. 00:04:59.324 --> 00:05:00.300 align:center line:90% [END VIDEO PLAYBACK] 00:05:00.300 --> 00:05:01.350 align:center line:84% NEIL BROWN: OK, so that's the end of that bit. 00:05:01.350 --> 00:05:04.140 align:center line:84% Can I just get a quick thumbs up from anyone who-- just check 00:05:04.140 --> 00:05:06.120 align:center line:90% you can hear the video, OK? 00:05:06.120 --> 00:05:07.570 align:center line:90% Because, OK. 00:05:07.570 --> 00:05:10.300 align:center line:84% Just want to check my sound was shared properly. 00:05:10.300 --> 00:05:12.780 align:center line:84% So that's kind of what a screen reader sounds like. 00:05:12.780 --> 00:05:15.160 align:center line:84% I actually gave a demo of this talk. 00:05:15.160 --> 00:05:17.040 align:center line:84% And one of the audience said like they 00:05:17.040 --> 00:05:19.230 align:center line:84% find it very hard just to even understand the screen 00:05:19.230 --> 00:05:20.938 align:center line:84% reader because it's a very robotic voice. 00:05:20.938 --> 00:05:25.020 align:center line:84% It doesn't really have natural sort of intonation or anything. 00:05:25.020 --> 00:05:27.960 align:center line:84% And it can be quite difficult to understand. 00:05:27.960 --> 00:05:30.030 align:center line:84% But I'm going to make some observations 00:05:30.030 --> 00:05:35.170 align:center line:84% on how these screen readers work for text based programming. 00:05:35.170 --> 00:05:38.070 align:center line:84% So when you change line and your cursor moves to a new line, 00:05:38.070 --> 00:05:41.490 align:center line:84% it reads the whole line out, but there is a bit of a problem. 00:05:41.490 --> 00:05:43.740 align:center line:84% Some of the punctuation is just omitted 00:05:43.740 --> 00:05:44.990 align:center line:90% when it reads that whole line. 00:05:44.990 --> 00:05:46.698 align:center line:84% So I don't know if you noticed, but there 00:05:46.698 --> 00:05:48.352 align:center line:84% was a line with only a curly bracket, 00:05:48.352 --> 00:05:50.310 align:center line:84% and when you move there, the screen reader just 00:05:50.310 --> 00:05:51.270 align:center line:90% doesn't say anything. 00:05:51.270 --> 00:05:53.820 align:center line:84% So you don't actually know that the curly bracket is there 00:05:53.820 --> 00:05:55.060 align:center line:90% on that line. 00:05:55.060 --> 00:05:56.820 align:center line:84% So, obviously, that's very difficult, 00:05:56.820 --> 00:06:00.563 align:center line:84% if you also can't see that the curly brackets there. 00:06:00.563 --> 00:06:01.980 align:center line:84% And as you move the cursor around, 00:06:01.980 --> 00:06:04.167 align:center line:84% you get rid out the single character 00:06:04.167 --> 00:06:05.250 align:center line:90% that you've moved next to. 00:06:05.250 --> 00:06:06.875 align:center line:84% So it always reads the single character 00:06:06.875 --> 00:06:08.760 align:center line:90% that's ahead of your cursor. 00:06:08.760 --> 00:06:10.750 align:center line:84% But some of the names are a bit unexpected. 00:06:10.750 --> 00:06:13.740 align:center line:84% So what I would call an underscore, it called a line. 00:06:13.740 --> 00:06:17.212 align:center line:84% When it read out, it just said NV line. 00:06:17.212 --> 00:06:19.170 align:center line:84% And another thing that's a bit subtle at first, 00:06:19.170 --> 00:06:21.000 align:center line:84% but you kind of get used to it, is 00:06:21.000 --> 00:06:24.090 align:center line:84% that capitalization is indicated by a different pitch. 00:06:24.090 --> 00:06:27.270 align:center line:84% So when it actually said GL capitalized, 00:06:27.270 --> 00:06:31.140 align:center line:84% it used a slightly different pitch to GL lowercase. 00:06:31.140 --> 00:06:33.540 align:center line:84% So there's various sort of interesting features of screen 00:06:33.540 --> 00:06:36.210 align:center line:84% readers, but you can already see that it's a bit awkward even 00:06:36.210 --> 00:06:39.420 align:center line:90% for text based programming. 00:06:39.420 --> 00:06:42.140 align:center line:84% So now we're going to move on to talk 00:06:42.140 --> 00:06:44.040 align:center line:90% about block based programming. 00:06:44.040 --> 00:06:46.970 align:center line:84% So block based programming is very popular with kids, 00:06:46.970 --> 00:06:49.100 align:center line:84% for teaching programming to young children, 00:06:49.100 --> 00:06:52.043 align:center line:84% especially sort of under the ages of 12. 00:06:52.043 --> 00:06:53.960 align:center line:84% And the problem is that a lot of these systems 00:06:53.960 --> 00:06:57.690 align:center line:84% are simply inaccessible for those with impaired vision. 00:06:57.690 --> 00:06:59.190 align:center line:84% And that's actually kind of twofold. 00:06:59.190 --> 00:07:01.482 align:center line:84% One is the lack of screen reading, which is mainly what 00:07:01.482 --> 00:07:02.900 align:center line:90% I'm going to be talking about. 00:07:02.900 --> 00:07:05.430 align:center line:84% But also just the fact that it's primarily mouse-driven. 00:07:05.430 --> 00:07:08.810 align:center line:84% So if you have impaired vision, using a mouse 00:07:08.810 --> 00:07:10.880 align:center line:84% is generally more difficult than using a keyboard 00:07:10.880 --> 00:07:14.060 align:center line:84% because you can't always see clearly where the cursor is. 00:07:14.060 --> 00:07:15.530 align:center line:84% Or if you have no vision, then you 00:07:15.530 --> 00:07:18.100 align:center line:90% can't see the cursor at all. 00:07:18.100 --> 00:07:20.830 align:center line:84% So I set out to make a video of what Scratch 00:07:20.830 --> 00:07:22.720 align:center line:90% is like with a screen reader. 00:07:22.720 --> 00:07:25.330 align:center line:84% So let me just make sure it's up. 00:07:25.330 --> 00:07:28.120 align:center line:84% And as I made the video, I was trying 00:07:28.120 --> 00:07:31.000 align:center line:84% to touch all sorts of keys to see what they did. 00:07:31.000 --> 00:07:34.360 align:center line:84% So this first bit of the video, this 00:07:34.360 --> 00:07:36.430 align:center line:84% is the response to me banging all kinds of keys. 00:07:36.430 --> 00:07:39.526 align:center line:90% [VIDEO PLAYBACK] 00:07:39.526 --> 00:07:42.454 align:center line:90% SCREEN READER: Space space. 00:07:42.454 --> 00:07:43.863 align:center line:90% [END PLAYBACK] 00:07:43.863 --> 00:07:46.030 align:center line:84% NEIL BROWN: So it at least recognize the space keep, 00:07:46.030 --> 00:07:47.190 align:center line:90% but nothing else happened. 00:07:47.190 --> 00:07:49.920 align:center line:84% So I said OK well I'll use the mouse to see what happens. 00:07:49.920 --> 00:07:51.420 align:center line:84% So I went and clicked in this field. 00:07:51.420 --> 00:07:58.494 align:center line:90% 00:07:58.494 --> 00:07:59.960 align:center line:90% [VIDEO PLAYBACK] 00:07:59.960 --> 00:08:01.715 align:center line:90% SCREEN READER: Three. 00:08:01.715 --> 00:08:02.270 align:center line:90% Zero. 00:08:02.270 --> 00:08:03.130 align:center line:90% [END PLAYBACK] 00:08:03.130 --> 00:08:05.005 align:center line:84% NEIL BROWN: So you can see that when I typed, 00:08:05.005 --> 00:08:07.737 align:center line:84% it actually read it out, but otherwise it's just silent. 00:08:07.737 --> 00:08:09.820 align:center line:84% There's no way to actually read the content of any 00:08:09.820 --> 00:08:11.740 align:center line:90% of the blocks. 00:08:11.740 --> 00:08:14.470 align:center line:84% It didn't even read the content of the text field 00:08:14.470 --> 00:08:15.880 align:center line:90% when I focused it. 00:08:15.880 --> 00:08:18.880 align:center line:84% I expected to at least get 15, and then 15 00:08:18.880 --> 00:08:20.380 align:center line:90% as I moved the cursor around. 00:08:20.380 --> 00:08:23.110 align:center line:84% But it was just completely silent 00:08:23.110 --> 00:08:26.840 align:center line:84% until I managed to focus something and type something. 00:08:26.840 --> 00:08:29.475 align:center line:84% So the issue there is that it's not just fairly silent 00:08:29.475 --> 00:08:31.600 align:center line:84% with the blocks, but it's actually even more silent 00:08:31.600 --> 00:08:32.362 align:center line:90% than I expected. 00:08:32.362 --> 00:08:34.570 align:center line:84% I thought at least the text fields would sort of have 00:08:34.570 --> 00:08:36.262 align:center line:90% standard accessibility on it. 00:08:36.262 --> 00:08:38.679 align:center line:84% So you can see that this is essentially unusable if you're 00:08:38.679 --> 00:08:39.850 align:center line:90% using a screen reader. 00:08:39.850 --> 00:08:43.510 align:center line:84% You just can't get any sense of what code is there 00:08:43.510 --> 00:08:48.070 align:center line:84% or sort of even enter the code yourself. 00:08:48.070 --> 00:08:49.817 align:center line:84% So it's not all quite as bad as that. 00:08:49.817 --> 00:08:51.400 align:center line:84% There's been a few attempts to improve 00:08:51.400 --> 00:08:52.790 align:center line:90% some other block based systems. 00:08:52.790 --> 00:08:55.250 align:center line:84% So I'm going to look at, not all of them, 00:08:55.250 --> 00:08:57.640 align:center line:84% but just a couple that have been done for Blockly. 00:08:57.640 --> 00:09:01.270 align:center line:84% So Blockly is a Google originated system 00:09:01.270 --> 00:09:04.550 align:center line:90% for block based programming. 00:09:04.550 --> 00:09:07.060 align:center line:84% So one attempt I think was by the Google developers, 00:09:07.060 --> 00:09:12.320 align:center line:84% but it wasn't completely clear, and that looked like this. 00:09:12.320 --> 00:09:14.530 align:center line:84% So really this is quite interesting 00:09:14.530 --> 00:09:16.810 align:center line:84% because it doesn't look like blocks at all. 00:09:16.810 --> 00:09:19.490 align:center line:84% It's a sort of completely different display. 00:09:19.490 --> 00:09:22.180 align:center line:84% So what we've got here is sort of a nested list. 00:09:22.180 --> 00:09:24.220 align:center line:84% At the topmost level, it says if true, 00:09:24.220 --> 00:09:27.610 align:center line:84% do print "abc," which is the content of the whole code. 00:09:27.610 --> 00:09:30.220 align:center line:84% And then there's nested sublists that actually breaks that 00:09:30.220 --> 00:09:32.390 align:center line:84% apart, so you can read the separate components. 00:09:32.390 --> 00:09:35.140 align:center line:84% So the first item is if, which on the block 00:09:35.140 --> 00:09:37.090 align:center line:90% would be a sort of little label. 00:09:37.090 --> 00:09:39.010 align:center line:84% Then the second item is actually the sort 00:09:39.010 --> 00:09:42.910 align:center line:84% of manipulatable true part of the condition of the if. 00:09:42.910 --> 00:09:45.430 align:center line:84% Then you've got another label which is the do. 00:09:45.430 --> 00:09:49.480 align:center line:84% And then there's the body of the if, which here is printing abc, 00:09:49.480 --> 00:09:52.242 align:center line:84% and then that's also kind of decomposed into different bits. 00:09:52.242 --> 00:09:53.950 align:center line:84% So it's quite interesting because they've 00:09:53.950 --> 00:09:56.500 align:center line:84% taken this, actually, the sort of syntax tree 00:09:56.500 --> 00:09:58.270 align:center line:84% that was always underlying the blocks, 00:09:58.270 --> 00:10:01.000 align:center line:84% and turned it into a very text friendly representation. 00:10:01.000 --> 00:10:03.400 align:center line:84% So this works well with a screen reader actually. 00:10:03.400 --> 00:10:04.940 align:center line:84% You can move up and down the list. 00:10:04.940 --> 00:10:06.730 align:center line:90% You can navigate around. 00:10:06.730 --> 00:10:09.280 align:center line:84% It will read out all the text that you would expect. 00:10:09.280 --> 00:10:13.100 align:center line:84% It's just-- is this sort of blocks at this point? 00:10:13.100 --> 00:10:15.250 align:center line:84% Is it a kind of different structured editor, 00:10:15.250 --> 00:10:17.080 align:center line:84% but it's certainly an interesting attempt. 00:10:17.080 --> 00:10:19.940 align:center line:90% 00:10:19.940 --> 00:10:21.500 align:center line:90% So I think I've covered those. 00:10:21.500 --> 00:10:24.930 align:center line:84% So yeah, the screen reader has good support for nested lists. 00:10:24.930 --> 00:10:28.520 align:center line:84% So it's quite a good representation. 00:10:28.520 --> 00:10:31.600 align:center line:84% And a different independent attempt at Blockly 00:10:31.600 --> 00:10:34.150 align:center line:84% accessibility was by a set of researchers 00:10:34.150 --> 00:10:36.430 align:center line:90% led by Mountapmbeme. 00:10:36.430 --> 00:10:40.280 align:center line:84% And that was just published only about a month ago or so. 00:10:40.280 --> 00:10:42.610 align:center line:84% So what I've done here is I've borrowed the demo video 00:10:42.610 --> 00:10:44.705 align:center line:84% that they submitted with their ACM paper, 00:10:44.705 --> 00:10:46.330 align:center line:84% so you can go and check this out if you 00:10:46.330 --> 00:10:48.643 align:center line:90% find their paper afterwards. 00:10:48.643 --> 00:10:50.560 align:center line:84% So I'll just let their video run for a minute. 00:10:50.560 --> 00:10:50.950 align:center line:90% [VIDEO PLAYBACK] 00:10:50.950 --> 00:10:52.575 align:center line:84% SCREEN READER: Meeting controls window. 00:10:52.575 --> 00:10:53.710 align:center line:90% Blockly interface. 00:10:53.710 --> 00:10:54.610 align:center line:90% Google Chrome. 00:10:54.610 --> 00:10:55.540 align:center line:90% Tree view. 00:10:55.540 --> 00:10:57.940 align:center line:84% Logic not selected 1 of 8 level 1. 00:10:57.940 --> 00:11:00.690 align:center line:90% 00:11:00.690 --> 00:11:02.970 align:center line:90% Loops 2 of 8 level 1. 00:11:02.970 --> 00:11:03.780 align:center line:90% Repeat. 00:11:03.780 --> 00:11:05.100 align:center line:90% Blank. 00:11:05.100 --> 00:11:06.585 align:center line:90% Blockly interface document. 00:11:06.585 --> 00:11:11.720 align:center line:90% 00:11:11.720 --> 00:11:13.430 align:center line:90% Edit mode entered now. 00:11:13.430 --> 00:11:16.890 align:center line:90% Top connect times connection. 00:11:16.890 --> 00:11:17.970 align:center line:90% Tree view. 00:11:17.970 --> 00:11:20.160 align:center line:90% Loops 2 of 8 level 1. 00:11:20.160 --> 00:11:21.990 align:center line:90% Math 3 of 8 level 1. 00:11:21.990 --> 00:11:23.850 align:center line:90% Number block. 00:11:23.850 --> 00:11:25.530 align:center line:90% Blockly interface document. 00:11:25.530 --> 00:11:27.360 align:center line:90% Edit mode entered now. 00:11:27.360 --> 00:11:28.560 align:center line:90% Edit mode exited. 00:11:28.560 --> 00:11:30.480 align:center line:90% 5 unselected. 00:11:30.480 --> 00:11:33.310 align:center line:90% Blockly interface document. 00:11:33.310 --> 00:11:34.435 align:center line:90% Repeat 5 times. 00:11:34.435 --> 00:11:37.260 align:center line:90% 00:11:37.260 --> 00:11:38.670 align:center line:90% Edit mode entered now. 00:11:38.670 --> 00:11:39.840 align:center line:90% Top connection. 00:11:39.840 --> 00:11:41.130 align:center line:90% Times connection. 00:11:41.130 --> 00:11:43.050 align:center line:90% Do connection. 00:11:43.050 --> 00:11:43.950 align:center line:90% Tree view. 00:11:43.950 --> 00:11:44.490 align:center line:90% Block. 00:11:44.490 --> 00:11:46.290 align:center line:90% Loops 2 of 8 level 1. 00:11:46.290 --> 00:11:49.320 align:center line:90% Math 3 of 8 level 1. 00:11:49.320 --> 00:11:51.120 align:center line:90% Text 4 of 8 level 1. 00:11:51.120 --> 00:11:52.650 align:center line:90% Disabled text block. 00:11:52.650 --> 00:11:55.920 align:center line:84% Disabled prompt for text with message text block. 00:11:55.920 --> 00:11:56.730 align:center line:90% Print. 00:11:56.730 --> 00:11:57.810 align:center line:90% Block. 00:11:57.810 --> 00:12:00.090 align:center line:90% Blockly interface document. 00:12:00.090 --> 00:12:01.920 align:center line:90% Edit mode entered now. 00:12:01.920 --> 00:12:03.270 align:center line:90% Top connection. 00:12:03.270 --> 00:12:06.450 align:center line:90% Text connection. 00:12:06.450 --> 00:12:07.470 align:center line:90% Tree view. 00:12:07.470 --> 00:12:08.400 align:center line:90% Logic not. 00:12:08.400 --> 00:12:10.620 align:center line:90% Loops 2 of 8 level 1. 00:12:10.620 --> 00:12:14.340 align:center line:90% Math 3 of 8 level 1. 00:12:14.340 --> 00:12:16.200 align:center line:90% Text 4 of 8 level 1. 00:12:16.200 --> 00:12:18.890 align:center line:90% Text block. 00:12:18.890 --> 00:12:20.660 align:center line:90% Blockly interface document. 00:12:20.660 --> 00:12:23.000 align:center line:90% Edit mode entered now. 00:12:23.000 --> 00:12:23.660 align:center line:90% Edit mode-- 00:12:23.660 --> 00:12:26.929 align:center line:90% 00:12:26.929 --> 00:12:28.338 align:center line:90% [END PLAYBACK] 00:12:28.338 --> 00:12:29.880 align:center line:84% NEIL BROWN: OK, so at this stage I'll 00:12:29.880 --> 00:12:32.370 align:center line:84% pause it because we've probably seen enough. 00:12:32.370 --> 00:12:32.900 align:center line:90% Sorry. 00:12:32.900 --> 00:12:34.450 align:center line:90% There we go again. 00:12:34.450 --> 00:12:35.880 align:center line:84% So a few observations about that. 00:12:35.880 --> 00:12:38.940 align:center line:84% I mean, obviously we're better than the sort of silence 00:12:38.940 --> 00:12:41.820 align:center line:84% that we saw with Scratch, but it still 00:12:41.820 --> 00:12:44.490 align:center line:84% seems that it sort of reveals the underlying awkwardness 00:12:44.490 --> 00:12:46.360 align:center line:90% of some of the blocks aspects. 00:12:46.360 --> 00:12:48.840 align:center line:84% So there seems to be a lot of excess speech, 00:12:48.840 --> 00:12:51.420 align:center line:84% and it was sort of quite laborious to navigate the block 00:12:51.420 --> 00:12:53.700 align:center line:84% palette in order to find the blocks that you wanted, 00:12:53.700 --> 00:12:57.450 align:center line:84% and then actually insert them into the structure. 00:12:57.450 --> 00:12:59.430 align:center line:84% So I mean, that's because Blockly already 00:12:59.430 --> 00:13:04.170 align:center line:84% had this catalog of blocks type system or palette, 00:13:04.170 --> 00:13:06.250 align:center line:90% it's sometimes called. 00:13:06.250 --> 00:13:08.132 align:center line:84% So that's not so well suited for a keyboard. 00:13:08.132 --> 00:13:09.340 align:center line:90% It's quite good with a mouse. 00:13:09.340 --> 00:13:11.257 align:center line:84% And if you can immediately see all the blocks, 00:13:11.257 --> 00:13:14.070 align:center line:84% but with keyboard navigation, and not necessarily seeing them 00:13:14.070 --> 00:13:17.113 align:center line:84% then it's a lot more difficult. And also 00:13:17.113 --> 00:13:18.780 align:center line:84% I think you need to understand the block 00:13:18.780 --> 00:13:21.400 align:center line:84% structure quite well in order to manipulate it effectively. 00:13:21.400 --> 00:13:23.640 align:center line:84% So although it does reveal to the screen reader 00:13:23.640 --> 00:13:26.880 align:center line:84% and to the keyboard manipulation all the possibilities, 00:13:26.880 --> 00:13:28.380 align:center line:84% you have to navigate top connect, 00:13:28.380 --> 00:13:31.020 align:center line:84% to do connect, to bottom connect, 00:13:31.020 --> 00:13:32.700 align:center line:84% to navigate around those and actually 00:13:32.700 --> 00:13:34.920 align:center line:84% be able to go through the structure. 00:13:34.920 --> 00:13:37.140 align:center line:84% I feel like if you knew Blockly well already, 00:13:37.140 --> 00:13:38.640 align:center line:84% you could sort of do well with this. 00:13:38.640 --> 00:13:42.990 align:center line:84% But if you don't know it, and you're just revealing it 00:13:42.990 --> 00:13:45.700 align:center line:84% through the screen reader, it might be quite difficult. 00:13:45.700 --> 00:13:47.610 align:center line:84% Also, the block canvas can make things 00:13:47.610 --> 00:13:49.710 align:center line:84% maybe a little bit awkward, I think, 00:13:49.710 --> 00:13:52.560 align:center line:84% to then navigate around all the different sort of groups 00:13:52.560 --> 00:13:54.270 align:center line:84% of blocks that you might have in a canvas 00:13:54.270 --> 00:13:58.630 align:center line:90% and how to maybe place those. 00:13:58.630 --> 00:14:00.790 align:center line:84% But it seems all the possible solutions 00:14:00.790 --> 00:14:04.010 align:center line:84% for accessible blocks, they all have one thing in common, 00:14:04.010 --> 00:14:07.120 align:center line:84% and that's the idea of having a single point of attention-- so 00:14:07.120 --> 00:14:08.980 align:center line:90% cursor or focus. 00:14:08.980 --> 00:14:11.870 align:center line:84% I particularly mean the keyboard cursor here. 00:14:11.870 --> 00:14:13.720 align:center line:84% So we're used to this concept, of course, 00:14:13.720 --> 00:14:15.550 align:center line:84% already whenever we're doing or editing 00:14:15.550 --> 00:14:17.050 align:center line:90% a Word document or something. 00:14:17.050 --> 00:14:20.020 align:center line:84% The cursor gives a focal point where we can then insert text. 00:14:20.020 --> 00:14:21.130 align:center line:90% We can delete text. 00:14:21.130 --> 00:14:23.260 align:center line:90% We can select it. 00:14:23.260 --> 00:14:25.900 align:center line:84% But also, as we've seen with the text based programming, 00:14:25.900 --> 00:14:28.613 align:center line:84% it gives a focal point for the screen reading as well. 00:14:28.613 --> 00:14:30.280 align:center line:84% So it seems like all the block solutions 00:14:30.280 --> 00:14:33.310 align:center line:84% I've seen involve some notion of a cursor or some kind 00:14:33.310 --> 00:14:35.860 align:center line:84% of persistent focus that moves around the different block 00:14:35.860 --> 00:14:36.670 align:center line:90% structure. 00:14:36.670 --> 00:14:39.580 align:center line:84% So it isn't usually there in a sort of purely kind of mouse 00:14:39.580 --> 00:14:42.230 align:center line:90% driven block based system. 00:14:42.230 --> 00:14:44.650 align:center line:84% So it seems like the cursor gives us screen reading, 00:14:44.650 --> 00:14:46.960 align:center line:84% and it tends to give us keyboard manipulation. 00:14:46.960 --> 00:14:51.150 align:center line:84% And so that's what we really need for accessibility. 00:14:51.150 --> 00:14:54.230 align:center line:84% So I'm going to talk a little bit about some of our ideas now 00:14:54.230 --> 00:14:57.690 align:center line:84% and they do involve the notion of a cursor. 00:14:57.690 --> 00:14:59.870 align:center line:84% So we've been working for a few years on something 00:14:59.870 --> 00:15:01.640 align:center line:84% that we call frame based editing, which 00:15:01.640 --> 00:15:04.280 align:center line:84% is a lot like blocks, but it's just kind of different enough 00:15:04.280 --> 00:15:06.200 align:center line:84% that we decided to give it a different name 00:15:06.200 --> 00:15:07.860 align:center line:90% to distinguish it. 00:15:07.860 --> 00:15:09.200 align:center line:90% So we have this cursor. 00:15:09.200 --> 00:15:14.840 align:center line:84% There's a blue cursor there that sits between the frames. 00:15:14.840 --> 00:15:16.430 align:center line:90% That can be moved up and down. 00:15:16.430 --> 00:15:19.740 align:center line:84% It allows insertion using the keyboard of different frames. 00:15:19.740 --> 00:15:23.510 align:center line:84% It also allows selection, as we'll see in a minute. 00:15:23.510 --> 00:15:26.780 align:center line:84% But unlike blocks, expressions are edited solely 00:15:26.780 --> 00:15:27.810 align:center line:90% using the keyboard. 00:15:27.810 --> 00:15:31.190 align:center line:84% So if, in a block based system, you want to do something like 1 00:15:31.190 --> 00:15:33.950 align:center line:84% plus 2 times 3, you'd need to drag in a plus block. 00:15:33.950 --> 00:15:35.780 align:center line:84% You'd need to drag in a times block. 00:15:35.780 --> 00:15:38.690 align:center line:84% You'd need to then fill in all the different fields separately 00:15:38.690 --> 00:15:40.310 align:center line:90% using the keyboard. 00:15:40.310 --> 00:15:42.860 align:center line:84% In our system, you'll just type 1 plus 2 times 00:15:42.860 --> 00:15:45.510 align:center line:84% 3, and type it in with the keyboard. 00:15:45.510 --> 00:15:49.290 align:center line:84% So that's one vital difference between the two. 00:15:49.290 --> 00:15:53.510 align:center line:84% So I've pre-recorded a demo, so I'll just play that for you 00:15:53.510 --> 00:15:54.374 align:center line:90% now. 00:15:54.374 --> 00:15:56.150 align:center line:90% [VIDEO PLAYBACK] 00:15:56.150 --> 00:15:59.360 align:center line:84% Frame based editing has the same basic idea as blocks. 00:15:59.360 --> 00:16:02.780 align:center line:84% The code is structured as a series of snapable items. 00:16:02.780 --> 00:16:07.160 align:center line:84% So although they're formatted to look like they flow together, 00:16:07.160 --> 00:16:09.340 align:center line:84% the increment hunger and increment age method 00:16:09.340 --> 00:16:13.070 align:center line:84% calls here are actually frames that can be selected 00:16:13.070 --> 00:16:15.950 align:center line:90% and dragged around and dropped. 00:16:15.950 --> 00:16:18.080 align:center line:84% The key difference between frames and blocks 00:16:18.080 --> 00:16:20.870 align:center line:90% is the idea of the frame cursor. 00:16:20.870 --> 00:16:24.770 align:center line:84% So the blue bar here is essentially a cursor 00:16:24.770 --> 00:16:26.810 align:center line:90% that sits between frames. 00:16:26.810 --> 00:16:28.940 align:center line:84% Using my keyboard, I can move it up and down 00:16:28.940 --> 00:16:32.000 align:center line:90% with the arrow keys. 00:16:32.000 --> 00:16:35.750 align:center line:84% And I can also then use it to insert a new frame 00:16:35.750 --> 00:16:36.840 align:center line:90% if I want to. 00:16:36.840 --> 00:16:39.890 align:center line:84% So the shortcuts are shown on the right hand side here. 00:16:39.890 --> 00:16:43.490 align:center line:84% If I press I, then I get an if frame. 00:16:43.490 --> 00:16:47.000 align:center line:84% So just as with blocks, this is one cohesive frame. 00:16:47.000 --> 00:16:50.180 align:center line:84% I can, again, drag it around if I want to. 00:16:50.180 --> 00:16:52.640 align:center line:84% But the difference to blocks is that when 00:16:52.640 --> 00:16:54.680 align:center line:84% I come to enter the condition, I don't then 00:16:54.680 --> 00:16:59.270 align:center line:84% continue to drag in a little smaller blocks or smaller 00:16:59.270 --> 00:17:01.520 align:center line:84% frames in order to fill the condition in. 00:17:01.520 --> 00:17:03.200 align:center line:84% I just type it using the keyboard, 00:17:03.200 --> 00:17:06.170 align:center line:84% much as I would in a text based editor. 00:17:06.170 --> 00:17:10.339 align:center line:84% So I can write this method call here canBreed, 00:17:10.339 --> 00:17:12.300 align:center line:84% and then if I press down on my keyboard, 00:17:12.300 --> 00:17:17.369 align:center line:84% I can go inside the if, and I can choose to add more frames. 00:17:17.369 --> 00:17:19.819 align:center line:84% So if I press the shortcut space here 00:17:19.819 --> 00:17:25.190 align:center line:84% that inserts a method, which again, I can just type here. 00:17:25.190 --> 00:17:28.150 align:center line:90% 00:17:28.150 --> 00:17:37.780 align:center line:84% So I can then tell it to print out a message like so. 00:17:37.780 --> 00:17:40.210 align:center line:84% So as you can see, some of the syntax 00:17:40.210 --> 00:17:42.070 align:center line:90% here is automatically inserted. 00:17:42.070 --> 00:17:45.130 align:center line:84% So when I added the open quote, I automatically 00:17:45.130 --> 00:17:46.630 align:center line:90% got the closed quote. 00:17:46.630 --> 00:17:49.360 align:center line:84% There's no way that I can delete this closed quote now 00:17:49.360 --> 00:17:51.620 align:center line:84% without deleting the entire string. 00:17:51.620 --> 00:17:56.230 align:center line:84% So the string is, therefore, always sort of either complete 00:17:56.230 --> 00:17:57.580 align:center line:90% or not present. 00:17:57.580 --> 00:17:59.980 align:center line:90% Same with the brackets here. 00:17:59.980 --> 00:18:02.350 align:center line:84% When I added the open brackets, the closed bracket 00:18:02.350 --> 00:18:04.060 align:center line:90% was automatically inserted. 00:18:04.060 --> 00:18:06.070 align:center line:84% And I can't delete this closed bracket 00:18:06.070 --> 00:18:08.230 align:center line:84% without deleting the open brackets. 00:18:08.230 --> 00:18:09.970 align:center line:84% So they're always paired, and this 00:18:09.970 --> 00:18:12.190 align:center line:84% reduces the amount of possible syntax errors, 00:18:12.190 --> 00:18:13.990 align:center line:90% much like blocks does. 00:18:13.990 --> 00:18:15.520 align:center line:84% But the difference here is the entry 00:18:15.520 --> 00:18:17.110 align:center line:84% can be done with the keyboard, so it's 00:18:17.110 --> 00:18:20.110 align:center line:84% faster and easier than the blocks, where you 00:18:20.110 --> 00:18:25.450 align:center line:90% have to drag in blocks, rather. 00:18:25.450 --> 00:18:28.960 align:center line:84% And so with this frame cursor, I can move around. 00:18:28.960 --> 00:18:30.940 align:center line:90% So there's extra shortcuts. 00:18:30.940 --> 00:18:32.800 align:center line:84% If I hold down the Control key here when 00:18:32.800 --> 00:18:35.350 align:center line:84% I'm pressing up and down, I move up and down 00:18:35.350 --> 00:18:36.500 align:center line:90% just at the same level. 00:18:36.500 --> 00:18:39.880 align:center line:84% So rather than going into this if, I skip past it. 00:18:39.880 --> 00:18:43.520 align:center line:90% So that's useful for navigation. 00:18:43.520 --> 00:18:46.000 align:center line:84% I could also select multiple frames, 00:18:46.000 --> 00:18:48.940 align:center line:84% which is sometimes the pain in block based systems. 00:18:48.940 --> 00:18:50.680 align:center line:84% I can then choose to delete these, 00:18:50.680 --> 00:18:52.300 align:center line:90% if I wanted to, as a selection. 00:18:52.300 --> 00:18:54.520 align:center line:84% Or I can choose to grab them both 00:18:54.520 --> 00:18:57.130 align:center line:90% and move around and manipulate. 00:18:57.130 --> 00:18:59.050 align:center line:84% Although I use the mouse for that interaction, 00:18:59.050 --> 00:19:01.120 align:center line:84% the same is available with the keyboard. 00:19:01.120 --> 00:19:04.150 align:center line:84% For example, if I select these again, if I hit Control-X, 00:19:04.150 --> 00:19:05.250 align:center line:90% it cuts them. 00:19:05.250 --> 00:19:09.300 align:center line:84% If I hit Control-V, I pastes them. 00:19:09.300 --> 00:19:10.650 align:center line:90% [END PLAYBACK] 00:19:10.650 --> 00:19:12.870 align:center line:84% OK, so that's just a sort of overview of frame 00:19:12.870 --> 00:19:13.950 align:center line:90% based editing in general. 00:19:13.950 --> 00:19:16.200 align:center line:84% Not yet as accessibility capabilities, 00:19:16.200 --> 00:19:18.300 align:center line:84% but it's important to understand the general model 00:19:18.300 --> 00:19:23.560 align:center line:84% before we go on to talk about that in a little bit. 00:19:23.560 --> 00:19:27.150 align:center line:84% So some of the features there, so we 00:19:27.150 --> 00:19:29.235 align:center line:84% have a lot of the advantages of blocks, 00:19:29.235 --> 00:19:30.360 align:center line:90% but with a few differences. 00:19:30.360 --> 00:19:32.490 align:center line:84% There's no block canvas, where you 00:19:32.490 --> 00:19:34.680 align:center line:84% can get lots of little different bits of code 00:19:34.680 --> 00:19:36.240 align:center line:90% hanging around in Scratch. 00:19:36.240 --> 00:19:39.030 align:center line:84% So everything is just in a single sort 00:19:39.030 --> 00:19:42.480 align:center line:84% of cohesive structure like in text based programming. 00:19:42.480 --> 00:19:44.080 align:center line:84% There's no white space to deal with. 00:19:44.080 --> 00:19:45.540 align:center line:90% You don't type any white space. 00:19:45.540 --> 00:19:47.340 align:center line:84% It's all derived from the placement 00:19:47.340 --> 00:19:49.980 align:center line:84% of the frames what's inside what, 00:19:49.980 --> 00:19:51.977 align:center line:90% just as you have with blocks. 00:19:51.977 --> 00:19:54.060 align:center line:84% You don't get any of the structural syntax errors, 00:19:54.060 --> 00:19:56.550 align:center line:84% like unterminated scopes, you know, open curly bracket, 00:19:56.550 --> 00:19:59.460 align:center line:84% but no close because you're not dealing with those. 00:19:59.460 --> 00:20:01.230 align:center line:84% Also the errors are fairly localized, 00:20:01.230 --> 00:20:03.772 align:center line:84% so when you do get an error, you could usually narrow it down 00:20:03.772 --> 00:20:05.430 align:center line:84% to one particular sort of text field 00:20:05.430 --> 00:20:08.460 align:center line:84% and say, OK, the error is in this particular bit. 00:20:08.460 --> 00:20:10.950 align:center line:84% So they don't sort of span an entire file 00:20:10.950 --> 00:20:15.130 align:center line:84% when you have an terminated method or something like that. 00:20:15.130 --> 00:20:17.250 align:center line:84% But compared to blocks, we also get keyboard entry 00:20:17.250 --> 00:20:18.360 align:center line:90% and keyboard manipulation. 00:20:18.360 --> 00:20:21.280 align:center line:84% I was talking through there some of the shortcuts I was using. 00:20:21.280 --> 00:20:23.070 align:center line:84% You can see that you can insert frames. 00:20:23.070 --> 00:20:24.180 align:center line:90% You can manipulate frames. 00:20:24.180 --> 00:20:26.320 align:center line:84% You can delete frames all using the keyboard. 00:20:26.320 --> 00:20:29.135 align:center line:84% So you don't have to use the mouse for any interaction. 00:20:29.135 --> 00:20:31.260 align:center line:84% You can do it all with the keyboard if you want to, 00:20:31.260 --> 00:20:33.330 align:center line:84% and that's important if you're someone 00:20:33.330 --> 00:20:37.250 align:center line:90% that is unable to use a mouse. 00:20:37.250 --> 00:20:39.520 align:center line:84% So we can talk now about some of the higher level 00:20:39.520 --> 00:20:42.100 align:center line:90% issues of accessibility. 00:20:42.100 --> 00:20:46.750 align:center line:84% So from a different paper by Mountapmbeme from 00:20:46.750 --> 00:20:49.000 align:center line:84% earlier this year, they had a sort 00:20:49.000 --> 00:20:51.330 align:center line:84% of review of accessibility and blocks. 00:20:51.330 --> 00:20:54.590 align:center line:84% They talked about five aspects of programming to consider. 00:20:54.590 --> 00:20:57.890 align:center line:84% So there's navigating your way around the code, 00:20:57.890 --> 00:21:01.570 align:center line:84% being able to comprehend the code, editing the code, 00:21:01.570 --> 00:21:04.450 align:center line:84% skimming-- which is sort of like just sort of glancing up 00:21:04.450 --> 00:21:08.380 align:center line:84% and down, rather than sort of specific navigation-- 00:21:08.380 --> 00:21:10.090 align:center line:90% and also debugging. 00:21:10.090 --> 00:21:11.800 align:center line:84% And so this is a nice classification, 00:21:11.800 --> 00:21:14.650 align:center line:84% and to some extent, I would have liked to start the talk here 00:21:14.650 --> 00:21:16.360 align:center line:84% and say, OK, here's the higher level 00:21:16.360 --> 00:21:20.140 align:center line:84% aspects of a really good, accessible design. 00:21:20.140 --> 00:21:21.850 align:center line:84% But it felt kind of like whipping out 00:21:21.850 --> 00:21:24.790 align:center line:84% some Maslow's hierarchy of needs pyramid of all these sort 00:21:24.790 --> 00:21:26.560 align:center line:84% of complicated things we could do, 00:21:26.560 --> 00:21:29.200 align:center line:84% when the problem really is that the patient is flatlining 00:21:29.200 --> 00:21:31.180 align:center line:84% and they need a defibrillator to get going. 00:21:31.180 --> 00:21:33.185 align:center line:84% Like the bar is so low at the moment that 00:21:33.185 --> 00:21:34.810 align:center line:84% talking about these higher level issues 00:21:34.810 --> 00:21:36.700 align:center line:84% feels like a bit of a luxury when 00:21:36.700 --> 00:21:39.160 align:center line:84% so many of the systems just kind of don't really 00:21:39.160 --> 00:21:42.790 align:center line:84% have any support for accessibility at all. 00:21:42.790 --> 00:21:44.980 align:center line:84% But nevertheless, we can talk about a couple 00:21:44.980 --> 00:21:45.790 align:center line:90% of these aspects. 00:21:45.790 --> 00:21:47.530 align:center line:84% I'll show a couple of existing examples 00:21:47.530 --> 00:21:51.100 align:center line:84% and show some of the things that we've been working on. 00:21:51.100 --> 00:21:54.870 align:center line:84% So one piece of existing work is Struct Jumper. 00:21:54.870 --> 00:21:59.080 align:center line:84% So Struct Jumper was a plug-in for Eclipse which takes 00:21:59.080 --> 00:21:59.580 align:center line:90% your code-- 00:21:59.580 --> 00:22:01.413 align:center line:84% I think it was text-based code, but the idea 00:22:01.413 --> 00:22:03.690 align:center line:84% would apply just the same, I think, to blocks. 00:22:03.690 --> 00:22:06.510 align:center line:84% And then it would turn it into a sort of tree view, 00:22:06.510 --> 00:22:08.710 align:center line:84% showing the general structure of your code. 00:22:08.710 --> 00:22:11.940 align:center line:84% So it would show the method header as one item, 00:22:11.940 --> 00:22:14.890 align:center line:84% and then sort of any scopes inside. 00:22:14.890 --> 00:22:18.300 align:center line:84% So for loops, while loops, if statements would then 00:22:18.300 --> 00:22:20.890 align:center line:84% appear as nested items within the tree view. 00:22:20.890 --> 00:22:22.410 align:center line:84% So you can sort of skim the code. 00:22:22.410 --> 00:22:24.180 align:center line:84% It's almost a bit like code folding 00:22:24.180 --> 00:22:26.563 align:center line:84% that you sort of fold it up your code. 00:22:26.563 --> 00:22:28.230 align:center line:84% And then you can go down the tree view-- 00:22:28.230 --> 00:22:31.890 align:center line:84% but it is a proper tree with sort of nesting and so on-- 00:22:31.890 --> 00:22:34.530 align:center line:84% in order to find your way to the piece of code that you want 00:22:34.530 --> 00:22:37.390 align:center line:84% or just to get an impression of what the code is like overall. 00:22:37.390 --> 00:22:40.560 align:center line:84% And then you could return to the main editor. 00:22:40.560 --> 00:22:42.540 align:center line:84% So it's a sort of extra navigation pane. 00:22:42.540 --> 00:22:45.070 align:center line:90% 00:22:45.070 --> 00:22:48.400 align:center line:84% Another idea is this CodeMirror blocks. 00:22:48.400 --> 00:22:51.640 align:center line:84% So this is a plug-in for an editor that already existed 00:22:51.640 --> 00:22:53.530 align:center line:90% for text called CodeMirror. 00:22:53.530 --> 00:22:56.350 align:center line:84% And what it would do is allow you to take some of the text 00:22:56.350 --> 00:22:58.720 align:center line:84% and turn it into blocks, so you get 00:22:58.720 --> 00:23:00.460 align:center line:84% a sort of block like display that you can 00:23:00.460 --> 00:23:02.710 align:center line:90% navigate as if it were a tree. 00:23:02.710 --> 00:23:04.660 align:center line:84% So it's a bit like some of the hybrid editors 00:23:04.660 --> 00:23:06.410 align:center line:84% that you get with things like Pencil Code, 00:23:06.410 --> 00:23:10.330 align:center line:84% where you can switch back and forth between text and blocks, 00:23:10.330 --> 00:23:12.850 align:center line:84% but they had a particular focus on making sure 00:23:12.850 --> 00:23:14.800 align:center line:84% that it was accessible, and that it could all 00:23:14.800 --> 00:23:16.975 align:center line:90% be navigated with the keyboard. 00:23:16.975 --> 00:23:19.720 align:center line:90% 00:23:19.720 --> 00:23:23.700 align:center line:84% So some of our ideas for how we might improve frame 00:23:23.700 --> 00:23:27.400 align:center line:90% based editing for accessibility. 00:23:27.400 --> 00:23:29.420 align:center line:90% So one thing is the idea of-- 00:23:29.420 --> 00:23:31.170 align:center line:84% I don't know the proper term I think here, 00:23:31.170 --> 00:23:32.970 align:center line:84% but let's call it speech hierarchy, 00:23:32.970 --> 00:23:35.380 align:center line:90% like typographic hierarchy. 00:23:35.380 --> 00:23:39.180 align:center line:84% So for screen readers, you often want the important text first. 00:23:39.180 --> 00:23:40.770 align:center line:84% And so the reason for that is let's 00:23:40.770 --> 00:23:43.530 align:center line:84% say you're going down a list of items, 00:23:43.530 --> 00:23:46.620 align:center line:84% if you get a lot of boilerplate text before each one, 00:23:46.620 --> 00:23:48.540 align:center line:84% then you have to listen to the boilerplate 00:23:48.540 --> 00:23:50.850 align:center line:84% before you can decide if this is the item you want. 00:23:50.850 --> 00:23:53.700 align:center line:84% Whereas if you have the unique, distinguishing text right 00:23:53.700 --> 00:23:55.493 align:center line:84% upfront and said first, then you can 00:23:55.493 --> 00:23:56.910 align:center line:84% say, oh, that's not the one I want 00:23:56.910 --> 00:23:58.795 align:center line:84% and skip down the list much faster. 00:23:58.795 --> 00:24:01.170 align:center line:84% So you really want to order it so that the most important 00:24:01.170 --> 00:24:04.022 align:center line:90% thing is spoken first. 00:24:04.022 --> 00:24:05.730 align:center line:84% The problem is in most programming syntax 00:24:05.730 --> 00:24:07.620 align:center line:84% like you'd use in text based programming, 00:24:07.620 --> 00:24:09.360 align:center line:84% you get a whole load of bits at the front 00:24:09.360 --> 00:24:10.840 align:center line:90% that you don't necessarily want. 00:24:10.840 --> 00:24:13.770 align:center line:84% So in Java, for example, most of the fields 00:24:13.770 --> 00:24:15.940 align:center line:84% usually have their access modifier at the front. 00:24:15.940 --> 00:24:17.580 align:center line:90% So the first thing is private. 00:24:17.580 --> 00:24:19.650 align:center line:84% So I mean the classic public static void 00:24:19.650 --> 00:24:22.558 align:center line:84% main, the public static void are not that interesting. 00:24:22.558 --> 00:24:24.600 align:center line:84% The key part is the main, because that's actually 00:24:24.600 --> 00:24:26.475 align:center line:84% the name of the method that distinguishes it. 00:24:26.475 --> 00:24:29.460 align:center line:84% So you'd like to hear the main first. 00:24:29.460 --> 00:24:31.530 align:center line:84% And with frames we actually have the ability 00:24:31.530 --> 00:24:34.210 align:center line:84% to change what's being read first, 00:24:34.210 --> 00:24:36.640 align:center line:84% so we don't just have to follow the order. 00:24:36.640 --> 00:24:40.860 align:center line:84% So here the frame cursor in our model of accessibility 00:24:40.860 --> 00:24:44.290 align:center line:84% reads out the frame that comes after it. 00:24:44.290 --> 00:24:46.595 align:center line:84% So I'll just play what that sounds like. 00:24:46.595 --> 00:24:47.262 align:center line:90% [VIDEO PLAYBACK] 00:24:47.262 --> 00:24:51.470 align:center line:84% SCREEN READER: field age of type int with private access button. 00:24:51.470 --> 00:24:52.940 align:center line:84% Field foodlevel level of type int 00:24:52.940 --> 00:24:54.620 align:center line:90% with protected access button. 00:24:54.620 --> 00:24:55.310 align:center line:90% [END PLAYBACK] 00:24:55.310 --> 00:24:57.893 align:center line:84% NEIL BROWN: So maybe we left the screen on a bit faster there, 00:24:57.893 --> 00:25:01.700 align:center line:84% but what it was saying was like field foodLevel of type int 00:25:01.700 --> 00:25:03.120 align:center line:90% with protected access. 00:25:03.120 --> 00:25:05.060 align:center line:84% So we've got the name right up front. 00:25:05.060 --> 00:25:06.410 align:center line:90% Actually, it says field first. 00:25:06.410 --> 00:25:08.785 align:center line:84% I've been wondering as I made this talk whether we should 00:25:08.785 --> 00:25:11.270 align:center line:84% scrap that, whether we should instead 00:25:11.270 --> 00:25:16.860 align:center line:84% say foodLevel field of type int with protected access. 00:25:16.860 --> 00:25:18.860 align:center line:84% There's also a question for sort of users 00:25:18.860 --> 00:25:20.485 align:center line:84% who use this a lot whether you actually 00:25:20.485 --> 00:25:23.570 align:center line:84% need the sort of stop words in there about of type 00:25:23.570 --> 00:25:25.407 align:center line:90% and with protected access. 00:25:25.407 --> 00:25:26.990 align:center line:84% If you've been going for a little bit, 00:25:26.990 --> 00:25:29.120 align:center line:84% you can pretty guess that private protected public 00:25:29.120 --> 00:25:30.835 align:center line:90% is an access level. 00:25:30.835 --> 00:25:32.210 align:center line:84% So there's a few design decisions 00:25:32.210 --> 00:25:34.543 align:center line:84% that you could tweak either way, but the principle there 00:25:34.543 --> 00:25:37.730 align:center line:84% was that we can actually put the field name at the front, which 00:25:37.730 --> 00:25:40.760 align:center line:84% is the most important bit, even though in terms of text order, 00:25:40.760 --> 00:25:43.100 align:center line:84% it's actually at the end of the line. 00:25:43.100 --> 00:25:44.930 align:center line:90% Our syntax derives from Java. 00:25:44.930 --> 00:25:48.740 align:center line:84% The idea is that people can use our Stride editor here, 00:25:48.740 --> 00:25:51.420 align:center line:84% and then sort of transition into Java afterwards 00:25:51.420 --> 00:25:52.670 align:center line:90% once they're familiar with it. 00:25:52.670 --> 00:25:55.630 align:center line:90% 00:25:55.630 --> 00:25:58.200 align:center line:84% So another issue we can consider is that of context. 00:25:58.200 --> 00:26:01.410 align:center line:84% So knowing where you are within the code is useful, 00:26:01.410 --> 00:26:04.260 align:center line:90% and I don't just mean line 362-- 00:26:04.260 --> 00:26:07.650 align:center line:84% I mean, sort of within the actual structure 00:26:07.650 --> 00:26:09.330 align:center line:90% proper of the code. 00:26:09.330 --> 00:26:11.400 align:center line:90% And you also want to know-- 00:26:11.400 --> 00:26:13.620 align:center line:84% so the text based programming system, 00:26:13.620 --> 00:26:15.190 align:center line:90% it reads out the line you're on. 00:26:15.190 --> 00:26:16.920 align:center line:84% So you at least know the full line, 00:26:16.920 --> 00:26:19.780 align:center line:84% but you don't necessarily know what's around you. 00:26:19.780 --> 00:26:24.060 align:center line:84% So what we've been experimenting with is having more information 00:26:24.060 --> 00:26:29.340 align:center line:84% read out so that with each frame that you're next to, 00:26:29.340 --> 00:26:31.980 align:center line:84% it not only tells you the information about the frame, 00:26:31.980 --> 00:26:33.640 align:center line:84% as I showed you in the previous video, 00:26:33.640 --> 00:26:36.070 align:center line:84% but also it gives you information on the context. 00:26:36.070 --> 00:26:38.805 align:center line:84% So here's a demonstration of what that sounds like. 00:26:38.805 --> 00:26:42.765 align:center line:90% [VIDEO PLAYBACK] 00:26:42.765 --> 00:26:44.550 align:center line:84% SCREEN READER: Method incrementHunger 00:26:44.550 --> 00:26:48.000 align:center line:84% with private access and void return type of documentation. 00:26:48.000 --> 00:26:49.650 align:center line:90% Make this fox more hungry. 00:26:49.650 --> 00:26:51.540 align:center line:84% This will result in the fox's death button 00:26:51.540 --> 00:26:55.110 align:center line:84% your're before a method in the class box. 00:26:55.110 --> 00:26:57.700 align:center line:84% Assign foodLevel with foodLevel minus 1 button 00:26:57.700 --> 00:27:00.660 align:center line:84% before an assignment frame in the method incrementHunger 00:27:00.660 --> 00:27:02.542 align:center line:90% in the class fox. 00:27:02.542 --> 00:27:04.234 align:center line:84% If frame with condition food level 00:27:04.234 --> 00:27:05.776 align:center line:84% is less than or equal to zero button. 00:27:05.776 --> 00:27:09.110 align:center line:84% You're before an if block in the method incrementHunger 00:27:09.110 --> 00:27:10.760 align:center line:90% in the class fox. 00:27:10.760 --> 00:27:13.100 align:center line:84% Method call set left bracket right bracket button. 00:27:13.100 --> 00:27:15.950 align:center line:84% You're before a method call in the [INAUDIBLE] frame 00:27:15.950 --> 00:27:18.260 align:center line:84% with condition foodLevel less equals 0 00:27:18.260 --> 00:27:20.930 align:center line:84% in the method incrementHunger in the class fox. 00:27:20.930 --> 00:27:21.870 align:center line:90% [END PLAYBACK] 00:27:21.870 --> 00:27:24.332 align:center line:84% NEIL BROWN: OK, so there's a few things to unpack. 00:27:24.332 --> 00:27:26.040 align:center line:84% I'm going to come back to it occasionally 00:27:26.040 --> 00:27:29.165 align:center line:90% says button later on. 00:27:29.165 --> 00:27:30.540 align:center line:84% So there's a few design decisions 00:27:30.540 --> 00:27:32.190 align:center line:90% we might sort of polish up here. 00:27:32.190 --> 00:27:34.080 align:center line:84% It gives you context every single time, 00:27:34.080 --> 00:27:38.010 align:center line:84% even though in this case, we had got repeating context. 00:27:38.010 --> 00:27:40.620 align:center line:84% You maybe don't need it every single time. 00:27:40.620 --> 00:27:42.570 align:center line:84% If you've heard that bit of context recently, 00:27:42.570 --> 00:27:45.185 align:center line:84% we could maybe iron out that we don't have to read that out. 00:27:45.185 --> 00:27:46.560 align:center line:84% But in general, what it was doing 00:27:46.560 --> 00:27:48.810 align:center line:84% was it's telling you the frame your next to, 00:27:48.810 --> 00:27:52.270 align:center line:84% and then if you wait there to hear more context, 00:27:52.270 --> 00:27:55.140 align:center line:84% then it will slowly kind of read backwards up the tree 00:27:55.140 --> 00:27:58.050 align:center line:84% to tell you, you're in this if, you're in this method, 00:27:58.050 --> 00:27:59.970 align:center line:90% and you're in this class. 00:27:59.970 --> 00:28:02.940 align:center line:84% And if users want to, they can just skip down 00:28:02.940 --> 00:28:04.942 align:center line:84% and that will sort of cancel the reading, 00:28:04.942 --> 00:28:06.400 align:center line:84% and it will read out the next item. 00:28:06.400 --> 00:28:08.910 align:center line:84% So it doesn't interfere because it's later on, as we 00:28:08.910 --> 00:28:10.513 align:center line:90% talked about this hierarchy. 00:28:10.513 --> 00:28:12.180 align:center line:84% Because it's later on, it doesn't really 00:28:12.180 --> 00:28:13.650 align:center line:84% interfere with their understanding it just 00:28:13.650 --> 00:28:15.483 align:center line:84% gives them more information if they want to. 00:28:15.483 --> 00:28:17.910 align:center line:84% We can maybe tweak quite how much they want, 00:28:17.910 --> 00:28:20.100 align:center line:84% but the idea seems reasonable that you 00:28:20.100 --> 00:28:22.860 align:center line:84% get this reverse hierarchical context of where 00:28:22.860 --> 00:28:26.340 align:center line:84% you are from the most distinguishing low level, up 00:28:26.340 --> 00:28:28.140 align:center line:84% to the least distinguishing high level. 00:28:28.140 --> 00:28:30.780 align:center line:90% 00:28:30.780 --> 00:28:34.120 align:center line:84% So another thing we could maybe improve on is navigation. 00:28:34.120 --> 00:28:39.050 align:center line:84% And so we had this view already in Stride, 00:28:39.050 --> 00:28:41.870 align:center line:84% which we call Birdseye view, which is essentially 00:28:41.870 --> 00:28:43.490 align:center line:90% a bit like code folding. 00:28:43.490 --> 00:28:45.870 align:center line:84% And so just actually in preparing this talk, 00:28:45.870 --> 00:28:47.900 align:center line:84% I've made that properly accessible. 00:28:47.900 --> 00:28:52.670 align:center line:84% So I'll show you what that looks like and sounds like. 00:28:52.670 --> 00:28:54.710 align:center line:84% So when you hit a shortcut, it folds up 00:28:54.710 --> 00:28:55.940 align:center line:90% as it will in the second. 00:28:55.940 --> 00:28:56.360 align:center line:90% [VIDEO PLAYBACK] 00:28:56.360 --> 00:28:57.290 align:center line:84% SCREEN READER: [INAUDIBLE] with parameter list 00:28:57.290 --> 00:29:00.030 align:center line:84% less animal greater new foxes with public access and void 00:29:00.030 --> 00:29:01.850 align:center line:90% return type button. 00:29:01.850 --> 00:29:04.610 align:center line:84% Method incrementAge with private access and void 00:29:04.610 --> 00:29:06.730 align:center line:90% return type button. 00:29:06.730 --> 00:29:09.815 align:center line:84% Method, method, method giveBirth with parameter list less animal 00:29:09.815 --> 00:29:12.840 align:center line:84% greater new foxes with private access and void return 00:29:12.840 --> 00:29:13.680 align:center line:90% type of button. 00:29:13.680 --> 00:29:14.430 align:center line:90% [END PLAYBACK] 00:29:14.430 --> 00:29:16.472 align:center line:84% NEIL BROWN: So it kind of gives you a quicker way 00:29:16.472 --> 00:29:18.960 align:center line:84% to move up and down the code to see what methods 00:29:18.960 --> 00:29:22.028 align:center line:84% are within that class without having 00:29:22.028 --> 00:29:23.070 align:center line:90% sort of interfering bits. 00:29:23.070 --> 00:29:25.140 align:center line:84% So because it's folded away the documentation, 00:29:25.140 --> 00:29:28.050 align:center line:84% it no longer bothers reading out the documentation. 00:29:28.050 --> 00:29:31.140 align:center line:84% And simply there, I was just using the down cursor key 00:29:31.140 --> 00:29:32.860 align:center line:90% to go between the methods. 00:29:32.860 --> 00:29:36.897 align:center line:84% So it makes it kind of an easy way to navigate around the code 00:29:36.897 --> 00:29:39.480 align:center line:84% and find out what's there, and then you can press another key, 00:29:39.480 --> 00:29:41.220 align:center line:90% and it folds back out again. 00:29:41.220 --> 00:29:44.280 align:center line:90% 00:29:44.280 --> 00:29:48.208 align:center line:84% And so in general we've kind of tried to improve the movement 00:29:48.208 --> 00:29:50.250 align:center line:84% and give you different options with the keyboard. 00:29:50.250 --> 00:29:52.902 align:center line:84% So we've seen a lot of these, but up and down moves 00:29:52.902 --> 00:29:54.110 align:center line:90% the frame cursor up and down. 00:29:54.110 --> 00:29:56.990 align:center line:84% If you press right, it actually enters the text fields 00:29:56.990 --> 00:29:59.180 align:center line:84% in the frame and lets you manipulate the expressions 00:29:59.180 --> 00:29:59.810 align:center line:90% in there. 00:29:59.810 --> 00:30:01.490 align:center line:84% If you press left, it does the reverse. 00:30:01.490 --> 00:30:04.790 align:center line:84% So it comes back out, and then continually pressing left moves 00:30:04.790 --> 00:30:05.820 align:center line:90% back up the tree. 00:30:05.820 --> 00:30:08.780 align:center line:84% So if you think of it as moving left on the indent levels, 00:30:08.780 --> 00:30:11.120 align:center line:84% it's slowly taking you further and further back up 00:30:11.120 --> 00:30:12.652 align:center line:90% your ancestors. 00:30:12.652 --> 00:30:14.360 align:center line:84% And we also have this control up and down 00:30:14.360 --> 00:30:15.693 align:center line:90% that moves at the current level. 00:30:15.693 --> 00:30:18.515 align:center line:84% So we feel like this gives a good variety of options 00:30:18.515 --> 00:30:19.640 align:center line:90% for moving around the code. 00:30:19.640 --> 00:30:21.980 align:center line:84% But the nice thing about having frames rather than text 00:30:21.980 --> 00:30:24.620 align:center line:84% is it's quite constrained in a good way. 00:30:24.620 --> 00:30:27.200 align:center line:84% You don't sort of end up at a destination you don't really 00:30:27.200 --> 00:30:30.290 align:center line:84% want to be in, like the middle of a white space indents 00:30:30.290 --> 00:30:31.610 align:center line:90% and wondering where you are. 00:30:31.610 --> 00:30:34.160 align:center line:84% You're always sort of at a useful location with the frame 00:30:34.160 --> 00:30:36.080 align:center line:84% cursor, because you're always beneath-- 00:30:36.080 --> 00:30:39.830 align:center line:84% so just between or above a particular frame. 00:30:39.830 --> 00:30:42.228 align:center line:90% 00:30:42.228 --> 00:30:44.770 align:center line:84% So there's various challenges of working with screen readers, 00:30:44.770 --> 00:30:47.187 align:center line:84% and I thought it might be good to highlight some of these, 00:30:47.187 --> 00:30:48.980 align:center line:90% having worked on it for a bit. 00:30:48.980 --> 00:30:52.780 align:center line:84% They can be slightly frustrating pieces of software. 00:30:52.780 --> 00:30:54.730 align:center line:90% So cross-platform difficulties. 00:30:54.730 --> 00:30:58.090 align:center line:84% So Java and JavaFX, which is the GUI toolkit we use, 00:30:58.090 --> 00:31:01.540 align:center line:84% offers a unified accessibility API that, in theory, 00:31:01.540 --> 00:31:03.010 align:center line:90% works the same everywhere. 00:31:03.010 --> 00:31:04.450 align:center line:90% Of course, it doesn't. 00:31:04.450 --> 00:31:06.520 align:center line:84% There are differences between operating systems 00:31:06.520 --> 00:31:07.378 align:center line:90% as to what happens. 00:31:07.378 --> 00:31:09.670 align:center line:84% But actually there are probably even bigger differences 00:31:09.670 --> 00:31:12.190 align:center line:84% between different pieces of screen reader software. 00:31:12.190 --> 00:31:14.950 align:center line:84% So the one I used for these demonstrations 00:31:14.950 --> 00:31:19.330 align:center line:84% was NVDA, which is a popular free piece of software. 00:31:19.330 --> 00:31:21.670 align:center line:84% There's also a built-in one on Mac. 00:31:21.670 --> 00:31:23.620 align:center line:84% There's a built-in one on Windows. 00:31:23.620 --> 00:31:25.450 align:center line:84% There's some paid for screen readers. 00:31:25.450 --> 00:31:29.080 align:center line:84% And they all will read out something slightly different 00:31:29.080 --> 00:31:33.320 align:center line:84% depending on the context that's there, trying to be helpful. 00:31:33.320 --> 00:31:35.950 align:center line:84% And of course that can make designing and testing 00:31:35.950 --> 00:31:38.980 align:center line:84% this kind of thing awkward, because sometimes you 00:31:38.980 --> 00:31:41.530 align:center line:84% change something on the Windows and it works wonderfully 00:31:41.530 --> 00:31:44.128 align:center line:84% with NVDA, and then try it with a Mac built-in screen reader, 00:31:44.128 --> 00:31:45.670 align:center line:84% and all of a sudden you've completely 00:31:45.670 --> 00:31:48.010 align:center line:84% broken screen reader sort of accessibility. 00:31:48.010 --> 00:31:49.810 align:center line:84% We definitely had that a couple of times 00:31:49.810 --> 00:31:53.770 align:center line:90% when we tried to be too clever. 00:31:53.770 --> 00:31:55.760 align:center line:90% Punctuation is an issue. 00:31:55.760 --> 00:31:57.860 align:center line:84% So punctuation in natural language 00:31:57.860 --> 00:31:59.980 align:center line:84% is generally used for clarification or emphasis. 00:31:59.980 --> 00:32:02.830 align:center line:84% It's not necessarily a crucial part of things, 00:32:02.830 --> 00:32:05.320 align:center line:84% whereas of course in program code, it's critical. 00:32:05.320 --> 00:32:06.400 align:center line:90% It indicates structure. 00:32:06.400 --> 00:32:09.250 align:center line:84% We had that line with a curly bracket 00:32:09.250 --> 00:32:11.890 align:center line:84% on at the beginning that wasn't read out by the screen reader. 00:32:11.890 --> 00:32:14.620 align:center line:84% But actually whitespace is not any better. 00:32:14.620 --> 00:32:16.660 align:center line:84% If you've got whitespace sensitive languages 00:32:16.660 --> 00:32:19.055 align:center line:84% like Python, it can be difficult with a screen reader 00:32:19.055 --> 00:32:21.055 align:center line:84% to know if you've got the right number of spaces 00:32:21.055 --> 00:32:24.430 align:center line:84% so that it lines up with the line above. 00:32:24.430 --> 00:32:27.830 align:center line:84% And it's not solved by moving to frame based editing either. 00:32:27.830 --> 00:32:28.840 align:center line:90% We still have brackets. 00:32:28.840 --> 00:32:29.500 align:center line:90% We have plus. 00:32:29.500 --> 00:32:31.630 align:center line:90% We have minus. 00:32:31.630 --> 00:32:33.940 align:center line:84% And so those aren't always read out quite 00:32:33.940 --> 00:32:36.790 align:center line:90% as you would think you would. 00:32:36.790 --> 00:32:40.420 align:center line:84% So one solution is to do something with like Quorum. 00:32:40.420 --> 00:32:44.470 align:center line:84% So Quorum is designed to reduce the amount of punctuation 00:32:44.470 --> 00:32:46.990 align:center line:84% and weird curly brackets and stuff 00:32:46.990 --> 00:32:52.270 align:center line:90% to use more text based syntax. 00:32:52.270 --> 00:32:54.698 align:center line:84% But again, like ours it's still got pluses and brackets 00:32:54.698 --> 00:32:55.990 align:center line:90% and your expression and things. 00:32:55.990 --> 00:32:59.912 align:center line:90% You can't escape it entirely. 00:32:59.912 --> 00:33:02.370 align:center line:84% I was reminded of this quote that I gave in a separate talk 00:33:02.370 --> 00:33:05.580 align:center line:84% last week, which from Kahneman said, 00:33:05.580 --> 00:33:07.470 align:center line:84% "I knew all I needed to know, but I did not 00:33:07.470 --> 00:33:08.440 align:center line:90% think it through." 00:33:08.440 --> 00:33:10.590 align:center line:84% And that's kind of summed up screen readers for me 00:33:10.590 --> 00:33:13.160 align:center line:84% because they actually have all the necessary information. 00:33:13.160 --> 00:33:16.230 align:center line:84% It's not a failure of our system or a Visual Studio 00:33:16.230 --> 00:33:18.610 align:center line:84% or whatever to not supply the punctuation, 00:33:18.610 --> 00:33:20.610 align:center line:84% the screen reader has the literal string. 00:33:20.610 --> 00:33:23.310 align:center line:84% It just decides not to read it out to you. 00:33:23.310 --> 00:33:25.618 align:center line:84% So it's a problem with a screen reader. 00:33:25.618 --> 00:33:26.910 align:center line:90% And I can see why they do that. 00:33:26.910 --> 00:33:28.590 align:center line:84% They're not built only for programming. 00:33:28.590 --> 00:33:30.007 align:center line:84% They're built for other people who 00:33:30.007 --> 00:33:31.770 align:center line:84% don't have to worry about punctuation, 00:33:31.770 --> 00:33:34.050 align:center line:84% but it seems generally quite difficult or actually 00:33:34.050 --> 00:33:36.330 align:center line:84% impossible with some of the built-in screen 00:33:36.330 --> 00:33:38.190 align:center line:84% readers on Windows and Mac to actually get 00:33:38.190 --> 00:33:41.800 align:center line:84% it to read out the necessary information for the screen 00:33:41.800 --> 00:33:42.300 align:center line:90% reader. 00:33:42.300 --> 00:33:44.880 align:center line:90% 00:33:44.880 --> 00:33:47.610 align:center line:84% Another thing is that the screen reader is kind of declarative, 00:33:47.610 --> 00:33:48.480 align:center line:90% not imperative. 00:33:48.480 --> 00:33:52.410 align:center line:84% So in general, you're kind of presenting a sort of structure 00:33:52.410 --> 00:33:55.260 align:center line:84% to the screen reader, and it will decide what to read out 00:33:55.260 --> 00:33:56.682 align:center line:90% and when to read it out. 00:33:56.682 --> 00:33:58.140 align:center line:84% So in a way, it's a good principle. 00:33:58.140 --> 00:34:01.260 align:center line:84% The user is in control, and the only speech happens in response 00:34:01.260 --> 00:34:04.800 align:center line:84% to them performing an action, often moving the focus 00:34:04.800 --> 00:34:06.840 align:center line:90% from one component to another. 00:34:06.840 --> 00:34:08.940 align:center line:84% And they don't get speech interrupting them 00:34:08.940 --> 00:34:11.552 align:center line:84% at random times, which I think would be frustrating. 00:34:11.552 --> 00:34:13.469 align:center line:84% But at the same time, there's not much support 00:34:13.469 --> 00:34:15.370 align:center line:90% for interrogating other data. 00:34:15.370 --> 00:34:17.190 align:center line:84% So let's say you move into a text 00:34:17.190 --> 00:34:18.900 align:center line:84% field in the frame based editing, 00:34:18.900 --> 00:34:19.980 align:center line:90% there's an error there. 00:34:19.980 --> 00:34:21.239 align:center line:90% How do we indicate that? 00:34:21.239 --> 00:34:24.471 align:center line:84% There's not an obvious way to tell the screen reader there's 00:34:24.471 --> 00:34:26.429 align:center line:84% something else here the user might want to know 00:34:26.429 --> 00:34:28.409 align:center line:90% about this being an error. 00:34:28.409 --> 00:34:30.570 align:center line:84% Or if the user wants to ask more information. 00:34:30.570 --> 00:34:32.040 align:center line:90% They might want to know-- 00:34:32.040 --> 00:34:35.699 align:center line:84% maybe refresh themselves as their position in the code, 00:34:35.699 --> 00:34:38.010 align:center line:84% or ask for some other information 00:34:38.010 --> 00:34:39.600 align:center line:90% about the item they're beneath. 00:34:39.600 --> 00:34:41.639 align:center line:84% And there's often not good ways to actually 00:34:41.639 --> 00:34:44.250 align:center line:84% do that-- both in terms of a standard way for the user 00:34:44.250 --> 00:34:47.429 align:center line:84% to ask for more information, but also to give it back, 00:34:47.429 --> 00:34:50.639 align:center line:84% because generally the screen reader only acts, like I say, 00:34:50.639 --> 00:34:53.489 align:center line:84% when you change focus on an item perhaps. 00:34:53.489 --> 00:34:55.139 align:center line:84% So you then need to maybe sort of 00:34:55.139 --> 00:34:57.570 align:center line:84% have a pop-up in response to the key that they've pressed, 00:34:57.570 --> 00:34:59.070 align:center line:84% but then that's awkward because they 00:34:59.070 --> 00:35:00.820 align:center line:90% have to dismiss the pop up. 00:35:00.820 --> 00:35:04.360 align:center line:84% And so it's kind of extra clicks to get where they want to go. 00:35:04.360 --> 00:35:07.290 align:center line:84% So that was something we found difficult in trying to design 00:35:07.290 --> 00:35:09.442 align:center line:90% more interesting interfaces. 00:35:09.442 --> 00:35:11.400 align:center line:84% We've sort of run up against this screen reader 00:35:11.400 --> 00:35:14.410 align:center line:84% model where it's like, no, it doesn't work like that. 00:35:14.410 --> 00:35:17.340 align:center line:84% And basically you sort of can't add the extra information 00:35:17.340 --> 00:35:20.810 align:center line:90% that you want, not easily. 00:35:20.810 --> 00:35:24.500 align:center line:84% And on that theme, so screen readers generally 00:35:24.500 --> 00:35:27.410 align:center line:84% have a sort of idea about common GUI widgets, 00:35:27.410 --> 00:35:29.870 align:center line:90% especially kind of 1990s or so. 00:35:29.870 --> 00:35:31.730 align:center line:90% So there's text fields. 00:35:31.730 --> 00:35:32.670 align:center line:90% There's buttons. 00:35:32.670 --> 00:35:33.650 align:center line:90% There's labels. 00:35:33.650 --> 00:35:34.790 align:center line:90% There's lists of texts. 00:35:34.790 --> 00:35:36.175 align:center line:90% There's tree views. 00:35:36.175 --> 00:35:37.550 align:center line:84% And when you want to do something 00:35:37.550 --> 00:35:39.830 align:center line:84% maybe a bit different, then you run into problems. 00:35:39.830 --> 00:35:41.450 align:center line:90% What is a frame cursor? 00:35:41.450 --> 00:35:42.890 align:center line:84% I said earlier I'd tell you why it 00:35:42.890 --> 00:35:44.270 align:center line:90% was saying button in the video. 00:35:44.270 --> 00:35:46.062 align:center line:84% That's because the frame cursor is actually 00:35:46.062 --> 00:35:48.500 align:center line:84% a button as far as the screen reader's concerned. 00:35:48.500 --> 00:35:50.540 align:center line:84% That doesn't really make a lot of sense, 00:35:50.540 --> 00:35:53.720 align:center line:84% but you have to make it something and button was 00:35:53.720 --> 00:35:55.220 align:center line:90% kind of the closest option. 00:35:55.220 --> 00:35:56.960 align:center line:84% Some screen readers will say button. 00:35:56.960 --> 00:35:57.800 align:center line:90% Some don't. 00:35:57.800 --> 00:36:00.008 align:center line:84% That's the difference between different screen reader 00:36:00.008 --> 00:36:01.040 align:center line:90% software. 00:36:01.040 --> 00:36:02.810 align:center line:84% But maybe you say, OK, well, that's 00:36:02.810 --> 00:36:04.790 align:center line:84% silly because in text you don't say that I've 00:36:04.790 --> 00:36:06.500 align:center line:90% got the text cursor focused. 00:36:06.500 --> 00:36:08.630 align:center line:84% I've just got the text area focused. 00:36:08.630 --> 00:36:11.390 align:center line:84% So we'll just say we've got the sort of frame area focused. 00:36:11.390 --> 00:36:13.610 align:center line:84% But what is that in the screen reader's mind? 00:36:13.610 --> 00:36:15.710 align:center line:84% It's got to be a standard GUI control. 00:36:15.710 --> 00:36:18.710 align:center line:84% Maybe it's a list or a tree, but then both of those 00:36:18.710 --> 00:36:21.920 align:center line:84% are generally expected to be a list or a tree of text items. 00:36:21.920 --> 00:36:23.600 align:center line:84% But we don't have a tree of text items. 00:36:23.600 --> 00:36:25.010 align:center line:84% We've got a very complicated tree 00:36:25.010 --> 00:36:28.970 align:center line:84% of frames, which inside it has sort of a list of text fields. 00:36:28.970 --> 00:36:31.940 align:center line:84% And so you just sort of run into this restriction 00:36:31.940 --> 00:36:34.370 align:center line:84% that it's got a kind of look to the screen reader 00:36:34.370 --> 00:36:36.710 align:center line:90% like a sort of Windows 3.1 GUI. 00:36:36.710 --> 00:36:39.110 align:center line:84% And how can we actually take what 00:36:39.110 --> 00:36:42.050 align:center line:84% we want to have, and manipulate it, and present it 00:36:42.050 --> 00:36:44.880 align:center line:84% to the screen reader to make sense. 00:36:44.880 --> 00:36:47.210 align:center line:84% So I mentioned this to Stefik at one point. 00:36:47.210 --> 00:36:49.910 align:center line:84% He said maybe try making it a tree view, 00:36:49.910 --> 00:36:52.790 align:center line:84% but I had a little go with that, but again, 00:36:52.790 --> 00:36:55.340 align:center line:84% we got into this problem that the screen reader is fetching 00:36:55.340 --> 00:36:57.410 align:center line:84% a very classic tree view and because ours 00:36:57.410 --> 00:37:00.140 align:center line:84% is more complex, at some point, it would just kind of 00:37:00.140 --> 00:37:01.460 align:center line:90% break the system. 00:37:01.460 --> 00:37:05.090 align:center line:84% So it is a bit of a challenge to take novel interfaces 00:37:05.090 --> 00:37:10.680 align:center line:84% and fit them into the screen readers worldview. 00:37:10.680 --> 00:37:13.380 align:center line:84% So there's definitely room for improvement. 00:37:13.380 --> 00:37:15.380 align:center line:84% So we actually had a meeting about accessibility 00:37:15.380 --> 00:37:17.840 align:center line:84% at some point where we presented all this to Michael, 00:37:17.840 --> 00:37:19.640 align:center line:90% who's the head of our group. 00:37:19.640 --> 00:37:22.250 align:center line:84% And he said, well, should we just abandon this and work 00:37:22.250 --> 00:37:23.660 align:center line:90% on a better screen reader? 00:37:23.660 --> 00:37:26.450 align:center line:84% And, OK, that's way outside our scope and a bigger job 00:37:26.450 --> 00:37:28.980 align:center line:84% than it should be, but we're not going to do that. 00:37:28.980 --> 00:37:30.410 align:center line:90% But he's not wrong in a sense. 00:37:30.410 --> 00:37:32.540 align:center line:84% Some of the problems that we're running into 00:37:32.540 --> 00:37:34.207 align:center line:84% are kind of caused by the screen reader. 00:37:34.207 --> 00:37:36.123 align:center line:84% That's why as I understand it, Stefik actually 00:37:36.123 --> 00:37:38.420 align:center line:84% ended up sort of mostly replacing the screen reader. 00:37:38.420 --> 00:37:39.680 align:center line:90% Is that right? 00:37:39.680 --> 00:37:42.290 align:center line:90% If you give me a nod? 00:37:42.290 --> 00:37:42.790 align:center line:90% No. 00:37:42.790 --> 00:37:42.970 align:center line:90% No. 00:37:42.970 --> 00:37:43.660 align:center line:90% OK, not quite. 00:37:43.660 --> 00:37:44.140 align:center line:90% So he can talk more about that. 00:37:44.140 --> 00:37:46.330 align:center line:84% ANDREAS STEFIK: On a very early project, we did. 00:37:46.330 --> 00:37:49.330 align:center line:84% On a very early project that we did, but that had all 00:37:49.330 --> 00:37:53.025 align:center line:84% sorts of other complexity that I won't bore people with. 00:37:53.025 --> 00:37:53.650 align:center line:90% NEIL BROWN: OK. 00:37:53.650 --> 00:37:56.835 align:center line:84% So basically it's hard to get rid of them. 00:37:56.835 --> 00:37:58.460 align:center line:84% You've kind of got to live within them. 00:37:58.460 --> 00:38:01.220 align:center line:84% One possibility I saw NVDA allows add-ons. 00:38:01.220 --> 00:38:03.070 align:center line:84% I haven't looked hugely into that yet, 00:38:03.070 --> 00:38:05.770 align:center line:84% but maybe that's a route to have an add on that actually reads 00:38:05.770 --> 00:38:08.980 align:center line:84% out punctuation like a sort of programmer-friendly mode 00:38:08.980 --> 00:38:13.320 align:center line:84% perhaps for text and for blocks and frames. 00:38:13.320 --> 00:38:16.770 align:center line:84% So just a few final bits then to sum up. 00:38:16.770 --> 00:38:18.930 align:center line:84% So we're planning to continue developing 00:38:18.930 --> 00:38:20.550 align:center line:84% Stride, which is the editor I've been 00:38:20.550 --> 00:38:22.350 align:center line:90% showing for screen readers. 00:38:22.350 --> 00:38:25.230 align:center line:84% We want to add similar support to our new Strype system. 00:38:25.230 --> 00:38:27.570 align:center line:84% So Strype is another frame based editor, 00:38:27.570 --> 00:38:30.605 align:center line:84% but it lives in the web browser, which 00:38:30.605 --> 00:38:31.980 align:center line:84% is a whole new set of challenges, 00:38:31.980 --> 00:38:34.050 align:center line:84% I imagine, to make it accessible. 00:38:34.050 --> 00:38:36.420 align:center line:84% And we're interested to connect with potential users. 00:38:36.420 --> 00:38:38.220 align:center line:84% We're hoping in the new year to return 00:38:38.220 --> 00:38:40.380 align:center line:84% to do some more development on this, 00:38:40.380 --> 00:38:43.650 align:center line:84% and it would be good to actually have some users who 00:38:43.650 --> 00:38:47.310 align:center line:84% are expert in this area, who are vision impaired themselves 00:38:47.310 --> 00:38:50.770 align:center line:84% in order to sort of test out some of these ideas. 00:38:50.770 --> 00:38:53.967 align:center line:84% So let us know if you're interested. 00:38:53.967 --> 00:38:56.050 align:center line:84% If you want to try the work at its current state-- 00:38:56.050 --> 00:38:58.320 align:center line:84% so pretty much everything you saw today 00:38:58.320 --> 00:39:01.620 align:center line:84% in Stride is there except for the folding view because I only 00:39:01.620 --> 00:39:03.750 align:center line:84% fixed that a few days ago when I realized 00:39:03.750 --> 00:39:05.470 align:center line:90% it wasn't very accessible. 00:39:05.470 --> 00:39:07.660 align:center line:84% But it's in our BlueJ and Greenfoot system. 00:39:07.660 --> 00:39:09.270 align:center line:84% So if you download BlueJ or Greenfoot, 00:39:09.270 --> 00:39:10.770 align:center line:84% when you make a new class, there's 00:39:10.770 --> 00:39:12.360 align:center line:90% a Java or Stride toggle. 00:39:12.360 --> 00:39:14.310 align:center line:84% If you do Java, you get text based editing. 00:39:14.310 --> 00:39:17.070 align:center line:84% If you say Stride, you get frame based editing. 00:39:17.070 --> 00:39:19.590 align:center line:84% Strype is available, but we haven't yet 00:39:19.590 --> 00:39:21.460 align:center line:84% worked on the accessibility for that. 00:39:21.460 --> 00:39:23.550 align:center line:84% So it's not yet properly accessible, 00:39:23.550 --> 00:39:26.190 align:center line:84% but that's available at Strype.org if you're 00:39:26.190 --> 00:39:27.670 align:center line:90% interested to try that out. 00:39:27.670 --> 00:39:30.560 align:center line:90% So Strype with a y. 00:39:30.560 --> 00:39:33.590 align:center line:84% So in summary, the current state of block accessibility 00:39:33.590 --> 00:39:35.420 align:center line:90% is pretty terrible. 00:39:35.420 --> 00:39:37.640 align:center line:84% There have been some attempts at solutions. 00:39:37.640 --> 00:39:40.720 align:center line:84% All of them seem to involve some notion of cursor or focus. 00:39:40.720 --> 00:39:43.310 align:center line:84% So I think it's fair to say that's a critical element 00:39:43.310 --> 00:39:46.310 align:center line:84% to provide accessibility for a screen reader. 00:39:46.310 --> 00:39:47.903 align:center line:84% Even once you implement the basics, 00:39:47.903 --> 00:39:49.820 align:center line:84% there's then the sort of high level challenges 00:39:49.820 --> 00:39:52.040 align:center line:84% around things like navigation, around skimming. 00:39:52.040 --> 00:39:53.750 align:center line:84% I think that's when it gets particularly 00:39:53.750 --> 00:39:56.870 align:center line:84% interesting from a designer's point of view. 00:39:56.870 --> 00:39:59.210 align:center line:84% But some aspects of screen readers provide a challenge. 00:39:59.210 --> 00:40:01.160 align:center line:84% You've got to fit into their worldview. 00:40:01.160 --> 00:40:04.220 align:center line:84% They're not reading out punctuation properly. 00:40:04.220 --> 00:40:07.280 align:center line:84% So although they're necessary, they're 00:40:07.280 --> 00:40:10.080 align:center line:84% also quite a hindrance in what you want to do. 00:40:10.080 --> 00:40:11.600 align:center line:84% So we had some ideas, and I've shown 00:40:11.600 --> 00:40:13.190 align:center line:84% you those about developing support 00:40:13.190 --> 00:40:16.310 align:center line:84% that's unique to frames, but actually the point I'd make 00:40:16.310 --> 00:40:19.040 align:center line:84% is although it's pretty terrible in blocks at the moment, 00:40:19.040 --> 00:40:22.010 align:center line:84% I think blocks or frames, if it's done correctly, 00:40:22.010 --> 00:40:24.500 align:center line:84% could actually potentially be more accessible than text 00:40:24.500 --> 00:40:27.020 align:center line:84% because you don't have this issue with kind 00:40:27.020 --> 00:40:30.110 align:center line:84% of having to navigate around a lot of whitespace and quite so 00:40:30.110 --> 00:40:31.820 align:center line:90% much punctuation. 00:40:31.820 --> 00:40:35.120 align:center line:84% You can do things like the sort of structural context a lot 00:40:35.120 --> 00:40:35.720 align:center line:90% more easily. 00:40:35.720 --> 00:40:38.660 align:center line:84% Just think about reordering which order things are read out 00:40:38.660 --> 00:40:41.610 align:center line:84% in, you can do in frames more easily than you can in text. 00:40:41.610 --> 00:40:44.120 align:center line:84% So I think, in theory, a proper sort of structured editor 00:40:44.120 --> 00:40:47.360 align:center line:84% like blocks and frames could be more accessible than text. 00:40:47.360 --> 00:40:48.980 align:center line:84% It's just that at the moment we're 00:40:48.980 --> 00:40:51.830 align:center line:84% sort of starting from nothing, but I 00:40:51.830 --> 00:40:54.380 align:center line:84% think the ceiling is actually potentially higher. 00:40:54.380 --> 00:40:56.180 align:center line:84% So thank you very much for listening, 00:40:56.180 --> 00:40:58.190 align:center line:84% and I'm happy to move to questions. 00:40:58.190 --> 00:41:00.095 align:center line:84% I guess, Stefik, you will say what's next. 00:41:00.095 --> 00:41:02.567 align:center line:90% 00:41:02.567 --> 00:41:03.900 align:center line:90% ANDREAS STEFIK: Thank you, Neil. 00:41:03.900 --> 00:41:06.220 align:center line:84% That was really, really fantastic. 00:41:06.220 --> 00:41:08.453 align:center line:84% I'm super excited about all the stuff 00:41:08.453 --> 00:41:09.870 align:center line:84% that you guys have been doing just 00:41:09.870 --> 00:41:13.230 align:center line:84% because, like you, said the state of things 00:41:13.230 --> 00:41:17.448 align:center line:84% is so nonfunctional that you could throw a dart 00:41:17.448 --> 00:41:19.740 align:center line:84% and come up with something that's better than something 00:41:19.740 --> 00:41:21.090 align:center line:90% that doesn't read at all. 00:41:21.090 --> 00:41:23.790 align:center line:84% But having more scholars thinking 00:41:23.790 --> 00:41:26.460 align:center line:84% about more things in this space is a really big deal 00:41:26.460 --> 00:41:28.620 align:center line:84% because we need lots of ideas to get this right. 00:41:28.620 --> 00:41:30.790 align:center line:90% So I'm just saying, thank you. 00:41:30.790 --> 00:41:32.850 align:center line:84% But at this point, I will open it up 00:41:32.850 --> 00:41:34.500 align:center line:90% to questions from anybody. 00:41:34.500 --> 00:41:38.040 align:center line:84% You're welcome to either flip on your camera if you'd like to, 00:41:38.040 --> 00:41:39.930 align:center line:84% or raise your hand if you prefer, 00:41:39.930 --> 00:41:41.190 align:center line:90% or type something in the chat. 00:41:41.190 --> 00:41:45.203 align:center line:84% Whatever works for you is just fine. 00:41:45.203 --> 00:41:46.870 align:center line:84% If no one has a question, I have plenty, 00:41:46.870 --> 00:41:48.460 align:center line:84% but I don't want to capitalize either. 00:41:48.460 --> 00:41:59.440 align:center line:90% 00:41:59.440 --> 00:42:00.220 align:center line:90% OK, well, fine. 00:42:00.220 --> 00:42:03.070 align:center line:84% Then I'll steal the first question, then. 00:42:03.070 --> 00:42:05.835 align:center line:84% So I had-- one of the things that we've 00:42:05.835 --> 00:42:07.210 align:center line:84% been thinking about on ours, too, 00:42:07.210 --> 00:42:09.640 align:center line:84% because we've been really inspired by the Strype work 00:42:09.640 --> 00:42:13.660 align:center line:84% that you guys have done, is there's this thing that you 00:42:13.660 --> 00:42:18.730 align:center line:84% mentioned where in a frame you use the typing for many 00:42:18.730 --> 00:42:21.400 align:center line:84% of the expressions, which is something that we will probably 00:42:21.400 --> 00:42:22.630 align:center line:90% end up doing as well. 00:42:22.630 --> 00:42:27.700 align:center line:84% But obviously, the actual block editors usually don't do that. 00:42:27.700 --> 00:42:30.580 align:center line:84% You drag and drop them or you press keys or something 00:42:30.580 --> 00:42:33.910 align:center line:84% hypothetically for even expressions like plus and minus 00:42:33.910 --> 00:42:36.220 align:center line:90% and stuff like that. 00:42:36.220 --> 00:42:39.190 align:center line:84% Do you think that typing will have-- 00:42:39.190 --> 00:42:40.690 align:center line:84% what do you think the impact will be 00:42:40.690 --> 00:42:42.655 align:center line:90% on younger versus older people? 00:42:42.655 --> 00:42:44.530 align:center line:84% One of the considerations we've been thinking 00:42:44.530 --> 00:42:47.770 align:center line:84% is if you have a kid that's like six or seven, 00:42:47.770 --> 00:42:50.680 align:center line:84% is that a downside or a good thing? 00:42:50.680 --> 00:42:54.130 align:center line:84% Or what are you thinking about the various ages related 00:42:54.130 --> 00:42:56.510 align:center line:90% to blocks with that decision? 00:42:56.510 --> 00:42:58.870 align:center line:84% NEIL BROWN: Yeah, so I think we thought definitely 00:42:58.870 --> 00:43:02.140 align:center line:84% for someone who's more proficient, it's quicker. 00:43:02.140 --> 00:43:04.750 align:center line:84% Like 1 plus 2 is quicker than dragging in a plus. 00:43:04.750 --> 00:43:06.610 align:center line:84% Like it seems fairly obvious once 00:43:06.610 --> 00:43:08.260 align:center line:90% you're sure what you want to do. 00:43:08.260 --> 00:43:10.450 align:center line:84% I think something that's also a bit 00:43:10.450 --> 00:43:12.820 align:center line:84% underappreciated with the block dragging model 00:43:12.820 --> 00:43:15.650 align:center line:84% is that it forces you into a prefix mode. 00:43:15.650 --> 00:43:19.252 align:center line:84% So if I want to do type 1 plus 2, I'm 1 then plus then 2. 00:43:19.252 --> 00:43:21.460 align:center line:84% Whereas if I want to do the blocks, I've got to think 00:43:21.460 --> 00:43:22.400 align:center line:90% what's the operator? 00:43:22.400 --> 00:43:23.140 align:center line:90% So it's plus. 00:43:23.140 --> 00:43:24.820 align:center line:84% So it's more like a scheme type thing 00:43:24.820 --> 00:43:28.752 align:center line:84% where I'm doing plus 1, 2 in order to make it prefixed. 00:43:28.752 --> 00:43:31.210 align:center line:84% And at the risk of offending any scheme people on the call, 00:43:31.210 --> 00:43:33.070 align:center line:90% I think infix is better. 00:43:33.070 --> 00:43:36.880 align:center line:84% It just-- it's how we're taught in-- especially for younger 00:43:36.880 --> 00:43:40.000 align:center line:84% people, we teach them in math and we teach infix. 00:43:40.000 --> 00:43:43.250 align:center line:84% So to then say, oh now you've got to sort of enter a prefix, 00:43:43.250 --> 00:43:46.300 align:center line:84% like it just seems like a bit of extra cognitive load that's 00:43:46.300 --> 00:43:47.710 align:center line:90% going to sort of wear on them. 00:43:47.710 --> 00:43:50.030 align:center line:84% So I don't really see a huge advantage, 00:43:50.030 --> 00:43:53.290 align:center line:84% especially in like plus minus times of having 00:43:53.290 --> 00:43:54.850 align:center line:90% the block model to drag in. 00:43:54.850 --> 00:43:57.370 align:center line:84% It seems like an easy win to change it to text. 00:43:57.370 --> 00:43:59.170 align:center line:84% Something more like the method calls, 00:43:59.170 --> 00:44:01.210 align:center line:84% OK, we can have all kinds of debate about that 00:44:01.210 --> 00:44:03.675 align:center line:84% because if you want to drag in like a square root 00:44:03.675 --> 00:44:05.050 align:center line:84% or something, maybe that's easier 00:44:05.050 --> 00:44:06.417 align:center line:90% than typing the square root. 00:44:06.417 --> 00:44:07.750 align:center line:90% Maybe it's a different decision. 00:44:07.750 --> 00:44:09.208 align:center line:84% But I think at least for operators, 00:44:09.208 --> 00:44:12.250 align:center line:84% I'm confident that typing is just better. 00:44:12.250 --> 00:44:13.730 align:center line:90% ANDREAS STEFIK: Interesting. 00:44:13.730 --> 00:44:14.230 align:center line:90% Thank you. 00:44:14.230 --> 00:44:15.370 align:center line:90% Richard, go ahead. 00:44:15.370 --> 00:44:18.070 align:center line:90% 00:44:18.070 --> 00:44:20.450 align:center line:84% RICHARD LADNER: Neil, thank you very much. 00:44:20.450 --> 00:44:23.650 align:center line:84% That was fantastic and really enjoyed that. 00:44:23.650 --> 00:44:26.860 align:center line:84% Getting back to Stefix's question a little bit more, 00:44:26.860 --> 00:44:30.100 align:center line:84% a lot of the block's programming environments 00:44:30.100 --> 00:44:32.380 align:center line:84% were designed for little children-- 00:44:32.380 --> 00:44:35.440 align:center line:90% like Stefix said, six-year-olds. 00:44:35.440 --> 00:44:39.100 align:center line:84% And for blind children, at least in the United States, 00:44:39.100 --> 00:44:42.820 align:center line:84% they're introduced to the iPad fairly early. 00:44:42.820 --> 00:44:46.030 align:center line:84% And so they learn voiceover on the iPad 00:44:46.030 --> 00:44:49.660 align:center line:84% and are very proficient at a pretty young age. 00:44:49.660 --> 00:44:51.490 align:center line:84% And so that was one of the motivations 00:44:51.490 --> 00:44:55.540 align:center line:84% for creating Blocks For All with Lauren Milne, 00:44:55.540 --> 00:44:59.420 align:center line:84% so that there would be some alternative that's accessible. 00:44:59.420 --> 00:45:04.520 align:center line:84% So if you have any thoughts about what you describe 00:45:04.520 --> 00:45:08.900 align:center line:84% today would be fine for high school maybe, 00:45:08.900 --> 00:45:11.693 align:center line:90% but what about younger children? 00:45:11.693 --> 00:45:13.860 align:center line:84% NEIL BROWN: Yeah, so I think there's sort of there's 00:45:13.860 --> 00:45:14.470 align:center line:90% two issues. 00:45:14.470 --> 00:45:16.690 align:center line:84% One is the voice entry, which is something 00:45:16.690 --> 00:45:17.940 align:center line:90% we haven't really looked into. 00:45:17.940 --> 00:45:19.470 align:center line:84% I think is a very interesting area, 00:45:19.470 --> 00:45:23.790 align:center line:84% but we've mainly been focused on keyboard entry and screen 00:45:23.790 --> 00:45:26.670 align:center line:84% reader display, rather than alternative sort of methods 00:45:26.670 --> 00:45:27.630 align:center line:90% of entry. 00:45:27.630 --> 00:45:29.880 align:center line:84% So I think that's interesting, but it's just something 00:45:29.880 --> 00:45:32.452 align:center line:84% that we haven't looked at all either the way. 00:45:32.452 --> 00:45:33.910 align:center line:84% I think the other aspect about age. 00:45:33.910 --> 00:45:36.510 align:center line:84% I mean, obviously in our editor, you can see that. 00:45:36.510 --> 00:45:38.460 align:center line:84% We've got a very Java like syntax. 00:45:38.460 --> 00:45:41.580 align:center line:84% It's a Java semantics because our existing tools were all 00:45:41.580 --> 00:45:44.587 align:center line:84% Java, and so we were doing Stride to then reach Java. 00:45:44.587 --> 00:45:46.920 align:center line:84% But I think the decisions we've made for the frame-based 00:45:46.920 --> 00:45:48.030 align:center line:90% editor-- 00:45:48.030 --> 00:45:50.460 align:center line:84% if we wanted to make a Quorum frame-based editor 00:45:50.460 --> 00:45:52.950 align:center line:84% or some other language that was even 00:45:52.950 --> 00:45:55.163 align:center line:84% simpler and suitable for kids, then I 00:45:55.163 --> 00:45:56.580 align:center line:84% think some of our design decisions 00:45:56.580 --> 00:45:59.250 align:center line:90% would transfer reasonably. 00:45:59.250 --> 00:46:01.200 align:center line:84% It's not wedded to the fact, that we've 00:46:01.200 --> 00:46:03.150 align:center line:84% got this Java semantics that's just 00:46:03.150 --> 00:46:05.400 align:center line:84% a particular implementation of our language. 00:46:05.400 --> 00:46:07.170 align:center line:84% But I think the idea is of frames 00:46:07.170 --> 00:46:10.470 align:center line:84% and frame manipulation would transfer to other sort 00:46:10.470 --> 00:46:13.182 align:center line:90% of simpler languages. 00:46:13.182 --> 00:46:14.640 align:center line:84% ANDREAS STEFIK: We'll find out soon 00:46:14.640 --> 00:46:16.723 align:center line:84% because we literally copied that part of our stuff 00:46:16.723 --> 00:46:17.490 align:center line:90% for Quorum blocks. 00:46:17.490 --> 00:46:18.900 align:center line:90% That was not an accident. 00:46:18.900 --> 00:46:21.710 align:center line:84% So it's your fault when it fails, Neil. 00:46:21.710 --> 00:46:22.960 align:center line:90% I just want to point that out. 00:46:22.960 --> 00:46:25.020 align:center line:90% So [LAUGHS]. 00:46:25.020 --> 00:46:26.640 align:center line:90% No, I think it's a good idea. 00:46:26.640 --> 00:46:27.480 align:center line:90% Sorry. 00:46:27.480 --> 00:46:29.105 align:center line:84% Richard, did you have another question? 00:46:29.105 --> 00:46:31.258 align:center line:90% 00:46:31.258 --> 00:46:32.050 align:center line:90% RICHARD LADNER: No. 00:46:32.050 --> 00:46:34.180 align:center line:84% I've got to figure out how to lower my hand, though. 00:46:34.180 --> 00:46:35.305 align:center line:90% ANDREAS STEFIK: Oh, gotcha. 00:46:35.305 --> 00:46:37.410 align:center line:90% OK. 00:46:37.410 --> 00:46:39.850 align:center line:84% Jim Salmons asked, if we'll be sending out a recording. 00:46:39.850 --> 00:46:41.922 align:center line:84% And we will be sending out a recording. 00:46:41.922 --> 00:46:43.630 align:center line:84% We always do the closed captioning first, 00:46:43.630 --> 00:46:45.547 align:center line:84% just so that we can make sure it's accessible. 00:46:45.547 --> 00:46:49.150 align:center line:84% That does take a little while, but we will send out a video, 00:46:49.150 --> 00:46:52.360 align:center line:84% and we'll put it up on our website, which I imagine, 00:46:52.360 --> 00:46:54.607 align:center line:84% I just scooped Brianna by saying, but-- 00:46:54.607 --> 00:46:57.190 align:center line:84% BRIANNA BLASER: No, I was going to say that I will email folks 00:46:57.190 --> 00:46:58.670 align:center line:90% when that is available as well. 00:46:58.670 --> 00:46:59.830 align:center line:84% And I'm going to [CLEARS THROAT] excuse 00:46:59.830 --> 00:47:02.120 align:center line:84% me, drop a link in the chat for an evaluation in a second. 00:47:02.120 --> 00:47:03.995 align:center line:84% So you can fill that out and I will follow up 00:47:03.995 --> 00:47:06.403 align:center line:90% with that over email as well. 00:47:06.403 --> 00:47:08.320 align:center line:84% NEIL BROWN: I'm also happy to share my slides, 00:47:08.320 --> 00:47:10.330 align:center line:84% although ironically, because I've got some demo 00:47:10.330 --> 00:47:11.872 align:center line:84% videos in there, they're not actually 00:47:11.872 --> 00:47:15.910 align:center line:84% necessarily the most accessible by themselves 00:47:15.910 --> 00:47:17.080 align:center line:90% compared to the text. 00:47:17.080 --> 00:47:19.872 align:center line:84% But that was just kind of the nature. 00:47:19.872 --> 00:47:21.580 align:center line:84% ANDREAS STEFIK: Yeah, one thing we always 00:47:21.580 --> 00:47:25.600 align:center line:84% talk about with accessibility is perfect is 00:47:25.600 --> 00:47:26.890 align:center line:90% the enemy of the good. 00:47:26.890 --> 00:47:31.750 align:center line:84% When you're talking about a state of the practice 00:47:31.750 --> 00:47:34.040 align:center line:84% where people can't get access at all, 00:47:34.040 --> 00:47:35.830 align:center line:84% sometimes you have to live with the fact 00:47:35.830 --> 00:47:37.190 align:center line:90% that something is not perfect. 00:47:37.190 --> 00:47:39.358 align:center line:84% So if your slides are not perfect, 00:47:39.358 --> 00:47:40.900 align:center line:84% we're not going to complain too much. 00:47:40.900 --> 00:47:43.070 align:center line:84% We actually really appreciate you doing work in this space, 00:47:43.070 --> 00:47:43.570 align:center line:90% really. 00:47:43.570 --> 00:47:46.300 align:center line:90% 00:47:46.300 --> 00:47:50.125 align:center line:84% Other questions or thoughts for Neil here? 00:47:50.125 --> 00:47:54.010 align:center line:90% 00:47:54.010 --> 00:47:54.580 align:center line:90% OK. 00:47:54.580 --> 00:47:58.810 align:center line:84% Well, then I just want to say thank you one more time. 00:47:58.810 --> 00:48:02.680 align:center line:84% I will probably reach out at some point to you, Neil, 00:48:02.680 --> 00:48:04.960 align:center line:84% as we get going and we can compare notes 00:48:04.960 --> 00:48:06.110 align:center line:90% once we have our-- 00:48:06.110 --> 00:48:08.500 align:center line:84% we'll actually be ready with Quorum stuff 00:48:08.500 --> 00:48:10.120 align:center line:90% in a couple of weeks with some-- 00:48:10.120 --> 00:48:12.010 align:center line:84% I mean, it's not quite ready yet. 00:48:12.010 --> 00:48:13.870 align:center line:84% But we should compare notes just sort 00:48:13.870 --> 00:48:16.850 align:center line:84% of figure out what's working and what isn't and stuff like that. 00:48:16.850 --> 00:48:18.400 align:center line:84% And I will mention one last thing 00:48:18.400 --> 00:48:20.787 align:center line:84% that you can do on the metadata side before you go. 00:48:20.787 --> 00:48:22.870 align:center line:84% There is a way to tell the operating system you've 00:48:22.870 --> 00:48:24.430 align:center line:90% got a spelling error. 00:48:24.430 --> 00:48:27.880 align:center line:84% It's not an error, per se, but it 00:48:27.880 --> 00:48:29.690 align:center line:84% does cause it to have this little click, 00:48:29.690 --> 00:48:31.190 align:center line:84% and it works pretty well for errors. 00:48:31.190 --> 00:48:33.370 align:center line:84% So if you can figure out how to hack it in, 00:48:33.370 --> 00:48:34.430 align:center line:90% that does actually work. 00:48:34.430 --> 00:48:35.805 align:center line:84% NEIL BROWN: Yeah, unfortunately I 00:48:35.805 --> 00:48:38.290 align:center line:84% don't think the Java API has support for that. 00:48:38.290 --> 00:48:38.380 align:center line:90% ANDREAS STEFIK: It does not. 00:48:38.380 --> 00:48:40.005 align:center line:84% NEIL BROWN: And you know, you're always 00:48:40.005 --> 00:48:41.818 align:center line:90% at the mercy of these like APIs. 00:48:41.818 --> 00:48:43.360 align:center line:84% We've got the Java API, and then it's 00:48:43.360 --> 00:48:44.770 align:center line:84% talking to the screen reader API. 00:48:44.770 --> 00:48:47.020 align:center line:84% And actually the fault often lies with a screen reader 00:48:47.020 --> 00:48:47.950 align:center line:90% API in my mind. 00:48:47.950 --> 00:48:50.960 align:center line:84% But the Java API is sometimes missing things. 00:48:50.960 --> 00:48:53.740 align:center line:84% There's also this kind of idea of a help key 00:48:53.740 --> 00:48:55.990 align:center line:84% that seems to exist, and Java will 00:48:55.990 --> 00:48:57.460 align:center line:90% allow you to respond to it. 00:48:57.460 --> 00:48:59.410 align:center line:84% But we found that in some screen readers 00:48:59.410 --> 00:49:02.260 align:center line:84% it is not quite clear how to trigger that 00:49:02.260 --> 00:49:05.093 align:center line:84% or how to make it apparent that you could trigger that. 00:49:05.093 --> 00:49:06.010 align:center line:90% So again, it's tricky. 00:49:06.010 --> 00:49:07.755 align:center line:84% ANDREAS STEFIK: Oh, yes on those. 00:49:07.755 --> 00:49:09.880 align:center line:84% There's also a whole bunch of other metadata things 00:49:09.880 --> 00:49:12.663 align:center line:84% like those that get sent down to the screen reader 00:49:12.663 --> 00:49:14.330 align:center line:84% and the screen reader's all ignore them. 00:49:14.330 --> 00:49:16.930 align:center line:84% So just be aware that that-- it might not even be your fault. 00:49:16.930 --> 00:49:20.230 align:center line:84% It might be that it's actually there, but that NVDA is like, 00:49:20.230 --> 00:49:21.190 align:center line:90% meh. 00:49:21.190 --> 00:49:23.150 align:center line:84% I'm not going to respond to that. 00:49:23.150 --> 00:49:25.510 align:center line:90% So it's not that unusual, sadly. 00:49:25.510 --> 00:49:26.462 align:center line:90% So yeah. 00:49:26.462 --> 00:49:27.670 align:center line:90% NEIL BROWN: It's challenging. 00:49:27.670 --> 00:49:28.240 align:center line:90% Yeah. 00:49:28.240 --> 00:49:29.865 align:center line:84% RICHARD LADNER: Yeah, there's a comment 00:49:29.865 --> 00:49:32.650 align:center line:90% from Jim Salmons in the chat. 00:49:32.650 --> 00:49:35.380 align:center line:84% And maybe Jim could you introduce yourself? 00:49:35.380 --> 00:49:42.080 align:center line:90% [INAUDIBLE] It he wants to? 00:49:42.080 --> 00:49:43.130 align:center line:90% He's still muted. 00:49:43.130 --> 00:49:44.820 align:center line:84% ANDREAS STEFIK: I'll read it out. 00:49:44.820 --> 00:49:47.090 align:center line:84% He's a disabled developer working developer research 00:49:47.090 --> 00:49:48.830 align:center line:84% study and support program using GitHub 00:49:48.830 --> 00:49:52.130 align:center line:84% Copilot for disabled developers and disabled STEM students. 00:49:52.130 --> 00:49:53.540 align:center line:90% Then he has a link in there. 00:49:53.540 --> 00:49:55.040 align:center line:84% In the immediate term, he's working 00:49:55.040 --> 00:49:58.040 align:center line:84% on GitHub next developers at the Hey GitHub voice coding 00:49:58.040 --> 00:49:59.150 align:center line:90% assistive tech. 00:49:59.150 --> 00:50:02.185 align:center line:84% We have a need for 20 to 25 disabled developers 00:50:02.185 --> 00:50:03.560 align:center line:84% to volunteer for a feedback group 00:50:03.560 --> 00:50:05.840 align:center line:90% to assess the current state. 00:50:05.840 --> 00:50:07.340 align:center line:90% He can be reached at-- 00:50:07.340 --> 00:50:12.260 align:center line:84% and then there's some links in there for going. 00:50:12.260 --> 00:50:15.170 align:center line:84% It's possible that we can send that on a mailing list 00:50:15.170 --> 00:50:18.102 align:center line:90% or something possibly? 00:50:18.102 --> 00:50:19.810 align:center line:84% BRIANNA BLASER: Yeah, I'll follow up Jim. 00:50:19.810 --> 00:50:21.990 align:center line:84% But it sounds really interesting work. 00:50:21.990 --> 00:50:23.160 align:center line:90% ANDREAS STEFIK: Yeah. 00:50:23.160 --> 00:50:24.390 align:center line:90% So. 00:50:24.390 --> 00:50:24.960 align:center line:90% In any case. 00:50:24.960 --> 00:50:25.830 align:center line:90% Thank you all. 00:50:25.830 --> 00:50:28.240 align:center line:84% And thank you especially to Neil for your time. 00:50:28.240 --> 00:50:29.490 align:center line:90% So we appreciate it a lot. 00:50:29.490 --> 00:50:31.065 align:center line:90% Thanks for coming today. 00:50:31.065 --> 00:50:31.690 align:center line:90% NEIL BROWN: OK. 00:50:31.690 --> 00:50:33.230 align:center line:90% Thanks very much. 00:50:33.230 --> 00:50:34.030 align:center line:90% Bye. 00:50:34.030 --> 00:50:35.630 align:center line:84% ANDREAS STEFIK: Bye-bye everyone. 00:50:35.630 --> 00:50:37.480 align:center line:90% BRIANNA BLASER: Bye. 00:50:37.480 --> 00:50:41.000 align:center line:90%