BRIANNA BLASER: We're excited to have you today
for this webinar on accessible tools for K-12 computer science
education.
If you're not aware, this is Computer Science Education
Week.
And so in the theme of CS Education Week,
this year is celebrating progress.
And I think it's--
as I've been thinking about it this week,
it's pretty exciting to celebrate
the advances in accessibility over the last few years
for students with disabilities.
So today, we're going to hear from Richard Ladner
at the University of Washington, Laura Milne at Macalester
College, and Andreas Stefik at the University of Nevada Las
Vegas.
Richard will be talking about what is accessibility.
Lauren will be talking about her tool, Blocks4All.
And Stefik will be talking about Quorum.
So I'm excited to have them with us today.
And I'll turn it over to Richard.
RICHARD LADNER: Hello, everybody.
I'm Richard Ladner, professor emeritus
at the University of Washington and the Paul G Allen
School of Computer Science and Engineering.
And Brianna, there might be other people
that went to that--
BRIANNA BLASER: The only ones who
got the calendar invite were the speakers,
so that shouldn't have gotten to anybody else.
RICHARD LADNER: Oh, OK.
So we have like three segments to our presentation today.
I'll talk about just what is accessibility
from a certain kind of perspective.
And then Lauren will talk about her project, Blocks4All.
And Stefik will talk about his project, Quorum.
And these are all tools for accessible education.
So what does accessibility mean?
And I would like to come up with an example.
And this is a student, Raina, who I met back in 2013 when
she was on our program of mine.
It was, I think, about an eight week program
that met on Saturday mornings.
And I think this particular project was done in--
they built individual projects in Scratch.
And so that was kind of exciting.
Anyway, Raina has multiple disabilities.
She's deaf.
She's blind-- or at least she has very low vision.
And she also has cerebral palsy.
But I got to know her pretty well.
And she is completely intellectually capable
of doing computer science.
She was kind of a brilliant student
in her own right and a writer.
So I can recall when we had this Saturday program.
The parent of Raina came to me and asked me,
can Raina participate?
And of course, I didn't know her at the time.
And he was very reluctant.
He said, well, are you going to make it accessible?
And things like that.
And I said, we will do that.
We will make it as accessible as possible.
And then once I met Raina, we had to do a lot of stuff
to make sure that she could participate.
For example, we have there a sign language interpreter
on the same screen that she could
see well enough to see signs.
And her own sign language was somewhat distorted.
And so having somebody that was sort of a sign language expert
to read her signs was great.
She didn't talk at all.
And there's another picture of her
on the left in her wheelchair.
And by the way, she always had a smile on her face.
And she had fun.
And she really enjoyed herself.
And here's some of the technology that she used.
For one thing, because she had this low vision,
we had a large monitor that she could
look at her Scratch screen.
She was able to use a joystick as opposed
to a mouse, which is on the upper left there.
She also used a communication board because she didn't speak.
And that's called augmentative and alternative communication,
or AAC, on the upper right.
And then she actually was able to type,
but she needed a keyguard.
So when she hit one key, her finger would stick in there,
so you can see that sort of in three dimensions.
The Q there, if she wanted to hit a Q,
she wouldn't slide over to the W, or the A, or the S.
And so this is some of the technology that we use,
plus the human sign language interpreter.
So that's making it accessible.
So when we think about accessibility
for computer science, well, computer science
is about computers.
And so we want to have access to computers.
And fortunately, there is a lot of access technology
for computers.
There are screen readers.
A screen reader would use keyboard input or no mouse.
Or you could use touch on a touch screen.
The output would be speech or Braille.
So for example, the picture in the upper right
is a person getting an output both in speech and in Braille.
And the Braille is very helpful for programming
because you get all the details, while the speech might
be leaving things out, like curly braces
and strange symbols that occur in programs.
There's also switch access.
So it's a binary switch with scanning.
And so the elements on the screen
are scanned one after another at a rate that
is suitable for the person who has the switch.
And there's a picture of somebody using a head switch
on the bottom right there.
And so when the thing appears to be
scanned that they want to select, they click the switch.
And that gets selected.
So you can see, for example, typing
text would be very laborious because the scanning would,
for example, first go over each row of the keyboard.
You pick the row, and then it goes down
that row to pick the individual key.
And so typing it can be very, very slow.
But there are people that require switch access.
They don't have speech.
They don't have the other facilities.
That's the only thing they can do.
And I'm not saying there are lots of people like this,
but there are certainly people who
have high intellectual capacity that could do computer
science that use switches.
And there's also speech input, which is
getting more and more popular.
And I'll talk a little bit more about speech input
for Apple products in a moment.
And then, of course, special devices--
you saw the joystick that Raina used.
There's mouse keys, head mouse.
There's tons of other things--
a mouth mouse, and so on.
So these are the kinds of things that
allow for access to computers.
And then within the computer world--
Mac and iOS, Android and Windows--
there are a tremendous number of accessibility tools
that are available.
VoiceOver for Mac and OS is screen reader.
It's a very good, one high quality,
and developed by Apple.
They also allow for switch control and voice control.
So those are on Apple products.
For Android, there's a TalkBack screen reader.
And there may be some other tools
that I don't know about because I haven't investigated it
in the last few months.
For Windows, there's a narrator screen reader, a native one.
Plus, window supports a lot of third party screen readers,
including JAWS, which is the famous one, and NVDA, which
is more recent, but NVDA is open source and free.
So all of these things are available to these students who
are blind or have physical disabilities that
have difficulty accessing a computer the way
that many of us do using a keyboard and a mouse or touch
and things like that.
Now, the problem, of course, is that--
or the key observation is that computer applications
and websites are not automatically accessible.
They have to be designed and programmed to be so.
So things like Scratch and Scratch Junior,
they're not accessible because they weren't
designed to be accessible.
Screen readers don't work with them .
Switch access doesn't work with them, and so on.
So this is true for probably the vast majority of products
that are out there for kids.
And this has been a great concern of us
here at AccessCSforAll.
And we have been working on trying to make this better.
And we believe that the Scratch Foundation and Code.org
are finally on board.
And I think they're going to do the same thing,
make their products accessible.
So I'm going to switch now to Blocks4All.
And this will be Lauren.
So Lauren, can you take over?
And I'll advance the slides.
LAURA MILNE: Right, awesome.
Thanks.
Yeah, so hi, I'm Lauren Milne.
I'm assistant professor at Macalester College.
And I actually was one of Richard's students.
And I started to work on this project, Blocks4All,
which is an introductory programming
environment designed mainly for younger children.
And it's designed to work really well with VoiceOver,
so designed for children with visual impairments.
Now, we're branching out to switch control.
So first of all, I just wanted to introduce
block-based programming environments.
If you're not familiar with it, this
is a Blockly environment, sample environment.
And so block-based programming environments
are a way that you can introduce people to programming.
And they basically are like puzzle piece-like units of code
that you can drag and drop from the workspace,
which is on the left side of the screen, onto--
or sorry, the toolbox, which is on the left side of the screen,
onto the workspace, which is sort
of the middle of the screen.
And this is a simple loop that prints "Hello,
world" to the screen three times.
So in the middle is that loop in blocks.
And on the right is that same program in JavaScript.
And these are great.
It's hard to make syntax errors if you're using blocks of code
because you don't have typos.
You don't have to worry about curly braces, or parentheses,
or anything.
But they are not super accessible for screen readers
in particular, but also other forms of access
that Richard talked about.
So things that we noticed-- one, the drag and drop gesture
is very hard to perform with a screen reader.
And then also, initially when these were created,
you couldn't even access the blocks.
They weren't even read out loud correctly by the screen reader.
So Scratch is probably what most people
are familiar with for block-based languages.
And it's built on top of Blockly, which is Google's--
an open source library from Google.
And they are actually actively working
on making it more accessible.
I think the newest version of Blockly
is now keyboard accessible, so you can actually
move to the different blocks using the keyboard.
And they're working on integrating the screen reader
in.
But what we did to address this in accessibility
is we built our own programming environment designed
specifically for kids.
So you can go ahead and, yeah, press the next slide.
So I'll talk a little bit before the demo--
well, the video.
So this is a video showing the app.
And we designed it for the iPad, which is nice.
It works really well with the built-in screen reader.
And it controls-- I don't know if people
can see me-- but this very cute and mostly indestructible
robot.
I had it tested with five-year-olds
we tried throwing it around.
But yeah, so this video-- and I apologize.
The video, the VoiceOver, went to the screen reader.
Well, it sort of has this weird echo.
But hopefully, it's intelligible.
And the words are also at the bottom of the screen
to demo Blocks4All.
Richard, do you--
I don't know where the sound would be for this.
Do you have to unmute yourself ?
Or I can just talk.
Well, we need the VoiceOver.
BRIANNA BLASER: Richard, you might
need to reshare your screen and make sure to share the audio.
Oh, Richard, you're talking, but you are muted.
Apologies.
LAURA MILNE: And Richard, I can also just try sharing my iPad
and do the demo.
RICHARD LADNER: OK, I should be able to start it again.
And you should be able to get it now, the sound.
[VIDEO PLAYBACK]
- Hello, I'm going to do a quick introduction
to the Blocks4All programming environment with the screen
reader VoiceOver turned on.
So this is the opening screen in the Blocks4All environment.
There are a number of options here.
You can get started with the robot.
You can read through some instructions.
You can add a robot.
And then there's this toggle option.
You can either show the icons or show
text on the blocks in the Blocks4All environment.
And I'll choose Show icons.
And what I'm actually going to do first
is turn on the screen reader VoiceOver.
So when I turn it on, the screen will start talking to me.
- VoiceOver on, Blocks4All, Get Started Button, landscape,
Home button to the left.
- And if you have difficulty hearing what the VoiceOver is
saying, it also, on very small text,
has what is being said aloud written
on the bottom of the screen.
So now that we have VoiceOver on, when VoiceOver is on,
I can move my finger around on the screen.
And it will read aloud what is under my finger.
And then if I want to actually select something,
I use double tap.
So you can either move your finger around.
And I'll do that right now, just go through the options
and you'll hear them being read aloud.
- Instructions button, Add Robots, Show Icons, Show Text,
unselected button.
Double tap to select.
- And you can see it gives you a little bit more information
about how to actually select things.
So I'm going to go back up and select Get Started.
- Get Started button.
- And then I'll double tap to actually select it.
- Toolbox menu, heading.
- And that takes me to the main screen in Blocks4All.
And on the screen, on the left side,
you can see there's the toolbox, which
has a bunch of different categories with blocks in it.
So I can select blocks from the toolbox.
And I'll move them into the workspace,
which is the white area in the middle of the screen.
And the blocks will be moved to the bottom of the screen.
So I'm going to go ahead and collect--
select something from the toolbox.
So I'm actually going to swipe through.
So instead, of touching-- you can't see where
my finger is on the screen.
But instead of just touching, a lot of times,
people just use swipes.
And it'll swipe through everything
on this screen in order.
And that way, even if you don't know where something
is on the screen, if you're using a screen reader
and you're blind, you can still hear what all the options are.
So I'll swipe through right now.
And I'll go down to the Drive category in the toolbox.
- Sounds category.
Double tap to explore blocks in this category.
- And so once again, it has a hint
about how to get into those different categories.
- Control category.
Double tap to explore blocks in the category.
Drive category.
Double tap to explore blocks in this category.
- And I'm going to go ahead and double tap on Drive.
- Drive forward, in toolbox.
Double tap to place block in workspace.
- So that takes me into the drive toolbox, the Drive
category on the toolbox.
And you can see there's a number of different drive options
here.
We give a little bit of extra contextual
information about where you are.
So it says drive forward in the toolbox,
so you know you have selected a block that's
still in the toolbox.
It's not in your workspace.
And I'm going to go down.
And I'm going to actually select the Drive Backwards block.
- Drive backward in toolbox.
Double tap to place block in workspace.
- So I'll go ahead and double tap.
- Drive backward selected.
Select location in workspace to place it.
- And it's now selected on the screen.
So this was one of the first things
that we changed, compared to normal block space
environments.
In those environments, you typically drag and drop.
That's challenging to perform with a screen reader on.
There is a way--
Apple has provided a way that you can make the drag and drop
gesture accessible with a screen reader,
but it involves going deep into a secret menu setting
and it's challenging.
So we ended up switching drag and drop to select.
You select the block, and then you
select where to put it in the workspace.
So I've selected the block.
I'm now going to find where to put it in the workspace.
So once again, I'll swipe through my options.
- Main workspace.
Place Drive Backward at beginning.
- And now, the main workspace is selected.
There's nothing in the workspace,
so my only option is to place at the beginning.
So I'll double tap to place it there.
- Toolbox menu, Back button.
- So now, my Drive Backwards block
has been placed in the main workspace.
And you can see there's a block above the Drive Backwards
block.
It says 30 centimeters, normal.
And that's the modifier block.
So a lot of these blocks have modifiers.
You can change settings on that.
So I'm going to go ahead and actually select that modifier
block.
And I'll change things.
So instead of going 30 centimeters,
maybe I'll go 50 centimeters.
- 30 centimeters, normal button.
Double tap to set distance and speed.
- I'm going to go ahead and double tap.
- Back button.
- It takes me into this modifier screen.
And here, I can go ahead and change the distance and speed.
- Distance, select the distance.
Current distance is 30 centimeters.
Distance slider, adjustable.
30 centimeters, adjustable.
10 centimeters to 100 meters in increments of 10.
Swipe up or down with one finger to adjust the value.
- So you can do sliders with VoiceOver on.
You can swipe up or down.
So I'll swipe up.
- 40 centimeters, 50 centimeters.
- We'll go 50 centimeters.
Then I'll keep--
- Speed, slower, normal, faster.
Button add, faster, current speed, speed, fast, add.
- And I'll go ahead and go 50 centimeters fast.
Then I'll go back up to the Back button.
- Back button.
Back.
Toolbox menu, heading.
- And you can see the Drive Backwards
is now 50 centimeters and fast.
So that's how you add blocks to the workspace, how
you can modify them.
I'll go ahead and I'll add another block.
Maybe I'll add a Sounds block here.
- Sounds category.
Double tap to explore blocks in this category.
Animal noise in toolbox.
Double tap to place block in workspace.
- And I'll go ahead and add an animal noise here.
- Animal noise selected.
Select the location in the workspace to place it.
- And now, there's a couple more options.
I can either place it before or after the Drive Forward block.
- Main workspace, Place animal noise
at the beginning before Driving Backward 50 centimeters
at fast speed.
Workspace 1 of 1.
Double tap to put the animal noise here.
- I'll go ahead and add it there.
You can see there's a lot of information about what you're
actually doing it for do that.
- Animal noise placed.
Animal noise in toolbox.
Double tap to place block in workspace.
- Sometimes, the screen reader gets interrupted
what it's currently reading when it moves back
and the screen changes.
But I can go down to my workspace at any time.
It's on the bottom of the screen.
It's another modification we made.
We moved all of the blocks to the bottom of the screen,
so they're easy to locate when they're in their workspace.
Even if you have someone who is blind using it,
you can find the blocks on the bottom of the screen.
So I'll do that.
I'll start on the bottom of the screen.
- Animal noise, cat noise, workspace block 1 of 2.
Double tap to move block.
- Block 1 of 2.
- Drive backward 50 centimeters, fast speed,
workspace block 2 of 2.
Double tap to move block.
- All right.
So now, I've got these two blocks in my workspace.
I'm going to go back.
And I'm actually going to put a slightly more complicated block
in.
I'm going to put a repeat block in, so I can do this--
make this animal noise and drive backwards multiple times.
So I'll go back to my toolbox menu.
- Animal noise, toolbox menu, back button.
- To the back button.
- Toolbox menu, sounds category.
Double tap to explore this category.
- I'm going to go ahead to go to the Control category.
- Double tap to--
repeat in toolbox.
Double tap to place block in workspace.
- We can do a couple of different fun control blocks.
You can repeat.
The next one is repeat forever.
We have if statements, so there are certain conditions you
can wait for and only can do the code inside
based on those conditions.
[INAUDIBLE] So I'm going to go ahead and select this repeat
block.
- Repeat selected.
Select location in workspace to place it.
- And I'll just place it at the end.
- Place repeat at end.
- Back to the workspace.
- Toolbox menu, back button.
- So the repeat block actually has two blocks.
You can see there's the repeat and there's also an end repeat.
And if you read through in the workspace--
- Repeat 2 times.
Workspace block 3 or 4.
End repeat, workspace block 4 of 4.
- It shows up as two blocks.
- Double tap to move block.
- And if you want something to actually
be repeated multiple times, you can move it inside of the loop.
So I'll show you now how to move a block.
So I'm going to go ahead and select this animal noise block.
- Animal noise block, workspace block 1 to 4.
Double tap to move block.
- I'll move it.
- Animal noise selected.
Select location on workspace to place it.
- In the workspace, I'll put it before the end repeat block,
so it's inside of the repeat loop.
- Place animal noise before end repeat, workspace block 3 of 3.
Double tap to move animal noise block here.
Repeat, in toolbox.
Double tap to place block in workspace.
- And I'm going to go ahead and move the drive backwards block.
- Drive backward 50 centimeters, fast speed,
workspace block 1 of 4.
Double tap to move block.
Place drive backward in repeat.
Workspace block-- drive backwards, repeat.
In toolbox, double tap to place block in workspace.
- So that's a basic introduction to Blocks4All--
[END PLAYBACK]
BRIANNA BLASER: You got cut off at the end,
but that is the introduction to Blocks4All.
Any question?
[VIDEO PLAYBACK]
- I'm going to do a quick introduction
to the Blocks4All--
[END PLAYBACK]
LAURA MILNE: It's still going, repeating forever.
And also, possibly an introduction to VoiceOver,
if you're less familiar with it.
But yeah, so we basically built Blocks4All, designed
it to work really well with VoiceOver,
worked with some kids with visual impairments and TVI
to design it.
Now, we created an hour of code activity
last year, which really bumped up the number of downloads
from the App Store.
It's available on the App Store.
And I put in a QR code at the end,
if anyone wants to check it out.
It's freely available from the App Store,
but the robot is expensive.
But you can try it with all the robot.
And we have 4,500 or so downloads from the App Store.
The analytics was a little challenging to read for me
last night.
I've also been in contact with especially TVIs in Minnesota,
Oklahoma, Maryland, and Iowa who've used it successfully.
Richard, you can go to the next slide.
And currently, we're working on a project--
well, so we added in functions and variables,
so you can now add functions.
And there are some very basic variables
that you can set in Blocks4All.
And then we're also working on really improving switch control
accessibility.
So we're hoping to do a user study with people
who use switch control to just make sure Blocks4All really
works well for them.
Luckily, a lot of designing well for switch control
comes as a bonus if you design well for VoiceOver,
for the screen reader.
And yeah, you can move us to the next screen, Richard.
As I said, we did create an hour of code activity last year.
And there's two levels.
There's the K through 5 and a little bit more
advanced one with a function for up to grade 8.
And these are on our code website.
And then I also, at the very end,
included a QR code with the link,
if you want to check it out.
Oh, I think I put the website.
It's kind of just floating in the middle of that,
where it is.
But I doubt anyone's going to able to copy down the URL here.
Yeah, so I think the next screen has the QR codes on it,
if people want to look.
Yeah, so as I said, the Blocks4All app,
so it's an iOS app.
And then the hour of code activity
just takes you to our website.
RICHARD LADNER: We'll leave that up for a second, Lauren.
I think other--
Brianna, should we ask a few questions now
while she just finished?
BRIANNA BLASER: Yeah, I think that's a great idea.
Folks can ask questions in the chat or raise your hand.
Sorry, Richard, to cut you off there.
RICHARD LADNER: I'm going to ask a question.
Yeah, so Lauren, why did you pick the iPad in particular?
What was it about that that--
LAURA MILNE: Oh yeah, that's a really good question.
And so I will say that when I talked
about Blockly at the beginning, that's
Google's open source library.
And that is web-based.
So I haven't experienced trying to work on that
to make that more accessible.
The nice thing about the iPad is that, number one, VoiceOver,
the built-in screen reader for the iPad is really good.
So native iOS apps just have a lot more support for VoiceOver.
And they're a lot more accessible out of the box,
basically.
And so you just have a lot more--
it's easier to make them accessible
than web-based stuff.
So the one issue with web-based in general
is that you have the interaction between-- as Richard said,
there are a number of different screen readers.
There's a number of different browsers.
So even meeting like the best practices
for making things accessible, the browser screen reader
combination means that it may still not be accessible.
RICHARD LADNER: And I think there's one other thing too.
The iPad is kind of a standard for blind children
in the United States.
And so they're already familiar with it.
And they already know how to use VoiceOver.
And so getting into programming is fairly seamless
using Blocks4All.
LAURA MILNE: Yeah, so I'll say, yeah, we
did another project, where we talked
with TVIs at the Washington State School for the Blind.
And I mean, I think a lot of kids, especially pre-K kids,
they're learning touch screens really early,
especially since Apple devices come with a screen reader.
They'll start borrowing their parent's phone.
Or they'll start getting a tablet.
And a lot of schools now provide tablets for elementary school
age students.
RICHARD LADNER: Are there any questions, Brianna,
from the chat?
Or anybody can unmute and ask.
BRIANNA BLASER: Yeah, I don't see any questions at this time,
but folks can continue to put them in the chat as we go on.
RICHARD LADNER: Yeah, why don't we switch now
to Quorum and Andreas Stefik.
Oh, do you want to say something on this slide, Stefik?
ANDREAS STEFIK: No, it's OK.
We can go to the next one.
RICHARD LADNER: OK.
ANDREAS STEFIK: So hi, Andy Stefik.
I'm a professor of computer science in Vegas.
Before I say anything at all, just
thank you to Lauren for doing your part of this.
I find your work actually quite inspiring
because I think that very young children don't always
get a lot of attention in the eye space, including for me,
even though I care about the eye quite a lot.
So your designs are fascinating.
So I just really appreciate you going over them.
So I'm going to talk a little bit about Quorum today.
If you've never heard of it, it's programming language.
It's a language that we've been working on
for I'm not sure how long now.
I think I'm getting older.
Maybe 10 to 15 years, I'm not sure exactly.
Anyway, but it is a programming language
that has a lot of properties.
So it's online and offline, it has wide variety
of libraries and tools available for all sorts of stuff
nowadays--
data science, robotics, you name it.
We rather obsessively look at evidence,
but evidence doesn't mean magic.
It doesn't mean all possible things are easy to use.
It just means that we slowly use data to try to make it better
over time.
We have support for mobile devices, tons
of documentation, et cetera.
But what I want to mention the most
is if you look on the right on this slide, what you'll notice
is that there's an environment.
It says Quorum Studio.
And then inside of that environment,
there's all sorts of graphical things.
And intuitively, you might think, oh cool,
he made something graphical that doesn't
work for people with disabilities at all
if you're blind especially.
So part of the challenge and what's
actually taken so long at Quorum to build
is we've developed this thing that we
call accessible graphics.
And I'll show you an example later.
What is accessible graphics?
Accessible graphics means that you have magic tricks
to trick the graphics system so that they can be accessible
even if you can't see the screen.
So if you flip to the next slide, Richard,
please, what you'll notice on here is a series of charts.
And I'm just going to use charts as an example,
but it's not exclusive to charts.
Technically speaking, the accessible graphics
that we've been working on actually
applies to any arbitrary graphics--
3D, 2D, shaders, whatever you want.
And inside of Quorum, we've spent several years
developing little tricks to trick the hardware systems so
that you can also communicate with screen readers
at the same time.
Now, how does that work?
Well, it's complicated because it turns out
that trying to make something like OpenGL,
or WebGL, or stuff on various platforms
talk to screen readers isn't easy.
And there's all sorts of reasons for that I
won't go into related to how they
send stuff around in memory and things
that you wouldn't think would matter,
but that should matter a whole heck of a lot.
So let's take a look at some of these.
If you look on the left, you'll see a chart.
And in that chart, on the very top,
it says bar chart, 1 column, 2 factors.
So inside of Quorum, yes, it generates graphics.
But behind the scenes, depending upon how you're rendering it,
it actually sends structure to the operating system.
So notably, what it does is it breaks that chart down
into a series of little pieces.
And it says, hey, operating system,
here's how you interact with those pieces.
And then the screen reader can choose to optionally understand
that structure or not.
And when I say choose, I'm not joking around.
This goes back to what Lauren said,
which is that different browsers and screen readers can
make their own decisions.
That's completely true.
Operating systems can as well.
For example, I could tell the operating system
that that's a chart, but operating systems
don't understand charts.
There is no such thing as an accessible chart.
It doesn't exist.
But I can tell it that I can change the focus.
And I can tell it that there are bars
and that I can go up and down on them and things like that.
The web is actually different.
So if I take that exact same chart in Quorum
and I press a different button to make it magically
go somewhere else, then it'll generate it either
in, a, OpenGL on the web in WebGL,
or it will generate it as an SVG,
as scalable vector graphics.
Basically, it tricks the browsers in different ways
than it tricks other operating systems, like Windows and stuff
like that.
So just to give you a rough example--
I won't show you a direct demo of the charts,
but you can do things navigate between the charts.
You can go to the legends.
You can ask get information.
You can go to the titles.
You can navigate to things in different kinds of structures,
get the factors, get independent variables.
And there's a lot of interesting questions related to how you
manage things like graphics.
For example, if you had a box plot,
should it present the p value for the box plot between
groups/ I have no idea.
But the thing that's interesting about graphics
is that there's so little exploration in the literature,
you can kind of experiment a bit and play around
with what might be interesting for people with disabilities.
If you look at the other two charts--
I won't go over them in great detail.
But if you look at the one on the top in the middle,
that's what's called a multi-chart.
And those are accessible, too.
These are not alternate descriptions.
They are auto-generated structure
that Quorum knows how to manage.
So behind the scenes, if a person that is blind
is using this chart, then they can navigate
between the multi-chart.
They can navigate to the legend and get information
about p values and stuff.
They can then get information about the regression line.
It's massively more information that you
can get from the system compared to simply like
an alternate description, essentially.
On the bottom is something similar.
That's box plots.
And so there's lots of interesting questions.
Should you be able to get to the median line?
Should you be able to get to the whiskers?
And so to a certain extent, part of the challenge of the Quorum
project is trying to figure out what information you even
want to get, if you're a person that's blind.
And that often requires us to basically go talk to people
and ask them, what do you want here?
Stuff like that.
Oh, and I forgot to mention-- you
don't need to go back to it, Richard.
But on the previous slide, there was actually
a development environment.
And in that development environment,
that is also OpenGL rendered through the same Quorum
systems built in the same way.
It is not like anything but Quorum under the hood
because we need to be able to present all sorts of stuff.
On the right here is a picture of a phone.
The phone is running out of battery.
And there is a ball and a series of blocks on the screen.
Those blocks of a screen involve 3D graphics, not 2D.
But it turns out, once you have solved
accessible graphics, that actually doesn't
matter for accessibility.
Whether it's a 2D item or a 3D item,
it actually doesn't make any difference
because the operating system gets the structure.
And this works both on the web and it works
on Windows for the moment.
We're working on Mac and TalkBack support as well.
But the point is whether it's 2D 3D doesn't really matter.
And so Quorum actually has built-in
a series of game engine libraries
and also game editors, scene editors,
so that you could even do 3D game creation.
And you can use a screen reader to do it.
So you can actually go in and use Unity 3D-style game editors
and they're accessible.
They're not perfect, but they're accessible.
Next slide, please.
Besides all sorts of fun stuff with graphics,
my obsession is to try to get anything
as hyper graphical as possible, as accessible as possible.
It has been for a while.
By the way, I should say, that actually
came because we did some interviews with young children.
And there was a child--
I don't remember the person's name.
And I probably couldn't say it anyway.
But I asked them what they wanted to do.
And he said, I want to make computer games in 3D
with my friends.
And we said, but there's no technology to do that.
And he said, that's not my problem.
So that kind of inspired me over the years.
But in any case, besides all sorts
of utilities, and libraries, and stuff,
Quorum is used quite a bit at schools nowadays.
I don't know the exact number.
It's kind of hard to calculate, but somewhere
in the ballpark of 100,000 people.
And because of that, we have documentation
for a gazillion different things because people ask us for it,
and then we go and build it.
This is programming basics, graphics stuff, physics engine
stuff, audio libraries.
There's a digital signal processing library in there.
There's libraries for tactile and gesture support
on various mobile platforms.
We've got LEGO robotics support, networking, online stuff.
There's a database library, which we just
finished a study on not that long ago,
user interface design.
We have new libraries for data science.
Those are more coming online because of some recent NSF
work and include, so far as I can tell,
pretty much anything most scholars would
need all the way up through factor analysis,
but all the way as simple as calculate me the average
please.
Charts, all of which are accessible--
we have high school courses and computer science principles,
which we developed in collaboration
with Code.org with their blessing,
and a whole ton of other stuff that I'm not
mentioning just because I don't want to bore people.
If you look on the right, there is a picture of one
of the tracks.
It's just one of them that we did last summer with teachers.
We did an accessible data science track just for fun
over at our annual conference.
And this was teaching people to make pie charts, and bar
charts, and stack charts, and all sorts
of stuff, which was a lot of fun,
in part because this particular venue has
a lot of people that can't see.
And it was sort of fascinating watching their brains light on
once they realized that this technology was actually
possible.
I remember very vividly there was someone
from APH, the American Printing House for the Blind,
that said she would immediately be going home and using it
for her job because they couldn't do that before.
And that's always kind of nice, that a lot
of these graphical things, they seem esoteric.
But in fact, a lot of people need
to be able to interact with all the stuff everybody else does.
And this sort of provides them an opportunity to do that.
Next slide, please.
So the last thing that we're working on now,
and then I promise I'll leave you alone,
is we've been developing an accessible block system.
And this is to try--
partially inspired by Lauren, but others as well--
to try to push down, make it a little bit less difficult
for especially younger individuals.
This is not the final look and feel.
But you'll notice that if you understand visually what
tools like Scratch, and Blockly, and others look like,
it does look somewhat similar to those environments.
As of right now, we're planning a version
on the web, which is going to use a shadow DOM architecture,
if you're familiar with that, in WebGL.
Offline, it's using hardware accelerated graphics.
And everything's rendered through shaders,
so there's no weird SVG rendering.
And it turns out that actually matters because of the way
accessible graphics actually works.
So if you want accessible blocks to be fully accessible,
you need to control the hardware to a certain extent in order
for it to actually play nice with screen readers.
We have learned the hard way, partially this summer.
So in any case, if you look on the screen,
you'll notice that one other challenge that we have when
developing accessible blocks is that it's not just a look.
Blocks, in order to be accessible,
have to work differently on different platforms.
So for example, if you're in a desktop programming
environment, which many kids in high school and younger
actually don't use-- they often use the web because of school
systems and other reasons--
you need to carefully control keyboard controls and stuff
like that.
However, like Lauren said, it's very common at the very young
ages to use tablets.
And there's a lot of design considerations
to think about there.
Do you use vibration or not?
If it's on an iPad, you also don't get vibration.
But on an Android tablet, you do.
But Android tablets aren't use that much,
if you're specifically blind.
On the other hand, they're used much more commonly
outside the United States, which is something the Scratch Junior
team has told me very, very clearly,
that it's kind of US-centric.
So there's a lot of design considerations.
And in addition, Quorum is also general purpose.
So that means if we're going to be having work
for young children, we're going to have to scaffold similarly
to what Code.org does.
But we can't just ignore our users that
use it at a more professional level and say, oh sorry,
you don't get any of the benefits.
So that makes all sorts of interesting questions.
Do we include Git support in the design
of the blocks, which we have and some people use
in the community, including in high school?
Do we have line numbers, like you
would in a normal programming environment?
Should you have little buttons that
tell you what the affordances are,
which usually aren't in block environments?
There's some evidence that some of the locking mechanisms
are a little bit hard to use.
A nice little paper by Stephanie Ludi
came out recently talking about that.
And Neil Brown actually gave a wonderful talk
kind of summarizing some of those details recently for us.
Thank you, Neil.
In any case, this is not the final design of our block
system, but we are hoping to have--
we're starting dogfooding our block system next month,
I think, so that we're going to be using it
for all of our program activities,
including creating the block system itself,
which is going to be quite an adventure, I imagine.
And then we're hoping to release this
to schools next summer in both an online and offline form,
assuming nothing goes wrong, which you never know.
So I think that's it for me.
So that's the Quorum project, in general.
And I'll hand it back to Richard.
RICHARD LADNER: Yeah, thank you, Stefik.
That was fantastic.
I don't have any questions right now,
but people hopefully are queuing up questions in the chat.
So we'll just finish off this main part with what you can do.
And promote the adoption of accessible curriculum tools
and pedagogy, I think, is the main thing.
And here, there are five things that are listed.
One is Quorum, we just heard about.
Blocks4All we just heard about.
There's Swift Playgrounds, which is straight out of Apple.
And that's accessible and built to be accessible,
or borne accessible, as we say.
There's also Code Jumper, which is in the bottom left there,
which is sort of a physical manifestation of programming,
which is kind of neat.
And that would be particularly good for perhaps definitely
blind kids, but maybe kids with autism or other disabilities.
And then finally, use the universal design
for learning principles and practices in your teaching.
And we're not going to cover that in our tools today.
But Maya Israel from University of Florida
has really spoken a lot about this.
And so that's something to think about.
And in the bottom right there are the three main components
of UDL practices-- engagement, representation, action,
and expression.
That's kind of abstract.
But learn more about UDL and how you can incorporate that
into your lessons.
We have resources here.
And these slides will be available, Brianna?
They'll be on our website?
BRIANNA BLASER: Yeah, and I can--
I think if you share the slides with me,
I'll drop the link for them in the chat and a follow-up email.
RICHARD LADNER: Right.
And so these are all hidden links to some of the things.
A couple of things we didn't talk about
was the NCCSE Accessibility web page, which is excellent--
the National Center for Accessible Education Materials
really focuses on blind and visually impaired,
but there's really a lot of neat stuff there.
And then the Creative Technology Research Lab
at the University of Florida has a lot of stuff
about UDL practices in computer science,
so targeted at computer science.
So I think we're up to questions and discussion.
BRIANNA BLASER: Well, I can ask a question.
We talked a lot in the context of Blocks4All and Quorum
about students who are blind or have low vision.
But what should we be thinking and talking about with regards
to other students with disabilities?
ANDREAS STEFIK: Well, there's a couple of things.
And others can jump in, too.
But one thing to immediately remember
is that a lot of access-- there's no operating system
command for blindness, right?
It doesn't work that way.
It's for accessibility.
And so oftentimes, if you have an application
and you're connecting to all the right accessibility pieces,
you actually get other stuff for free
that you probably don't realize, like tools for people
that are neurodiverse.
If you have closed captioning, those are used by the blind.
And there's other things that happen, vice versa.
So oftentimes, accessibility really
is universal design in disguise.
And so Laura and I have been talking about that because it's
a particularly hard challenge.
But at the same time, they are connected.
The one exception-- and I think that it's a hard one.
This is one reason why, at least in our case,
we care about accessible graphics is because I think
the prevailing hunch is that people with neurodiversity
might actually benefit from such tech, like things like blocks
and having visual representations.
And certainly, that's true with people that are deaf.
And so I don't want to just ignore graphical components
and pretend that they don't exist.
It does feel like it's a hard balance,
but the tech is pretty difficult. I don't know.
Lauren, do you disagree at all?
LAURA MILNE: No, I think I agree a lot.
I think, yeah, a lot of it's just universal design.
I mean, so yeah, we focus a lot on screen readers.
And in general, accessibility research
focuses a lot on screen readers.
But a lot of the other stuff kind of falls out, right?
Especially when you're thinking about input and output devices
particularly, if you can make your programs work for screen
readers, a lot of times, it works for various other input
and output devices.
But yeah, there's a lot less research looking
at other types of disabilities.
BRIANNA BLASER: I see Neil's hand.
NEIL BROWN: Yeah, so I'm interested
about this accessible 3D graphics
that you kind of hinted at on your slide, Stefik.
It seems to be that there's a lot of different levels
of abstraction you could represent that as
and present it to the user.
Could you say a little bit about what you went with
and the considerations in deciding that?
ANDREAS STEFIK: Sure.
So let me say first I'm unaware of another 3D graphics
environment that is blind accessible.
So if I'm fully honest, a lot of what we did was total guess,
and then show it to people, and tweak.
So I mean, that's just honest.
But I'll see if I can't make a quick--
show you, so you don't have to guess.
Let me see if I can't pull up something.
Oops, let me give you a rough idea.
I don't have a demo ready, so there's that.
But let me see if I can't pull up a scene
and you can at least see what--
and I'll describe.
RICHARD LADNER: One comment, Stefik,
is that I don't think it was a complete guess because you
have many years of experience.
And I think that counts.
It doesn't make it just a guess.
ANDREAS STEFIK: That's totally fair.
So I'm not going to flip on the screen
because I'm on my Mac at the moment and we don't have--
the screen reader support for graphics,
it works on Windows, but not on Mac.
It's a flaw that we're working on that's going to take time.
You'll notice it's on the screen.
You've got a 3D box.
And inside of that 3D box, on the left,
you've got this palette of items.
So here's a castle battlement.
And inside of it, when you place it on the screen,
you can use the keys to do that.
And you can go in and then make items, and move them around,
and stuff like that.
And in terms of what the screen reader does,
this is actually really tricky because--
briefest demo in the universe, sorry Neil.
I should do something more sophisticated.
So the tricky part is that when we actually show this
to people, there's a number of challenges that are very clear.
One is, how do you manage spatial relationships is
already difficult for obvious reasons.
And when we actually show it to people,
oftentimes, we provide things like the xyz coordinates.
And that does tell you the spatial relationships,
but you have to keep it in your head.
And that's tough.
So I suspect that there's probably ways to get at that.
And we've tried a couple.
And I don't have evidence on whether they're good or bad,
but I'll give you some examples.
One is if you are creating linked structures,
we have certain ways to gather things that are in the view
or in the camera, so it can tell you information like that.
In other words, what am I pointing at/ what am I looking
at?
It's kind of an aspect of the where
am I problem with blindness.
Another one that I think we largely failed at
was whenever you have nestings, it gets really tough.
So I remember, I was in a meeting
with a blind person looking at our tool.
And they were using JAWS, I think.
And they were trying to make a house with a table in it
and a candle on the table.
But the problem is the xyz coordinates
don't give you that information.
They tell you where they are, but they
don't tell you the height of the model and stuff like that.
So they would have this situation
where they'd have a candle.
It'd be slightly below the table.
And I'm like, visually, you could immediately tell.
It wouldn't be a problem.
But for this person, it wasn't because they couldn't
tell the relationship between the model size
and the candle in order to know that it's
on the table in the right spot.
So I think that our system right now gives you
a pretty good idea of where you are and what you're looking at.
But I don't think it does a very good job of telling you
the relationships between things when they're not
in exact increments, like this is a unit of 1
and this candle is a unit of 1.
So there's a lot of challenges with 3D
that I think are really neat and fascinating.
And so far, we've really only focused
on the spatial relationships.
And there's 1,000 studies that can be done
to try to make that better.
I don't know if that's a good answer
or not, but that's sort of what we've tried so far.
Seriously, I wrote this on a plane
because I thought it would be fun,
and then the team kind of went with it a little bit.
But we were just kind of screwing around.
We just were curious if we could do it.
And so it was kind of fun.
So anyway.
RICHARD LADNER: This is Richard.
One thing I found both you and Lauren, Stefik,
that what you've done is based on research,
but you've made it public.
So Blocks4All can be downloaded and be used.
And of course, it's being improved every summer
when you have time, Lauren.
And there's continuous improvement
on Quorum, and with all these new things with data science,
and so on.
So what motivates both of you to bring things out
of the research realm and into the public?
Maybe start with Lauren.
LAURA MILNE: Yeah.
And for me, it's the people, right?
So when I was developing Blocks4All
and even when I was using it, there's
students who are just so excited that they could actually do
something and program a robot.
And I was not initially planning to--
I was kind of hesitant to put it on the App Store
because I was like, oh, I can make it better.
But I think just getting out there,
because there is so little out there for them to do,
that was really important.
And yeah, it just came out and people were just
pumped to be able to do something for these kids.
RICHARD LADNER: Stefik?
ANDREAS STEFIK: My story is the same, to be frank.
I can't come up with a better answer.
Although, I will say very specifically
why we decided to do that.
And that's because of a woman named
Sherry Hahn at the Washington State School for the Blind.
She's since retired.
But I remember she was talking about research
in disabilities work.
She basically went on a campaign to try to convince
me to do exactly that.
It wasn't in my blood because I'm an academic.
We write papers, right?
That's what we do.
But Sherry basically told me that the kids that
were at the Washington State School for the Blind,
they need things that they can learn programming with.
It's not a want.
It's a need because they want to participate
in the 21st century, right?
And that's, frankly, what--
if you want to be a computer scientist,
you have to do stuff like that.
So she basically convinced me that it would probably
have more tangible impact on real people
than just writing a paper about it.
But that's hard.
I mean, as a full professor, I don't have to care.
But as I was going through the ranks,
you don't always get academic credit
for making something public.
It's just not part of the lifeblood.
I mean, my university doesn't even
have a category for that kind of stuff, not really.
And so it is a real kind of challenge
to figure out how do I balance my activities.
And I imagine I'm not alone.
Just on this call, I can see a bunch of people
that make their stuff public.
And that how to swing that pendulum back and forth
is a tricky issue.
But for me, it's the same answer as Lauren.
It's the people.
So they need this stuff.
RICHARD LADNER: I think also this area of accessible
technology that I work in and lots of us
work in, I think this occurs to many of us
that, my goodness, we shouldn't just be doing this research,
we should be doing something that is out there,
so people can actually use it.
And I think there's kind of a virtuous circle
there because when people start using it, they find flaws,
they find errors, and it makes these products better
over time.
I was going to mention that there's
this event going on today and tomorrow called Sight Tech
Global.
And it has most of the presentations are recorded,
so you don't have to be there at a particular time.
But I'll put this into the chat, the link to it.
So one thing-- I went to one of the sessions today.
And it was about a new tactile graphics display
that's being developed by the American Printing
House for the Blind in cooperation with Humanware.
And so it really interesting to see that demo.
And I know we're big into screen readers,
but there still is tactile.
And tactile is very important for blind people.
So that's another question, maybe for Lauren.
And one of the things about the iPad
that we liked when we started working on it was it
was tactile.
You got feedback.
And so for example, you put the program or the workspace
at the very bottom along the edge
rather than freely floating.
And that was a conscious decision,
I remember that, because people can find the edge.
Also, they can't explore with their one
finger the different elements.
And they can look over their program and things
like that with the one finger exploration
that the screen readers allow with touch.
So it's not like tactile in the sense of a tactile graphic,
but it is tactile the sense of at least spatial tactile.
So I thought that was kind of interesting in its own right.
And so Stefik, your stuff is going
to be pretty much not tactile.
Is that right, Stefik?
ANDREAS STEFIK: No, actually, we are
going to have a tactile version for all this.
That's in collaboration with Nick Giudice in Maine and Jenna
Gorlewicz Saint Louis.
We do have a tactile version planned.
But there's a lot of design questions about that.
How do you navigate the charts when it's tactile?
How do you navigate the programming environment?
What should it include?
Because it's not the same on a tablet.
Lauren's shaking her head.
Obviously, that's true.
So it's a very different kind of experience.
So the design is TBD, but we are working on it.
So yeah, we're experimenting quite a bit.
It's one reason why we've been adding support for TalkBack
or VoiceOver into Quorum.
BRIANNA BLASER: Go ahead.
I was just going to say, we're just about at our time,
but this was a great discussion.
And if anybody has any questions,
it seems like Stefik and folks might
be willing to talk for another minute.