Visual Programming for the Visually Impaired

Hi
I am a tutor at a college in the UK. Last year I used UE4 with Blueprints as part of a level 3 Software Development and Games Programming course to great effect.

This year I have a blind student (no light perception at all) who is on the course and wants to learn UE4 using blueprints to create a game with uses audio, rather than visual, cues for the player.

To do this I am trying to get funding for a 3D printer, so I can print out various Blueprint nodes, with Braille, which the student can then connect using real, physical connectors. His learning support assistant can then upload his new network of nodes to the UE4 blueprint editor and he can then run it and see what happens.

To do this I have entered a JISC event, ‘Accessible by Design’ (If anyone wants to vote, I would really appreciate it BTW).

Has anyone on here tried anything like this before?

Any advice anyone can give will be very welcome.

Also, has anyone tried to parse the Blueprint source code you get when you copy and paste a network? We intend doing this in Python, any suggestions on which libraries to use?

Thanks,

Steve.

The blueprint code is just a text file in XML type format so you should be able to parse it pretty easily. There is actually a pretty good website where you can paste in blueprint code and it will display the nodes in a web browser, or you can create blueprints in it.

I do like the idea of physically building the blueprints but it sounds like it would get very difficult to navigate after a while, and there are hundreds of different blueprint nodes, and you would need dozen’s of the same node’s as well. How do you intend to get around this issue?

It seems like an audio reader would probably solve this problem a bit better than a physical solution. Or perhaps a braille screen of some sort?

Might be worth getting in touch with the folks who are making this, they might already have solutions for text/iconic mixed environments and this would be good for them as well as this use case of yours needs a really robust solution (also educational use is the best place to get stuff developed with all those massively imaginative students around :slight_smile: ).

(not affiliated BTW, just remember thinking it was good idea for screens in general to have touch feedback as well as the Braille use)

Hi NoisyGuy
That looks like a great idea, but so far it is just a design - there is no technology behind it and with a release date of late 2016, I won’t hold my breath!
Something like this would be really good though , not only for Visually Impaired users, but all users.
Thanks

Hi ZoltanJr
Yes, I have created a simple network, consisting of a couple of nodes and copied it into the clipboard. From there I pasted it into an editor and looked at the structured data. This should be a piece fo cake to parse and build 3D models from. I have also seen the Website where you can paste this code and it will render the nodes - good work by who ever did it!

I think navigating a large network could be a challenge, but it is the only approach I can think of that would make the user independent as far as building the network is concerned. By having a decent 3D printer on hand, new nodes can be printed quickly on demand, which should get round the issue of having a ready supply of nodes. Also, frequently used nodes can be printed ahead of time.

The trouble with audio readers is that they can’t handle drag and drop user interfaces and Braille screen readers are both very, very expensive and don’y have the resolution needed.

On previous tasks, I have used tactile resources and they worked well, hence the extension of the concept to UE4’s Blueprints.

Thanks for your ideas though - I am sure that there are lots of other things I have not thought about which I will find out as I progress this :slight_smile:

Unreal engine 4 blueprints was designed to use in virtual inviroment (endless supply of morphable objects) for a sighted people. And you cannot bypass those restrictions completely especting same outcome, even close to same outcome. You can design semi blueprint kit for a blind person, which would act like a lego and can be connected in a same way, with nodes for most used functions and variables, but thats would teach them only very basic stuff about programming and still not allow any practice usage. So for me, your idea sounds like spending time.
Blind people can program like any other sighted person, so from my point of view and if i was on your place, i would give up this idea and change priority to actual programming. I dont have teaching degree same as i dont have any experience with blind people, so my statements may be incorret, but i would change accent of this course for this person, from “lets make you feel like we all the same and your blindness is not a problem” to “we are not a same, i can see and you can not and this system was designer for people like me, so instead of designing a “fancy bicicle” to make you use stuff which shouldnt be even concidered as a proper time spending, we would teach you how to bypass this visual part of programming and start learning actual programming”(might sound like a dk statement, but denying the truth you wouldnt make better for anyone else).
My statement based on idea of me not be able to design easy to use system which would allow to make stuff like this more effecient than actual programming, but if you can, go on.

Even if you got it to work, it would be a dead end as it would not ever allow your student to effectively use blueprints.

Visual programming is supposed to make stuff easier. If it for some reason makes it harder, like in this case, don’t use it. Your student can become a kick *** C++ coder needing no help from an assistent.

But if you have to do it just for the course, I guess it would work…

thats amazing approach for the education

I understand what you both mean, but I am coming at it from a perspective of inclusion - i.e. the student wants to come on the course, so I should try to make it accessible for him. Parts of that process will be easy, some other parts, such as UE4, not so easy.

At the end of the course, the student will not be able to create a full, complex, game using Blueprints - no one is expecting that. But I do think he can give us an insight into his world, while also passing the assessment criteria, which is set by an external body.

It will give him more of a taste of Blueprints, which otherwise he would be denied.

Another benefit of this project, is to build software which can be used to create 3D models for other things, which can then be used by other people, some of which may be visually impaired.

In other parts of the course I teach other languages, which are text based and so more accessible, but I started teaching UE4 last year and the students really enjoyed it, why not let him enjoy it a little too :slight_smile:

I will post on here to let you know how it goes…