Assistive Technologies and Public Transport
Table of contents 1
Introduction................................................................................................................... 3
2
Methodology ................................................................................................................. 4 2.1
3
Research methodology
4
Personal mobility aids................................................................................................... 5 3.1
Example products
5
3.1.1
Electric wheelchair ..................................................................................................................................................... 5
3.1.2
Standing Electric wheelchair .................................................................................................................................... 5
3.1.3
Stair Climbing Electric wheelchair ........................................................................................................................... 6
3.1.4
Exoskeleton ................................................................................................................................................................. 8
3.1.5
Standing Desk ............................................................................................................................................................. 9
3.1.6
Human Support Robot ............................................................................................................................................ 10
3.1.7
Electric Wheelchair Controller ................................................................................................................................ 10
3.1.8
Accessibility Clarification ........................................................................................................................................ 10
3.1.9
Assistive Walking Stick............................................................................................................................................ 11
3.1.9
Portable Gyroscopic balance device .................................................................................................................... 11
3.1.11 Sip-and-puff ............................................................................................................................................................... 11
4
Wayfinding Technologies............................................................................................ 12 4.1
5
12
4.1.1
Talking Tactile Map .................................................................................................................................................. 12
4.1.2
RFID Torch ................................................................................................................................................................. 12
4.1.3
Video Streaming ....................................................................................................................................................... 13
4.1.4
Indoor Visual Simultaneous Localisation and Mapping ................................................................................... 13
4.1.5
Phone applications................................................................................................................................................... 13
4.1.6
Ultrasonic Distance Sensing .................................................................................................................................. 14
4.1.7
Visual Image Processing (External Device) ......................................................................................................... 15
4.1.8
AI Suitcase ................................................................................................................................................................. 16
4.1.8
Walking cane with embedded antenna ................................................................................................................ 16
Communication for the Deaf, the Blind, and the Cognitive Disability .......................... 17 5.1
5.2
1
Example products
Example products (Blindless)
17
5.1.1
Screen Reader ........................................................................................................................................................... 17
5.1.2
Dictation ..................................................................................................................................................................... 17
5.1.3
Braille Embosser ....................................................................................................................................................... 18
Example products (Deaf)
18
5.2.1
Smart phone or telecommunication device ........................................................................................................ 18
5.2.2
Voice Recognition Software ................................................................................................................................... 18
5.2.4
Bluetooth Hearing Aids ........................................................................................................................................... 19
La Trobe University
Assistive Technologies and Public Transport
5.2.5
5.3
6
Example products (Cognitive Disability)
6.2
5.3.1
Speech Generating Device ...................................................................................................................................... 20
5.3.2
Education Robot ....................................................................................................................................................... 21
Example products (Blindless)
6.3
22
6.1.1
Screen Reader ........................................................................................................................................................... 22
6.1.2
Dictation ..................................................................................................................................................................... 23
6.1.3
Gesture Recognition ................................................................................................................................................ 23
6.1.4
Braille Display ............................................................................................................................................................ 23
Example products (Deaf) 6.2.1
8
20
Interaction with the Internet for the Blind, People with a Physical Disability and Cognitive Disability ..................................................................................................... 22 6.1
7
Hearing loop .............................................................................................................................................................. 19
24
Caption/Subtitles...................................................................................................................................................... 24
Example products (Cognitive Disability)
25
6.3.1
Word Prediction ........................................................................................................................................................ 25
6.3.2
Simplified Word Processors ................................................................................................................................... 25
6.3.3
Simplified Smart Device .......................................................................................................................................... 25
6.3.3
Oversized Trackballs................................................................................................................................................ 26
Payment Methods for Deaf, Blind, and Cognitive Disabilities ..................................... 27 7.1.1
Closed-Loop VS Open-Loop System..................................................................................................................... 27
7.1.2
Simplified Smart Device (Open-Loop System).................................................................................................... 27
7.1.3
Voice Response System (Open-Loop System) ................................................................................................... 27
7.1.4
Electronic Tag (Closed-Loop System) .................................................................................................................. 27
7.1.5
Auto-renewing Subscription ................................................................................................................................... 28
7.1.6
Device-initiated Payment ........................................................................................................................................ 28
Next steps: Public Transport can benefit – start thinking about standards ................ 29 8.1
Common Standardised Aspect (Electric Wheelchair)
30
Summary and Conclusion.................................................................................................... 31 Table of Products ................................................................................................................ 33 List of Figures...................................................................................................................... 34 Abbreviations ...................................................................................................................... 36
2
La Trobe University
Assistive Technologies and Public Transport
1 Introduction Since Ancient Egyptian times when the crutch was invented, humans have consistently sought to improve the quality of life for those living with disability through assistive technology. From simple prosthetics invented in the Middle Ages, to the more recent inventions, such as autonomous wheelchairs and emotion monitoring wearables, assistive technologies have come a long way. Assistive technology can be understood as a broad term, covering inventions and technologies ranging from simple devices such as walking stick or reading glasses to more complex, high-tech systems such as exoskeletons and motion capturing software. The primary definition of assistive technology as suggested by the World Health Organisation (WHO) is the products where the “primary purpose is to maintain or improve an individual’s functioning and independence to facilitate participation and to enhance overall well-being”. [2] This report aims to provide an overview of assistive technologies currently available in the market or in development which may impact public transport operators. The report focuses on assistive technology that intends to support four different aspects which are 1 2 3 4
3
Wayfinding Personal mobility aids Communication Human Machine Interaction
La Trobe University
The first chapter of the report identifies a list of references that were used during the research process. The reference range varies from online research to conference papers and magazines. This serves as a way to ensure that the most updated trends in assistive technology have been analysed. The second chapter of the report will be divided into four parts, one for each functional category. In each section, technology trends within well-established assistive technology are explored in-depth, followed by an analysis of how the assistive technology may serve public transport. We hope that this overview will help public transport providers to inform their own technology roadmap. Public Transport is about to undergo a major ITS transformation, involving very large investments in technology and data systems, which should take into account technology developments in the field of assistive technologies in order to: 1 Leverage the opportunities and ensure incorporation of the major assistive technology devices, 2 Ensure national interoperability and consistent integration This report is a first stage of a project the Centre for Technology is undertaking to assess technical requirements. We thank David Sinclair, CEO of The Assistive Technology Suppliers Australia (ATSA) for his input. The next stage is to look into the technical requirements and systems and look for common factors that can be implemented by the transport sector, please let us know if you have any specific questions.
Assistive Technologies and Public Transport
2 Methodology 2.1
Research methodology
In order to make an inventory of the most up to date solutions for the current assistive technology, we were assisted by Assistive Technologies Supplier Australia ATSA - and looked at the recent conferences held by different assistive technology organisations. There are several technology-focused conferences where attendees can learn more about the latest updates and explore the technology available to people dealing with disability. The following table provide a list of conferences that hold the most value:
1.
The Assistive Technology Industry Association Conference
2.
The California State University, Northridge (CSUN) Assistive Technology Conference 2021
3.
The M-Enabling Summit
4.
The National Federation of the Blind and American Council of the Blind Conference and Conventions
5.
4
American Foundation for the Blind (AFB) Leadership Conference 2020
La Trobe University
While the conferences provide the most updated technology developments in the accessibility space, there may be people who are unable to attend the conference due to the current COVID situation in North America. Therefore, several magazines that covers many of the assistive technology conferences have also been reviewed. The list of magazines and reports are shown below
1.
AFB’s AccessWorld Magazine
2.
ABILITY Magazine
3.
Enable Magazine
4.
BOSS Magazine
5.
E&T (Engineering and Technology) Magazine
6.
WIPO Technology Trends 2021
7.
The Paypers - Payment Methods Report 2020
Assistive Technologies and Public Transport
3 Personal mobility aids 3.1
Example products
3.1.1 Electric wheelchair A powered wheelchair usually consists of four basic parts which are motor, battery, drive and controller. The market usually offers a choice of 2-pole motor and 4-pole motor. The 4-pole motor offers more carrying power and allows more important options. Power wheelchairs often employ lead acid batteries (SLA). The output ranges from 4 to 5 amps and can be recharged via standard electrical outlet. Many accessories and functionality can be added to the motorised wheelchair as personal customisation. These include sensors and computing capacity making this aid smarter.
Figure 2: Moby
The Genny
Phoenix AI Ultra-Light Wheelchair
The Genny is a wheelchair that is designed to indulge your adventurous side by performing just as well on beach fronts, snow, and muddy trails as in urban environments.
The Phoenix AI wheelchair, shown in Figure 3 is an ultralightweight manual wheelchair made from carbon-fibre. Using smart sensors, the chair will configure itself to what the user is doing so it remains in sync with how the user moves. The sensors detect if the user is leaning forward or back, algorithms will calculate the wheelchair's response. The chair will continually adjust its centre of gravity to fit what the user is doing making for a chair that is easier to push and turn by eliminating drag and uncomfortable, painful vibration while also making the chair safe from falling backwards.
With two large parallel wheels and an array of electronic sensors as shown in Figure 1, the Genny is based on technology of the Segway. It works by detecting shifts in the user’s body weight, which it uses to know whether to go forward, slow down, turn or brake. This also means it can be used hands free.
Figure 1:Genny Electric Wheelchair
Moby Integrated network of wheel-on powered devices, allowing users of manual wheelchairs convenience and benefits of a powered chair, accessible via app-based share scheme. Moby is the first mobility service created for wheelchair users. It is the cycle share scheme equivalent for wheelchair users. Consisting of a series of wheel-on electric devices, located in urban hubs, it will make travelling around cities much simpler and easier for people with lightweight manual wheelchairs.
5
La Trobe University
Figure 3: Phoenix Wheelchair
3.1.2 Standing Electric wheelchair Standing wheelchairs can be operated manually or power-operated which depends on the user. The aim of a standing wheelchair is to allow the user to achieve regular mobility and to stand the person up using hydraulics or other power source as presented in Error! Reference source not found.. Some standing wheelchairs can be driven in standing position.
Assistive Technologies and Public Transport
As the patient is standing up, studies have shown that standing wheelchair can generate benefits for the user. This can include improved circulation, urinary health, and bone density. In comparison to normal power wheelchairs, being able to stand up in a wheelchair also distribute the user’s weight and improves healing of pressure sores. These are often due to the lasting effect caused by sitting in one position for a long period of time. This can include joint and muscle stiffening.
Operation: The stair climber can be used in conjunction with any wheelchair whether it is provided by a transport company or a private one. The stair climber allows a physically disabled person to climb the stair while using a wheelchair. This allow users access to any train stations or public places where stairs are a necessity in order to get to the platforms or levels. It is important to note that all approaches fundamentally function based on the robust edges of the stairs. [1]
Domino People An add-on device that attaches to the manual wheelchair and allows the wheelchair to ascend and descend stairs as shown in
Figure 4: Standing Electric Wheelchair
QOLO (Quality of Life with Locomotion) QOLO is a mobile exoskeleton on wheels, allowing users to sit or stand with ease. The QOLO Standing Device consists of a lightweight, mobile exoskeleton on wheels which uses passive actuators to allow users to sit or stand, effectively removing the 'chair' from 'wheelchair'. Mobility is controlled using the upper body, allowing hands-free operation.
Figure 6Error! Reference source not found.. Domino People is designed to transport hand-operated and allows the wheelchair to be lifted up and down staircases. It also maintains comfort and control for the user while operating. [2]
Figure 6: Domino People Wheelchair Figure 5: QOLO Standing Wheelchair
3.1.3 Stair Climbing Electric wheelchair The stair climbing technology is an add-on device that attaches to the manual wheelchair and allow the wheelchair to ascend and descend stairs. There are currently several designs that allow stair climbing.
6
La Trobe University
Scewo The Scewo electronic wheelchair automatically maintains a level seating while driving. The large wheels are designed to master cobblestones or forest paths and allow the driver who does not have any upper body stability to use this equipment easily.
Assistive Technologies and Public Transport
the user to enter a standing position via raising the sitting level up to 6ft in height. [5]
Figure 7: Scewo Electric Wheelchair The Scewo electronic wheelchair also allows the user to climb stairs made from different materials which have a maximum gradient of 20 degree to 36 degree. It is important to note that the max step height is set to be 200 mm. [3] The wheelchair also offers versatility. The wheelchair can be adjusted via seat depth, back support, foot and arm supports.
Mobile Stairlift Similar to that of the options above, the Mobile Stairlift electric wheelchair provides an option for the disabled user to conquer the stairs with the help of another user as displayed in Figure 8. It also does not require any preinstalled tracks to climb up the stairs. The wheelchair can carry up to 500 pounds of load. Safety measures are included such as strapping in the user with belt and footrest. Note that this wheelchair is electrically operated on the stair climbing aspect only. [4]
Figure 9: iBot
Garaventa Stair-Trac Different than the examples given before, the Garaventa Stair-Trac is a portable wheelchair lift that can attach to most standard wheelchairs and can be used both indoor and outdoor as shown in Figure 10Error! Reference source not found.. The machine itself can carry a maximum 150kg user up to 30 individual stairs. The machine can be used on any indoor or outdoor stairways. The machine has a detachable support handle for compact storage and easy transport. The machine can still be used when electrical power falls or is not available. However, a second user is required in this case. [6]
Figure 10: Garaventa Stair-Trac Wheelchair
Figure 8: Mobile Stairlift
iBot The electric wheelchair known as iBot has two sets of powered wheels that can rotate and allow users to “walk” up and down the stairs. Like that of the option above, this electric wheelchair is fully electrically operated. It allows
7
La Trobe University
Assistive Technologies and Public Transport
3.1.4 Exoskeleton As the name suggests, an exoskeleton is an external frame that can be worn to support the body, either to help a person overcome an injury or to enhance their biological capacities. Powered by a system of electric motors, the frame gives limbs extra movement, strength and endurance. [7] Operation: Most exoskeletons currently in the market use an EMG signal to monitor the motion of the user’s body. The signal is then transmitted and activates a sequence of motion from the body model, assisting the user in doing various actions via a torque controller.
The following list provides examples of exoskeleton models:
Rewalk Rewalk is a wearable robotic exoskeleton that provides powered hip and knee motion to enable disabled people with spinal cord injury to engage in various motions such as standing up, walking, turning, climbing and descending stairs. The Rewalker controls the movement using changes in centre of gravity. If the user moves forward and tilts the upper body, the system will sense the change and initiate the first step. Repeating such action mimics a functional natural gait of the legs. [8]
8
La Trobe University
The exoskeleton can also combine with the user’s smart device, for instance in order to receive the most updated information regarding train arrival time and population. Some exoskeletons also include smart assistant software which allow users who can generate the slightest movement to command the exoskeleton and act upon the command. This functions in cases where the user has to climb up or down stairs. Different types of exoskeleton can be found in the table below:
Phoenix Medical One of the world’s lightest and most advanced exoskeleton designed to help people with mobility disorders to be upright and mobile. It has enabled people to stand up, walk about, and speak to peers’ eye-to-eye during work and at home. The exoskeleton has two actuators at its hip and knee joints that are designed to allow support during stance and ground clearance during swing. [9]
Lockheed Martin Lockheed Martin’s exoskeleton has a primary focus on powered exoskeletons. The Lockheed Martin exoskeleton focuses on lower-body exoskeleton, ONYX,
Assistive Technologies and Public Transport
has shown the ability to increase mobility and reduce fatigue of its user. By minimising the effort in walking and climbing, this technology can literally help soldiers and first responders go the extra mile while carrying mission-essential equipment. [10]
EksoNR (Lower Body) The EksoNR is designed to help patients to stand and walk during rehabilitation. It provides the necessary support to the spine, trunk and legs including hip, knee and ankle joints promoting correct movement patterns in all phases of physical rehabilitation and challenging patients as they progress towards walking out of the EksoNR rehabilitation setting. This lower limb exoskeleton even comes with touch screen controls which help clinicians set goals and alter assistance levels for patients of all different functional levels. Physical therapists can also easily access the medical device’s database to analyse complex movement patterns and record patient progress. [11]
tailor the behaviour of the system to specific impairment and gait needs. [13]
Myomo This exoskeleton aims to help people whose arms are paralysed by stroke, injury or disease. MyoPro is a powered orthosis (brace) that may help restore function in arms and hands paralysed by a stroke, brachial plexus injury (BPI), cerebral palsy or other neuromuscular disease or injury. The MyoPro arm and hand orthosis device works by reading the faint nerve signals (myoelectric signals) from the surface of the skin (fully non-invasive, with no implants) then activating small motors to move the limb as the user intends (no electrical stimulation). [14] The user is completely controlling their own hand, wrist, elbow, and arm. The robotic arm brace amplifies weak muscle signals to help move the upper limb.
3.1.5 Standing Desk The seat height of wheelchairs can vary, but typically, it is set lower than the seat height of traditional office chairs. Traditional tables are commonly designed to pair with the height of standard chairs, but not wheelchair height. This poses a challenge to wheelchair users in finding the right desk, as many fixed-height desks are unpleasant or ergonomically incorrect.
Figure 11: Lower Body Exoskeleton
EksoUE Similar to that of the exoskeleton above, EksoUE was designed to assist a patient affected by shoulder and arm restrictions and has been a revolution in upper limb rehabilitation. This allow the patients who are suffering from upper extremely paralysis or weakness to recover strength, endurance and range of motion. [12]
A height adjustable desk provides an excellent solution for disabled users as shown in Figure 12. Even if not used as a standing desk, the adjustment capabilities often mean that desktop height can be set lower than many conventional desks. The ability to modify the desk’s height increases openness and comfort for many who have struggled to find the right set-up. [15]
Parker Hannifin Parker Hannifin exoskeleton allows individuals with spinal cord injury to stand and walk again.
Figure 12: Standing Desk
The Therapy+ software suite, included with each Indego Therapy device, incorporates control algorithms based on proven motor learning principles and allows for an individualised, patient-centric training approach where the device responds to a patient’s active contribution and assists in gait only when necessary. Additionally, therapists have a range of customisable settings within the Therapy+ software suite which allow them to further
Operation: As the name suggests, the standing desk allows the user to adjust the height of the desk based on personal needs. In the case of a train station, the information desk that provides helpful information to travellers can be hard to reach for people who have physical disabilities. The standing desk solves this issue by lowering the desk and allowing the user to use the table.
9
La Trobe University
Assistive Technologies and Public Transport
3.1.6 Human Support Robot
future the Munevo Drive will identify the user’s location via GPS and share the user’s position. [17]
The human support robot is best described as a robotised ‘butler’ designed to assist around the home or public area. The Toyota Human Support Robot, Toyota is a leading company in Human Support Robots, designing a product that can be operated by voice command or by tablet computer, or that can recognise surroundings to move by itself. It has a lightweight cylindrical body with an extendable folding arm that enables it to pick up objects off the floor, suction up thin objects, retrieve objects from high locations, open curtains, and perform other household tasks. HSR has omni-directional wheels, enabling it to move smoothly in any direction. [16] Operation: The robot functions by voice command and uses vision algorithm to recognise the surroundings such as people or physical objects. The current robot is also able to retrieve objects from locations as seen in Figure 13Error! Reference source not found.. If google assistance is added to the robot, the robot can also provide the most updated transport information for the user without them moving around.
Figure 14: Munevo Drive Operation: This device functions as an alternative way to control the power wheelchair. The eyeglasses are connected to a device wirelessly and the device is connected via a cable to the power wheelchair. The integrated camera on the eyeglasses can translate head movements in control signals, which allow the user to control the wheelchair without any hand movement.
3.1.8 Accessibility Clarification While the wheelchair is a popular option for people experiencing physical disability, not all places are accessible to wheelchair users. This assistive technology aims to allow the user to “know before you go” whether a destination is wheelchair accessible.
Google Maps
Figure 13: Human Support Robot
3.1.7 Electric Wheelchair Controller Munevo Drive is a smart glass application that uses the built-in sensors to translate head movements into control signals. These signals are then transmitted to the wheelchair control unit via an adapter that is plugged into the electric wheelchair as mapped out in Error! Reference source not found.Error! Reference source not found.. This enables hands-free control of the electric wheelchair. ▪
▪
10
With the integrated camera of Muneva Drive, the user can capture special moments and share them with friends afterwards. If the user has an emergency, the user can send an emergency message to their chosen contacts. In the
La Trobe University
People can now turn on an “Accessible Places” feature to have wheelchair accessibility information more prominently displayed in Google Maps. When Accessible Places is switched on, a wheelchair icon will indicate an accessible entrance and the user will be able to see if a place has accessible seating, restrooms, or parking. If it is confirmed that a place does not have an accessible entrance, the information will be shown on the map.
Figure 15: Google Map
Abil.io
Assistive Technologies and Public Transport
Allows customers with limited mobility to use the public transport system through planning the trip in real time and provide comfortable walking distances. The app also avoids any significant uphill or downhill slopes.
3.1.9 Assistive Walking Stick
Operation: In the case of public transport or transport station where the ground would be unstable for physically disabled user, this backpack provides a solution and a sense of ease for them. The backpack which includes an open loop gyroscopic-assisted system can naturally balance the user at any state of time.
An assistive cane is a walking stick that relies on the user crutch. The aim of this assistive technology is to redistribute the weight and improve the stability of the user by increasing the base of the support. It also increases the area of ground contact for the user and thus, improving the mobility. One of the more popular option is the tripod cane that opens in a tripod fashion. It is often available for use with an attached seat. Operation: The assistive walking stick provides an alternation support for all travellers with mobility aid. The physical body of the walking stick acts as a weight distributor for the user to rely on and increase the total areas of surface in contact with the ground for further stability.
Figure 17: Portable gyroscopic-assisted system
3.1.11 Sip-and-puff SNP is one type of assistive technology that sends signals to a device using air pressure by inhaling and exhaling on a tube. It is commonly adapted in motorized wheelchair. The device which contains the SNP technology requires a specific amount of air pressure to be considered slipped or puffed by the user. This technology can be implemented in motorized wheelchair control input where an initial hard puff will enable the wheelchair to move forward and as such.
Figure 16: Walking Cane with seat attached
3.1.9 Portable Gyroscopic balance device This assistive technology aims to provide a corrective torque for the disabled user and automatically balance the user in times of need. It consists of a gyroscope with flywheel mounted in an inner gimbal. The assistive device is presented as a backpack that contains a set of a number of gyroscopes and is located close to the centre of the mass of the user. The adaptation of an open loop system working in conjunction with variable-speed control moment gyros allows the backpack to continuously balance the user without affecting the body movement.
11
La Trobe University
Operation: The Sip-and-puff system provides a solution for users who have both visual impairment and physical disability. As adapted to motorized wheelchair provided within the transport station, this system allows the user to control the electric wheelchair without any physical strain.
Figure 18: Sip and Puff wheelchair
Assistive Technologies and Public Transport
4 Wayfinding Technologies 4.1
Example products
4.1.1 Talking Tactile Map Talking or tactile maps have existed for some time and can increase the individual’s overall knowledge of an area. It can exhibit the environment in a small-scale model. This method places great demands on a persons’ memory as the overall image of the place is internalised from the hand movements and the fingertips, or by listening. This can lead to perceptual overload. The challenge is to display the minimum amount of detail necessary for the individual to learn, comprehend and navigate through environment.
4.1.2 RFID Torch This method is focused more on mobile navigation solution, using RFID tags as geographic markers. The tag triggers the torch to speak the information of the desired location (where the mark is placed.) When the user lowers the torch, the description will then stop. The information of the place is generally provided via a wiki-style website known as WikiNac, an online database for location-based information.
One solution is for the computer audio-based system to augment the linework on the tactile graphic by applying sound labels to point, lines and areas on the map.
Figure 20: RFID System Figure 19: Talking Tactile Map System Operation: The map itself is placed on a small tablet connected to a computer as shown in Figure 19. As the map is touched, the corresponding sound label is triggered and conveys the necessary information for the user. The audio is generated through internal and external speech synthesiser. Requires the map of the station to be drawn and presented as a 2D touch window. The corresponding sound file can be stored in an offline database. [18]
Tactile Map Automated Production (TMAP) TMAP users who have their own Braille embossers (Braille printers) can use a web interface to specify the location and size of the desired tactile map. The map file and related key information are generated and can be downloaded immediately. The tactile map can then be "printed out" on the user's own Braille embosser. TMAP is compatible with any graphics-capable Braille embosser, including those manufactured by View Plus, Enabling Technologies, and Index. [19]
12
La Trobe University
Operation: RFID uses electromagnetic fields to identify and track tags attached to objects. The system itself usually consists of a tiny radio transponder, radio receiver and transmitter as mapped out in Figure 20. The tag will transmit data when it is triggered by an electromagnetic interrogation pulse from a RFID device. For the architecture of this technology solution, the torch connects to a mobile phone through a Bluetooth link. This function requires the instalment of different geographical markers along the station. This can include the entrances, exits, stairs, lifts and even gates. The main purpose is to provide the user with a general sense of where they are located. The RFID torch uses electromagnetic interrogation to detect signs nearby. As the RFID triggers the tag, it will release the specific information to the RFID torch. As the RFID torch is connected to the phone via Bluetooth, the user can then listen to the sound audio via earphone. [20]
Assistive Technologies and Public Transport
4.1.3 Video Streaming Similar to that of livestreaming, the user first wears a portable device, which in this case is an eyeglass, that live streams the current situation to a trained agent. Operation: video streaming works by simultaneously recording small bits of video at a time and sending it through to the machine of the user. The machine then downloads the small audio file and plays it. This process repeats within a small timeframe and becomes “livestream”. [21] In the case of a train station, the eyeglasses can be provided for users and allow the transport agents to guide the users towards the specific platform or exit.
Aira (Be My Eye) The application connects you to a trained agent online. The trainer looks through the phone camera/eyeglasses to see where the person is and provide guidance based on the surroundings
Figure 21: Video Streaming
4.1.4 Indoor Visual Simultaneous Localisation and Mapping Visual Simultaneous Localisation and Mapping (VS-LAM) can be used for location and positioning using visual inputs from a camera. The technology itself requires only one single camera sensor (from smartphone.) This technology is mostly used as an indoor navigation solution that employs VS-LAM algorithm to solve issues regarding indoor localisation as GPS does not function in this case. The system employs a dynamic sub-goal selecting strategy to help users in navigating through avoiding obstacles on the way. One example sees the VS-LAM being connected with a cloud server and the major components include a helmet with stereo cameras, an android based smartphone, a web application and a cloud computing platform. [22]
13
La Trobe University
Figure 22: Stereo Vision Operation: This technology is mainly used in places where GPS is not provided and can also provide improved accuracy compared to GPS. The company first stores the program data in a database locally or in the cloud as shown in Figure 22. This includes speech processing function, path planning, object detection etc. Then, the user uses the smart phone to access the application via the internet. A helmet mounted with stereo camera is also used in this case in order to process the 3D image. This will allow a clear direction for the user to be made. In the case of an enclosed trains station, this provides a sense of security for the user as it provides a speech function that guides the user via the stereo camera. The stereovision functions in a similar way to that of 3D sensing in human vision. It firstly identifies the image pixels that correspond to the same point in a physical scene observed by multiple cameras. The 3D position of point can then be known by triangulation using a ray from each camera. The more corresponding pixels identified, the more 3D points can be determined with single set of images.
4.1.5 Phone applications Assistive technology is increasingly offered via the smart phone or tablet. The aim of assistive technology based on smart devices is to provide easier access to a user with a disability while minimising the cost to the end user. Most of the applications require the access of GPS and camera in order for the system to function.
Blindsquare BlindSquare uses GPS and compass to pinpoint your location. It gathers information about the surrounding environment from FourSquare. The application then applies a unique algorithm to decide what information is most relevant and presents to the user with speech synthesis. Blindsquare also functions in indoor facilities via the use of Beacon Positioning Systems (BPS). It requires a module, iBeacons in this case to send out
Assistive Technologies and Public Transport
signals that a smartphone receives and informs the user. [23]
NaviLens NaviLens reads signs out loud and provides train arrival information in audio to assist the blind and low-vision users navigating around the station. The application works in sync with Voice Over accessibility settings on any smartphone in order to guide the user around the interface. The phone does not need to focus and can scan tags up to a 160-degree angle even while in motion. The user simply needs to sweep their environment with a smartphone. Then, the audio cues will allow them to find and centre the tag in phone’s field of vision in order to decipher the QR code as shown in Figure 23. [24]
videos on the headset, and there’s a photo gallery for viewing and saving pictures you take. IrisVision customers can obtain tech support from service providers who can connect to the device remotely. [26]
Arianna (Indoor) The system combines sensors and interfaces present on commercial smartphones to guide the visually impaired in an effective way, even in unknown environments. Operation: the camera of the smartphone continuously captures the scene in front of the person and identifies the path by recognising special-coloured strips painted or stuck to the floor. The user receives a vibration signal through his cell phone. The vibrations are related to the position and direction of the user and allow him to understand the direction to follow and the presence of path variations. [27]
Aipoly
Figure 23: NaviLens
Aipoly Vision is a recognition app that helps the blind, visually impaired and colour-blind people understand their surroundings. Simply point your phone at the object of interest, push the recognition button at the bottom of the screen, and it will recognise the item and explain what it is to you.
Google “Look out” and Microsoft “Seeing AI”
Stop Announcer (NSW)
The application known as “Lookout” for Google Pixel devices can help users locate nearby objects, identify bar codes and read texts. The user can also take a snapshot and have the application describe the scene to you. The app also identifies the position of the nearby object using a clock face. [28] Similarly, the Microsoft’s Seeing AI allows user to explore objects by touch. Imagine a photo of a person standing next to a tree with a bench in the background. With this photo on the screen, a VoiceOver user can move a finger around the phone or tablet and each object will be spoken as it is touched. This allows for a greater level of spatial awareness and a better understanding of photos and images. [25]
The app is designed for customers with vision impairment. It is a route guidance application that provides audio notification of stops made along the traveller’s route as the transport approach and leave the stop.
The IrisVision Comprises of a Samsung VR headset and smartphone and is intended for people with central vision loss of the kind caused by macular degeneration, as well as users with field restrictions caused by glaucoma or RP, according to IrisVision. The updated platform, now called IrisVision Live, provides OCR with speech, and can project text in large print as it speaks. The device now includes voice commands for basic functions, like zoom and brightness. There’s also Google Assistant functionality, with support from Google and from Samsung engineering. The user can watch and search for YouTube
14
La Trobe University
CogNav A smartphone app that provides navigation assistance in mainly campuses and shopping malls. Similar to other assistive applications, it utilizes facial and object recognition technology to recognize all sort of landmarks. It then provides audio instructions for the user and alert him of the surroundings. The application recently added a personal object recognition system that help recognize personal items based on memory and deep-learning techniques.
4.1.6 Ultrasonic Distance Sensing A chirp is emitted from the speaker which is located at the front of the device. The ultrasound wave bounces off a nearby object and echo back to the microphone. The time it takes to travel back to the sensor is used to determine the distance.
Assistive Technologies and Public Transport
Miniguide Mobility aid Uses ultrasonic echo-location to detect nearby objects for the user. The aid vibrates at a different rate depending on the distance between the user and the object. The Miniguide will vibrate at a higher pace if the user is nearer to the object. An earphone jacket is also in place in order for the user to listen to any sound feedback. [28] The aim of this device is to act as a secondary aid in conjunction with guide dogs or walking cane. It is important to note that this device cannot detect drop offs in certain location such as railway station or stairs.
WeWalk Cane WeWalk has designed a smart cane that adapts both the original walking cane and modern-day technology to ease navigation for the blind. The cane detects above-ground obstacles with ultrasound while still retaining the standard white cane’s ground feedback. As the cane connects to the user’s smartphone, the user can keep one hand free and leave his device in the pocket. The smart cane allows the user to use his voice and touchpad for quick smartphone control. This may include setting up destinations or stops. The smart cane uses clock direction, destination tracking and low-vision mapping to ease the trouble for the user. WeWalk also has its own voice assistance to allow discovery of new places around the user. It provides automatic voice feedback as you walk past points of interests or restaurants. The decision is up to the user. [29]
ears allow for spatialised audio – the creation of sounds that seem to be coming from specific locations around the user. If the device detects a person one meter away on the user’s left side, the system will play a click that sounds like it is coming from one meter away on the left. If the system recognises the person’s face, it will play a bump sound, and if that person is also known to the system, it will announce their name. [30]
3D Assistive Camera As mapped out in Figure 25, the system includes a 3D camera that continuously scans the environment at 20 times per second. The information is then processed with computer vision algorithms and broken down into individual objects. Each individual object is then presented to the user through 3D computer generated sound. The user will be able to hear the object through vibrations on a belt on the user’s abdomen. The intensity of the feedback is again depending on the user’s distance from the object. [31] The device can also point out where the text is within the vision of the user. For example, the user can read a text by pressing the button and the system will look for the texts and indicate him where the texts are.
Figure 25: Visual Image Processing
Amazon “Show and Tell” With Show and Tell, blind and low vision customers can hold up an item to the Echo Show camera and ask, ‘Alexa, what am I holding,’ and Alexa helps identify the item through advanced computer vision and machine learning technologies for object recognition. Figure 24: Ultrasonic Distance
4.1.7 Visual Image Processing (External Device) Microsoft HoloLens A device that is in development for this user case. The device has an array of grayscale cameras that provide a near 180-degree view of the environment and a highresolution colour camera for high-accuracy facial recognition. In addition, the speakers above the user’s
15
La Trobe University
Operation: It functions similarly to that of the stereo camera where the 3D camera constantly monitors the surrounding. Each picture taken is then processed with computer vision algorithm in order to monitor the surrounding environment for the user. The processed information is then relayed to the user via computer speech and vibration on a belt on the user’s abdomen. This can be a prototype for train transport users as it is extremely bulky for casual use at the moment.
Assistive Technologies and Public Transport
4.1.8 AI Suitcase IBM, Japanese The small navigation robot is shaped as a suitcase that can plan an optimal route to a destination based on the user’s location and map data. It also relies on multiple sensors to observe its surroundings and AI functionality to prevent the user from bumping into incoming obstacles. The AI suitcase requires connection with the personal mobile device. As the user gives command on his personal device while connected to the suitcase, the AI system provides voice guidance and move accordingly to guide the user towards destination. It is also able to stop for doors and swerve to avoid other travellers. [32]
Figure 26: AI suitcase Demo
4.1.8 Walking cane with embedded antenna The walking stick in conjunction with embedded antenna aims to detect audio information from a module located at a bus stop or on a sidewalk. Acting as an external device on top of the smart cane, this technology allows the user to receive information of the surroundings with a specific had gesture or lifting the walking cane as a RFID torch.
Figure 27: Antenna embedded walking cane
16
La Trobe University
Assistive Technologies and Public Transport
5 Communication for the Deaf, the Blind, and the Cognitive Disability 5.1
Example products (Blindless)
5.1.1 Screen Reader The purpose of a screen reader is to read out loud the content that is on an electronic device screen. Screen readers allow users to access a computer and communicate with the people around the world when magnification no longer helps or when the user may experience visual fatigue. Operation: This technology functions as a screen reader on user’s smartphone. The image is first captured using the user’s smartphone or a conversation between users. The picture is then sent to the server where the vision algorithm runs and identifies the text in the photograph. As the information is relayed back to the user, the sentence is announced through a speech synthesizer to the user. In the case of a train station, it allows the user to read the current train timetable. Screen readers usually consists of two primary components which are the voice synthesizer and the screen reading program. The screen reading program first translates the written text displayed on the screen for the voice synthesizer which then produces the text as speech. The screen reader can also read back other information that may be presented on the screen such as text messages. This allows for communication and interact with blind users. [33]
KNFB Reader This application can be installed on a smartphone. It is used to detect text on a page and photographs it. It uses OCRs (Optical Character Recognition) to read the texts in the photograph and reads it in a life-like audio voice. The technology can also take in photos with signs, handouts, mail and other documents and read it out for the user. [34]
OrCam My Eye 2 This device can be attached to any temples of the eyeglasses frame. This technology consists of two different ways in which the user can use the device for reading text. Similar to that of the technology above, the first way is a simple text reading feature.[38] The second option allows the user to verbally instruct the device to find specific information in the text. More functionality is added, including AI used to tell people ‘what’s in front of them’.
5.1.2 Dictation Dictation software combines the effectiveness of speechto-text, voice recognition, voice-to-text and speech recognition which allow the user to interact with the internet by simply using his voice. The dictation software requires a mic to record the user’s speech. The speech is then translated into text using speech-to-text software and voice recognition. The action request is then run against the online database and acts upon the provided action. This allows the user to control the device by simply speaking to it. [35] ▪ ▪ ▪ ▪
Apple Dictation Window 10 Speech Recognition Dragon by Nuance Google Dictation
Operation: In the case of public transport, it allows the blind user to dictate the device without vision and gain all sorts of information from the internet and even communicate with other users on the net.
Figure 29: Google Dictation Software
Figure 28: KNFB Screen reader
17
La Trobe University
Assistive Technologies and Public Transport
5.1.3 Braille Embosser This technology prints out braille output using a computer by punching dots onto paper. It has a similar setup to that of a regular computer. In the case of public transport, by placing different braille printers, the users can identify their specific location and communicate with partners who are not nearby. There are different companies expanding on this technology, which depends on speed, ease of operation, quality of the braille, singe or continuous paper feed and graphics. ▪ ▪ ▪
VP Columbia Romeo 60 Embosser Braille Buddy
to communicate with transport officers even if they are deaf or hearing impairment.
Pedius (Phone application) Pedius is a communication application that allows deaf and people who are having trouble hearing make phone calls using speech-recognition and synthesis technologies. The voice recognition software translates what the person on the other phone has to say and displays it in an easy-to-read format. Using this technology eliminates the need of third-party translator to relay the communication between the two users. [37]
Ava (Portable Translator) The application Ava is known as a portable translator. As displayed in Figure 31, the application allows deaf or head-of-hearing individuals to engage another person in conversation by holding up the phone to the person who they are speaking to. The application then translates the speech into text instantly. It is also important to note that the application is not limited to one-on-one conversations. [38]
Figure 30: VP Columbia
5.2
Example products (Deaf)
5.2.1 Smart phone or telecommunication device The smartphone itself can run as a medium for all sorts of assistive technology applications. Furthermore, the screen can relay all sorts of information to deaf users. In the case of a train station, by providing different smart devices around the station, the users can find out all information needed without hearing anything.
5.2.2 Voice Recognition Software Voice recognition software that targets the Deaf users relies on the use speech identification to pinpoint the information being communicated in between. The software works by breaking down the audio of a speech recording into individual sounds and analyses each sound. The processed sounds are placed into an algorithm to find the most probable word fit in that language and transcribes those sounds into text. [36] As an add-on to the smart device as mentioned above, this software utilises speech recognition and synthesis technology to translate what the other person on the other smart device has to say. The information is then relayed to the user and allows the user to ‘read’ the speech. In the case of a train station, it allows the users
18
La Trobe University
Figure 31: Ava Portable Translator Operation: The voice recognition software allows the deaf or user with hearing impairment to engage with another person in conversation by holding up the phone to the person who they are speaking to. The adaptation of mic and speech recognition allows the sentence to be presented on the application, thus allowing the deaf user to “read” what the other person is saying. In the case of a train station, this should be placed all around the station and especially the information centre to provide the most support for hearing impaired travellers.
5.2.3 Sound Enhancer As described in the previous section 5.1.1, the definition of deaf also includes people who have hearing impairment. While the person may still hear specific sounds based on the environment, the person may not be able to communicate or analyse the surroundings based on the given information processed via sound. The aim of
Assistive Technologies and Public Transport
sound enhancer is to highlight specific parts of the sounds that may otherwise be lacking to that of the user and allow the user to process the surrounding sound fully.
Cochlear A cochlear implant is a small, complex electronic device that can help to provide a sense of sound to a person who is profoundly deaf or severely hard-of-hearing. The implant consists of an external portion that sits behind the ear and a second portion that is surgically placed under the skin. [39] An implant does not restore normal hearing. Instead, it can give a deaf person a useful representation of sounds in the environment and helps him or her to understand speech.
of earbuds. The aim of Bluetooth hearing aids is to make it possible to connect with personal electronic devices and stream the sound directly towards the hearing aids. As the Bluetooth technology in hearing aids elevate at each moment, it allows the user to experience highly personised custom audio.
Oticon Oticon, as shown in Figure 33, combines the ability of a hearing aid with that of the ear buds. The Oticon hearing aid delivers 30% more sound to the brain and gives the brain more of what it needs to make better sense of sound.
Chatable The Chatable app provides you with a clear and loud speech signal without unwanted background noise, wherever you are. It enhances how you hear speech so that you can follow the conversations even in the noisiest situations. The Chatable app reduces your listening effort. It uses a proprietary artificial intelligence system called VOXimity. It makes the voice of the person talking to you clear and loud for focused, effortless hearing. [40]
Figure 33: Oticon Earbuds
Apple and Android Apple and Android have patented specific Bluetooth connectivity with hearing aids so that certain hearing aids can communicate directly with the operating platform.
5.2.5 Hearing loop
Figure 32: Chatable App Operation: This application is mainly used for people who have a hearing impairment and are not completely deaf. The application similarly uses the speech recognition system to filter out any external surrounding sound and makes the voice of the person talking to the user loud and clear. This helps the user to listen the sentence in a more focused manner. In the case of train station, the application can be used in conjunction with portable translators to guarantee the translation rate.
5.2.4 Bluetooth Hearing Aids While the hearing aids allow the user to perceive the surroundings and communicate in a normal sense, it also leads to the inconvenience of having to switch to earbuds if the user wants to listen to music. The user has to remove the hearing aids in order to accommodate a pair
19
La Trobe University
The current standard in public transport; in an effort to assist travellers with hearing aid, a type of technology known as a hearing loop (audio induction loop) is used. The hearing loop is a sound system that provides a magnetic, wireless signal that is picked up by the hearing aid when it is set to ‘T’ (Telecoil) setting. [41] The hearing loop uses a microphone to pick up the spoken word, which travels through an amplifier for the signal and finally to an antenna where the message is broadcasted as a magnetic signal for the hearing aid to receive.
Assistive Technologies and Public Transport
Figure 35: Speech generating application Figure 34: Hearing Loop
GoogleTTS (Application)
Operation: The hearing loop should be placed in all of the assisting counters in order to provide sufficient help for all travellers with hearing impairment.
Speech generating devices are hand-held electronic devices that play pre-recorded words or phrases when the user touches a switch or key. As the technology advances, the device can also play phrases that are typed in real time using text-to-speech function. A family member or friend can observe the situation and use their own iPhone to transfer the message if necessary.
5.3
Example products (Cognitive Disability)
5.3.1 Speech Generating Device Speech generating devices and applications have become more popular augmentative and alternative communication option for nonverbal or minimally verbal children and adults. The assistive communication device will allow users who have speech impairment to type out their desired question or speech. The information can also be relayed online to or from the user, allowing the transport officer to communicate with a non-verbal user via the device.
Twinkl This application intends to allow the user to learn and ask for objects and showing interest in things. This involves learning to activate symbols on the device screen to indicate a desire or interest as shown in Figure 35. This application acts as a speech generating device and allows the user to ask for objects via clicking on symbols on the device screen to indicate a desire or interest. In the case of a train station, it can be placed in conjunction with smart devices placed around the station and allow users to click on the “train” icon to get access to information.
20
La Trobe University
The assistive communication device will allow users who have speech impairment to type out their desired question or speech. The information can also be relayed online to or from the user, allowing the transport officer to communicate with a non-verbal user via the device.
VocaliD While text to speech technology has been widely used in the current society, it may sound generic to users which leads to discomfort. VocaliD focuses on the fact that those who have severe speech impairments also produce distinctive sounds when crying, laughing and vocalising emotions. By blending the sounds of the recipient with the speech of a matched voice donor, the company can create a unique voice similar to that of the user. The main idea of VocaliD is to produce a more realistic speech sound to the users. This can be adapted to all sorts of applications that include the text-to-speech service. This enhances the quality of the conversation by removing the robotic machines sound and allows the users to be in a more environmentally realistic position. [42] Operation: The assistive communication device will allow users who have speech impairment to type out their desired question or speech. The information can also be relayed online to or from the user, allowing the transport officer to communicate with a non-verbal user via the device.
Assistive Technologies and Public Transport
5.3.2 Education Robot Children experiencing autism or developmental disabilities may experience a harder time to learn and absorb knowledge. The aim of an education robot is to use coloured lights, music, and entertaining methods to educate the children.
Leka Robot Leka Robot is fully customisable and connects to an app that allows you to interact with Leka. It has been aimed at schools that focus on autistic children and those with similar conditions. The robot lights up with colourful LED lights, plays music and vibrates as it is played with. It also contains educational games for the children to learn as presented in Figure 36.
Figure 36: Leka Robot
21
La Trobe University
Assistive Technologies and Public Transport
6 Interaction with the Internet for the Blind, People with a Physical Disability and Cognitive Disability The following section assesses how people with disabilities, including people with visual impairment, physical disability, and cognitive disability, interact with the internet. Examples include alternative input devices which allow individuals to control their computers through means other than a standard keyboard or pointing device. Examples include: Alternative keyboards—featuring larger- or smaller-thanstandard keys or keyboards, alternative key configurations, and keyboards for use with one hand. Electronic pointing devices—used to control the cursor on the screen without use of hands. Devices used include ultrasound, infrared beams, eye movements, nerve signals, or brain waves. Sip-and-puff systems—activated by inhaling or exhaling.
Operation: A screen reader application known as JAWS (Job Access with Speech) reads the information from the smartphone screen and reads the content of the page to the users. The application can also print out braille documents and uses a speech recognition system to fill out webforms. In the case of public transport, this is extremely useful for the users when filling out any application form. It also allows the user to surf the internet while blind. [43]
Jaws JAWS provides speech and Braille output for the most popular computer applications on your PC. You will be able to navigate the internet, write a document, read an email and create presentations from your office, remote desktop, or from home. The JAWS screen reader is currently used by ACT Government in translating Network Map for public transport.
Wands and sticks—worn on the head, held in the mouth or strapped to the chin and used to press keys on the keyboard Joysticks—manipulated by hand, feet, chin, etc. and used to control the cursor on screen. Trackballs—movable balls on top of a base that can be used to move the cursor on screen. Touch screens—allow direct selection or activation of the computer by touching the screen, making it easier to select an option directly rather than through a mouse movement or keyboard. Touch screens are either built into the computer monitor or can be added onto a computer monitor.
6.1
Example products (Blindless)
6.1.1 Screen Reader A screen reader allows visually impaired or blind people to read websites by firstly scanning the website and the content of the page. It then reads the content out loud to the person. There are several other benefits with the adaptation of screen readers. This includes reading documents and emails, use speech to fill put webforms and surf the internet with web browsing keystrokes.
22
La Trobe University
Figure 37: Jaws Screen Reader
OrCam My eye 2 Refer to Section 5.2.1
Google Chrome ChromeVox ChromeVox is a built-in screen reader for chrome. ChromeVox is an extension for Chrome on Windows and Mac OS X which operates as an alternative screen reader for Web content. ChromeVox speaks the content of the page, plays audio indicators for page load progress and
Assistive Technologies and Public Transport
objects on the page, and provides a way to navigate all web content from the keyboard. [44]
6.1.2 Dictation Dictation software combines the effectiveness of speechto-text, voice recognition, voice-to-text and speech recognition which allow the user to interact with the internet by simply using his voice. ▪
▪
Users can ask the speaker’s built-in voice assistant, Alexa, to ring people, send messages, update their calendars, create to-do lists, set alarms and timers, play music, provide news and weather updates, buy items online, manage their smartphone devices, and lots more. The google assistant provides a speech to text service and allows the user to input any questions towards the google database. The database then provides the best solution and utilises the text-tospeech function in order to relay the information back to the user. Similarly, this technology does not have a braille output.
6.1.3 Gesture Recognition The aim of gesture recognition is to identify and interpret human gestures using mathematical algorithms. This can include head, eyes or hand movements as displayed in Figure 38. It allows the user to use simple gestures to control and interact with devices without physically touching them. All of the software surround with the use of cameras and computer vision algorithms to interpret sign languages.
GestureTek GestureTek has developed advanced tracking and gesture recognition algorithms to define the relationship between computers and the people using them. With 3D cameras and our patented 3D computer vision software, computers can now identify, track, and respond to fingers, hands or full-body gestures. The multi-tracker detects multiple data points on the user's body (head, torso, hands) enabling the creation of immersive full-body representations and aspects of avatar control. The tracker can recognise circle and swipe gestures and can support activities such as scrolling, enlarging, shrinking, or rotating items.
Figure 38: Gesture Recognition Software
Gesture Sign Gesture Sign is a gesture recognition software for Windows. You can automate repetitive tasks by simply drawing a gesture with your fingers, pen or mouse. Similar to the software above, it is able to follow simple commands such as activating window, touch keyboard control, mouse simulations and etc.
GestIA The developer developed an interface that allows the user to link each hand gesture to a specific key. That is, for example, the user can associate a thumbs up gesture to the key the user desires: it can be the “a” letter on the keyboard, the “q” or maybe the “enter”. The developer coded a Python script which will mimic the input of pressing the selected keys based on the identified hand gesture.
Figure 39: GestIA
6.1.4 Braille Display For users who are deaf and blind at the same time, this technology provides a solution for them to interact with the internet. The Braille device is designed for users who are expert in Braille display and allows them to surf the internet without any significant issue.
23
La Trobe University
Assistive Technologies and Public Transport
Portable EIBraille As shown in Figure 40, the ElBraille is a portable device designed for users who are blind or deaf-blind and want to stay connected at school, work, or home as well as on the go. This take anywhere replacement for the dedicated notetaker, ElBraille combines the flexibility of a modern mainstream computer running the Windows 10 operating system with JAWS® and a 40-cell Braille display with a full Braille keyboard to offer a fully accessible compact solution. Students and professionals alike will appreciate being able to run any Windows application such as Microsoft Word, Excel, PowerPoint, or Outlook. [45]
with real time information regarding what is the current time.
Blitab Blitab is the first tactile tablet with a textured surface to help with reading and writing Braille. It’s essentially an ebook, which, instead of a screen, uses small physical bubbles to represent letters and words. It can deliver realtime information to people in Braille form from around the web, but can also be used to read files, such as those from Word or PDF. Operation: For users who are both deaf and blind at the same time, this technology provides a solution for them to interact with the internet. This type of assistive technology is designed for users who are expert in Braille display and allows them to surf the internet without any significant issue. This can be placed around the train station for the user’s comfort.
6.2
Example products (Deaf)
6.2.1 Caption/Subtitles Figure 40: Portable ElBraille 40
TalkBack Braille Keyboard The keyboard, known as TalkBack, uses a six-key layout. Each key represents one of the six Braille dots, which then form letters and symbols when tapped in combination. The Braille Keyboard provides a way for the blind users to utilise a keyboard when operating a computer. This technology can be used in conjunction with the text to speech application for the best effect. [46]
Braille Watch The Dot Watch allows iPhone and Android smartphone users to receive and view notifications on the watch in Braille. The Dot Watch consists of a simple design. On the face of the watch is a Braille display which has four cells, and below that are two touch sensors that allow users to scroll through and read each notification. The right side of the watch has three controls: the “Select” button, the “Dot Crown” dial and the “Home” button. These controls are used to operate the watch’s different functions. The watch is charged via a USB charger which is included with the device. Once paired, users can begin receiving notifications from their phone, tell the time and date, and use other features. [47] The watch works in a similar way to that of a portable notetaker where the information is relayed through a braille output. In the case of a train station, this can be placed alongside the smart device and provide the user
24
La Trobe University
The auto caption functions use an AI machine learning function to capture the audio of the video. The audio then runs through the speech to text function in order to generate the text as a subtitle. In the case of a train station, this can be used within the smart device display when the user is watching a video.
Earfy The application known as Earfy provides a real time captions for one-on-one conversations with forty languages available. It also provides optimisable colours, fonts and positioning.
YouTube Caption YouTube provides an Auto-Captioning process via transferring real time speech to captions as displayed in Figure 41. It also provides auto-synchronisation function in order to translate speech and transcript to captions, which provides a much more accurate caption. The automatic captions are generated by machine learning algorithms. Therefore, the captions are not 100% correct. The YouTube community also encourages the content creators to add professional captions for the video.
Assistive Technologies and Public Transport
Figure 41: YouTube Screen Reader
Facebook Caption This tool is currently only available in the US English for advertises in US and Canada region. It is extremely popular for users who frequently upload videos in this language to their profile. Similar to that of YouTube, the Facebook’s automatic caption feature is not 100% reliable.
VEED.IO VEED.IO is a simple website that helps you edit your own video and download it with no account required. One of the most recent features they have added is the ability to automatically generate subtitles and captions using AI. The software takes the audio from the video file and generates text and time codes. This allows Deaf people to interact with the user’s content.
6.3
Example products (Cognitive Disability)
6.3.1 Word Prediction Word prediction software offers user who have difficulty finding, choosing, or spelling the word that expresses their thoughts. By suggesting different words for the user, as seen in Figure 42, it can help for users who have learning and thinking differences. This computer software utilises the database dictionary to predict, choose and find words that may express the user’s thoughts by suggesting different words for users. [48] This allows users with cognitive disability to interact with the internet or relay messages in a more professional sense. ▪ ▪ ▪
Cowriter Word Q Read and Write Gold
Figure 42: Microsoft Word Prediction Software
6.3.2 Simplified Word Processors Any software that includes simplified word processor shows simplified words or synonyms of words. It also includes spelling and grammar checker functions to assist the users. ▪ ▪ ▪
Microsoft Word KWord WordPad
6.3.3 Simplified Smart Device The aim of simplified smart device is to offer a featurepacked device for users who want to use some of the modern features of a smart device but are not comfortable using a touchscreen or remote. Operation: The phone runs on a special version of android that aims to provide a simple and consistent interface for the users with cognitive disability. This allows the user to surf the internet without any complex interaction. In the case of a train station, this can be adapted as the new version of smart device so that all users can interact with the device easily.
BlindShell Classic This phone features a standard numeric keypad along with a number of additional buttons for routing and responding and ending calls. Features include a phone dialler, contacts, email support, an FM radio, voice recorder, book player, and a colour identifier. While it runs a version of Android, this is not apparent to the user and it is not possible to access other Android features beyond those included in the custom software. [49]
VoxiTV A set-top box and software package offering a basic interface for accessing dozens of entertainment and
25
La Trobe University
Assistive Technologies and Public Transport
information resources. Services like Spotify and YouTube can be accessed using a simplified menu interface from a remote control. The remote includes standard navigation controls as well as media buttons for adjusting playback and volume.
KOMP KOMP represents a simplified computer interface for both children and elders. The screen has high contrast and large text, making the content easy to see. As displayed in Figure 43, the KOMP has no touch screen and is operated with only one button. The sound is loud and clear, and seeing the person you talk with makes it easier to understand what they say. KOMP is easy to operate through a big, accessible button, with no complicated interactions to learn. [50]
6.3.3 Oversized Trackballs Ablenet Ablenet recently released a trackball mouse for little hands. The friendly looking large trackball can assist all children interacting with the internet. It is important to note that the giant yellow track ball in the middle can be moved with an elbow or a foot. It also provides an unique feature known as drag lock. It allows the user to move an object to another position within he screen without holding the mouse-click. The aim of the oversized tracball released by Ablent is to provide a versatile solution for cognitive disabled children who wants to interact with the internet.
Figure 44: Bigtrack Tackball Figure 43: KMOP Simplified Computer
Sesame Phone The Sesame Phone is a touch-free mobile that you control by moving your head. It is based on the Google Nexus 5, exploiting the Nexus’s ready-made accessibility features. The Sesame Phone is switched on by speaking the phrase “Open Sesame” into the mouthpiece. Once activated, the front-facing camera tracks the position of your head. A cursor will appear on the screen, allowing you to make calls, send texts and emails, use apps, and delve into your phone. Turning your head left will send the cursor leftwards, turning it to the right will send it to the right, with nodding up or down completing the simple navigation system.
Branto Orb (Universal Controller) Essentially a universal remote control, the Branto Orb hub connects wirelessly to compatible devices so you can change the channels on your telly, stream music and audiobooks, switch lights on and off, and even tweak the thermostat
26
La Trobe University
Assistive Technologies and Public Transport
7 Payment Methods for Deaf, Blind, and Cognitive Disabilities 7.1.1 Closed-Loop VS Open-Loop System In our current society, more and more people have begun using mobile payments, with a 41 percent increase in usage from 2018 to 2019 in United States alone. The open-loop payment system allows users to use one digital wallet on their mobile phone to pay at any given location that provides electronic transactions. In contrast, the closed-loop payment system requires the user to prepay a certain amount of money and pay at just one particular merchant. This system, for example, can include loyalty card or a gift card. Most of the large companies around Australia such as Event Cinema and Westfield offers closed-loop mobile wallets and gift cards. This allows the company to have the upfront cash flow. Similarly, the merchants can offer a percentage bonus on a top-up amount. This will benefit both the merchant and the customers. Furthermore, the closed-loop payment system also allows the company to monitor the customers’ spending and observe which products are the most popular. [51]
7.1.2 Simplified Smart Device (OpenLoop System) Online wallet serves as an alternate payment method technology for the visually impaired, as they make it much easier for customers to shop safely and independently. These platforms make it easier for visually impaired customers to find the card they wish to use as everything is in one place as shown in Figure 45. Screen reading technology can also help the customer identify their chosen card, as well as allow them to verify the price of an item and give access to their purchase history in order for the user to independently keep track of their finances. The tap to pay function is also much easier than having to use chip and pin for payments over certain amount. ▪ ▪
Google Wallet Apple Pay
Figure 45: Online Wallet
7.1.3 Voice Response System (OpenLoop System) Phone banking provides voice instructions and input, via the user’s phone keypad, whereas over the counter services frequently require dealing with or working around the limitations of paper forms and written signatures. Furthermore, automatic tellers have a range of access barriers including a mainly visual interface. Using a voice response system, the vision impaired can simply listen to the requirement and act upon it using own phone keypad. [52] ▪ ▪ ▪ ▪
Commonwealth Bank Australia and New Zealand Banking Group National Australian Bank Westpac Bank
7.1.4 Electronic Tag (Closed-Loop System) The e-TAG electronic tag, as displayed in Figure 46, is based on radiofrequency identification (RFID) transponders using the DSRC protocol. The system uses electronic transponders installed on the inside of the vehicles' windscreen. The gateway picks up the signal emitted from the e-Tag and record the entry of the vehicle. It then records the location where the vehicle exits the toll road and sums up the total fee. Similarly, this principal can work for travellers within a public transport station. The traveller can travel along with a tag and walk across the gateway. This will allow blind, users with a physically or cognitive disability to pay with less issue.
Figure 46: E-toll Tag
27
La Trobe University
Assistive Technologies and Public Transport
7.1.5 Auto-renewing Subscription
7.1.6 Device-initiated Payment
The auto-renewing subscription technology aims to minimize the time required for the user to renew a contract in order to be given access to content, services, or premium features in his own personal application in an ongoing basis. [53]
Device-initial payment is another method for conducting a transaction while minimizing the movement required for the user. The payment method involves the use of a static virtual payment account number (VPAN), an expiry date and a dynamic code generated by mobile device for further payment request. [54]
This technology makes the life easier for both the user and the providers in general terms. Instead of requiring the user to manually pay for an invoice each month, the user can store credit card details into the account with agreement to have that payment method charged for the recurring amount. The subscription type payment has been increasing popular in application service such as: •
Netflix
•
Disney+
•
Amazon Subscribe
Figure 47: Auto-Renew Subscription
28
La Trobe University
For example, if the customer is refuelling in a gas station, all the customer has to do is to unlock the chosen pump with the mobile app and remotely pay for the fuel in the car right after finish refuelling. While this current service is rara at sight, some companies has adapted this technology: •
Messina
Assistive Technologies and Public Transport
8 Next steps: Public Transport can benefit – start thinking about standards As shown in the examples within the report, assistive technologies offer significant opportunities to enhance the experience for public transport users with a disability. Whether it is wayfinding, HMI, frictionless payments, smart vision, or smart wheel-chairs, a national, non-siloed approach would be required to capture the requirements for all devices. This could be achieved by an analysis and cross comparison of all the examples mentioned in these report, identifying the commonalities between them, conduct a priority list and work with the industry to prioritize. For instance, beacons in vehicles, premises and could connect to a variety of assistive apps and devices which should consequently share the same wireless technology, using the same protocols and encryption. By way of sample, below we have indicated a few examples (Bluetooth, Wi-Fi, etc) in which technologies require standardization to offer:
29
•
Universal accessibility
•
Seamless interoperability
La Trobe University
Assistive Technologies and Public Transport
8.1
Common Standardised Aspect (Electric Wheelchair)
Universal Tie Down Automatic: o Follow Wheelchair Securement Standards (WC19) provided by PESNA and ISO [19] ▪ Four accessible, anchored, and labelled securement points with specific closed-loop geometry to allow one-hand attachment of tie-down strap hooks. They must be able to withstand the forces of a 30-mph, 20-g frontal impact. ▪ A commercially available wheelchair-anchored lap belt. This belt must include a pin-bushing anchorage on each half of the lap belt for attaching the lower end of a shoulder belt near the passenger’s hip to comprise a three-point belt restraint system. ▪ Crash testing compliance with the wheelchair-anchored lap belt placed around the pelvis of the appropriate-size crash-test dummy. Two ratings of the wheelchair’s accommodation of vehicle-anchored lap/shoulder belt restraints: one for ease of proper occupant restraint positioning, and a second for the degree to which proper belt placement is achieved. Wireless Transmitter and Receiver: o Require transmitter and receiver that is compatible with MU-MIMO connections to allow a larger capacity of radio link through multiple transmission and receiving antennas. [20] o Standard wireless access point (WAP) is necessary for other Wi-Fi devices to connect to the provided wired network. This should also be compatible with Wi-Fi 6. Wi-Fi Standards [21] o Wi-Fi 6 / IEEE 802.11ax-2021 o The Wi-Fi 6 is released based on the reference provided in the IEEE 802.11 standards. o Compatible with both 2.4 GHz and 5 GHz. o Wi-Fi 6 enhances throughput-per-area in high-density scenarios such as transport stations. Wi-Fi Protocol o Adaptation of Wired Equivalent Privacy (WEP) which is a security algorithm for IEEE 802.11 wireless networks. [22] o Aim to provide data confidentiality comparable to that of a traditional wired network. Wifi Security Method o Use router encryption to secure all information with authentication protocol. This usually requires a o network key or password. o Use Wi-Fi Protected Access version 3 (WPA3) as the standard security protocol. It helps protect against dictionary attacks and requires simultaneous authentication. In use of public area, it automatically encrypts the connection without any need for additional credentials. [23] Bluetooth Transceiver and antenna o The antenna function can be located in most Wi-Fi Hardware o Transceiver ▪ Range of around 10 meters ▪ Adapt Bluetooth 5 Module which provides a linger range and faster speech in comparison to older versions. [25] Bluetooth frequency o Ensure the Bluetooth 5 Module uses between 2.402 GHz and 2.480 GHZ frequency as the standard wireless communication protocol. [26] o Compatible with 5G in the future.
Bluetooth Protocol [27] o Core Protocols – Bluetooth radio, baseband, Link manager protocol, logical link control and adaptation protocol and service discovery protocol o Cable Replacement protocol – Provides serial interface with WAP o Adopted protocols – Point-to-Point protocol, internet protocol, user datagram Protocol, transmission control protocol and wireless application protocol o AT Commands – Attention Command set Bluetooth Security [28] o Bluetooth 5.0 Encryption – AES-CCM Encryption o LE Security Mode 1 : Authenticated LE Secure Connections pairing with encryption using a 128 bit strength encryption key o Elliptic Curve Diffie-Helman cryptography is used for key exchange Location Data Standards o Universal Location Standard (GPS) [29] ▪ Uses phone data to triangulate the user’s location and what time it is ▪ Require four satellites to provide accurate positioning ▪ Data governed under NMEA 0183 protocol ▪ Require OEM GPS receiver module from personal electronic device to receive data Universal Time Reference o All provided time information should be reference towards AEST time zone Language Standards o More than 21% of the Australians speak a language other than English in their home. o The most common language spoken is English, Mandarin, Arabic, Cantonese, and Vietnamese. [24]
30
La Trobe University
Assistive Technologies and Public Transport
Summary and Conclusion
In this report we have shown a large number of smart assistive technologies that are either in the market or in development, particularly those we believe will improve access to Public Transport The report focuses on assistive technology that intends to support four different aspects which are 1 2 3 4
Wayfinding Personal mobility aids Communication Human Machine Interaction
Significant market The statistics (from ABS 2015) show that the size of the population that use aids or equipment is significant at 2.2 million
Impact Once mature and sufficiently integrated, access to new assistive technology will undoubtedly change people with disabilities’ life, and that will occur in in the near future. For instance, access to increasingly advanced electric wheelchairs that help with mobility and public transport access could mean that more wheelchair users will use public transport more often to a greater variety of destinations. Similarly, access to vision assisting device such as 3D vision goggles and braille map will allow blind persons to travel around the station safely and confidently or on the public transport. As the provided assistive technology is standardised for all travellers, the entire population can benefit from the system, reducing their cost in being financially penalized for requiring specialized devices during travel and ultimately, allow them to safely travel to destination while minimizing chances of accidents. Timing and adoption factors The speed with which assistive technologies will emerge is hard to predict and depends on several connected factors, which include for instance: • • •
•
Maturity The product pricing The range of emerging technologies that can be linked towards assistive and therefore be adapted towards the public transport system. The acceptance and adoption of new assistive technology based on human rights and related assistive technology policies and legislation.
Gartner has a few recommendations for inclusive design:
31
La Trobe University
Assistive Technologies and Public Transport
devices is to alleviate the disability itself, e.g., a wheel chair provides mobility, but for the connectiveness and smart aspects of these devices we believe that Public Transport is the ‘killer app’. Next steps Public Transport can provide a rich area of applications if it is offers connectivity and data in a standard manner to the devises so that there is: Universal accessibility Figure 48Gartner’s Recommendations: Inclusive Design, Autonomous Things, Augmented Reality Accelerating adoption The real question is how to combine all the possibilities and adapt them towards the public transport with universal accessibility and seamless interoperability in mind. For example, the connected wheelchair would also have a universal Wi-Fi and Bluetooth receiver to ensure the quality of the connection. As the principle of this project is to allow everyone to travel safely and confidently, we advocate for a universal system approach that allows all travellers to make use of the provided assistive technology without concern. This report is intended to start the effort – an initiative from the Centre for Technology Infusion, as our mission is to accelerate the adoption of emerging technologies for good. Emerging technologies are in many cases looking for their so called ‘killer app’. While the first task of many of
32
La Trobe University
Seamless interoperability This can only be achieved from a top down, collaborative approach between OEM manufacturers, operators, road managers and government.
One necessary step is to seek common standards: an analysis and cross comparison of all the examples mentioned in this report, identifying the commonalities between them, conduct a priority list and work with the industry to prioritize. Technology is not the be all end all Finally, it is worth noting that there is much work to be done today to improve accessibility for people with disability which is not of a technical nature, such as consistently providing ramps to trams, providing consistent guidance and rails, consistent notifications, etc. etc.. But, if the work on standardization of these technologies starts now – hopefully the physical world improvements and opportunities digital assistance provides will converge.
Assistive Technologies and Public Transport
Table of Products
33
La Trobe University
Assistive Technologies and Public Transport
List of Figures Figure 1:Genny Electric Wheelchair............................................................................................................................ 5 Figure 2: Moby ............................................................................................................................................................. 5 Figure 3: Phoenix Wheelchair ..................................................................................................................................... 5 Figure 4: Standing Electric Wheelchair ...................................................................................................................... 6 Figure 5: QOLO Standing Wheelchair ......................................................................................................................... 6 Figure 6: Domino People Wheelchair ......................................................................................................................... 6 Figure 7: Scewo Electric Wheelchair .......................................................................................................................... 7 Figure 8: Mobile Stairlift .............................................................................................................................................. 7 Figure 9: iBot................................................................................................................................................................ 7 Figure 10: Garaventa Stair-Trac Wheelchair .............................................................................................................. 7 Figure 11: Lower Body Exoskeleton ........................................................................................................................... 9 Figure 12: Standing Desk ............................................................................................................................................ 9 Figure 13: Human Support Robot ............................................................................................................................. 10 Figure 14: Munevo Drive ........................................................................................................................................... 10 Figure 15: Google Map .............................................................................................................................................. 10 Figure 16: Walking Cane with seat attached ........................................................................................................... 11 Figure 17: Portable gyroscopic-assisted system .................................................................................................... 11 Figure 18: Sip and Puff wheelchair .......................................................................................................................... 11 Figure 19: Talking Tactile Map System .................................................................................................................... 12 Figure 20: RFID System ............................................................................................................................................. 12 Figure 21: Video Streaming ...................................................................................................................................... 13 Figure 22: Stereo Vision ............................................................................................................................................ 13 Figure 23: NaviLens .................................................................................................................................................. 14 Figure 24: Ultrasonic Distance ................................................................................................................................. 15 Figure 25: Visual Image Processing ........................................................................................................................ 15 Figure 26: AI suitcase Demo..................................................................................................................................... 16 Figure 27: Antenna embedded walking cane .......................................................................................................... 16 Figure 28: KNFB Screen reader ................................................................................................................................ 17 Figure 29: Google Dictation Software ...................................................................................................................... 17 Figure 30: VP Columbia ............................................................................................................................................ 18 Figure 31: Ava Portable Translator .......................................................................................................................... 18 Figure 32: Chatable App ........................................................................................................................................... 19 Figure 33: Oticon Earbuds ........................................................................................................................................ 19 Figure 34: Hearing Loop ........................................................................................................................................... 20 Figure 35: Speech generating application ............................................................................................................... 20
34
La Trobe University
Assistive Technologies and Public Transport
Figure 36: Leka Robot ............................................................................................................................................... 21 Figure 37: Jaws Screen Reader ................................................................................................................................ 22 Figure 38: Gesture Recognition Software ................................................................................................................ 23 Figure 39: GestIA ....................................................................................................................................................... 23 Figure 40: Portable ElBraille 40 ................................................................................................................................ 24 Figure 41: YouTube Screen Reader .......................................................................................................................... 25 Figure 42: Microsoft Word Prediction Software ...................................................................................................... 25 Figure 43: KMOP Simplified Computer .................................................................................................................... 26 Figure 44: Bigtrack Tackball ..................................................................................................................................... 26 Figure 45: Online Wallet ............................................................................................................................................ 27 Figure 46: E-toll Tag .................................................................................................................................................. 27 Figure 47: Auto-Renew Subscription ........................................................................................................................ 28 Figure 48: Disabled Statistics from ABS 2015 ........................................................... Error! Bookmark not defined. Figure 49: Gartner’s Recommendations: Inclusive Design, Autonomous Things, Augmented Reality ........... Error! Bookmark not defined.
35
La Trobe University
Assistive Technologies and Public Transport
Abbreviations Abbreviation
Description
AIMES
Australian Integrated Multimodal EcoSystem
AJAX
Asynchronous JavaScript and XML
ANPR
Automatic Number Plate Recognition
API
Application Programming Interface
ASP.Net
Microsoft's Active Server Pages built on top of the .Net Framework
CSS
Cascade Style Sheets
CRUD
Create-Read-Update-Delete
CTI
Centre for Technology Infusion
DSRC
Dedicated Short Range Communication
DJCS
Department of Justice and Community Safety
DOM
Document Object Model
EF
Entity Framework [ORM]
EPC
Electronic Product Code
GPS
Global Positioning System
HTML
Hypertext Mark-up Language
HTTP
Hypertext Transfer Protocol
ID
Identification
IIS
Internet Information Services (Microsoft’s Web Server Program)
IP
Internet Protocol
JSON
JavaScript Object Notation
LINQ
Language-Integrated Query
MVC
Model-View-Controller [Design Pattern]
OBU
On-Board Unit
OQL
Object Query Language
ORM
Object-Relational Mapper
OWIN
Open Web Interface for .Net
PC
Personal Computer
RDBMS
Relational Database Management System
REST
Representational State Transfer
RFID
Radio Frequency Identification
RSU
Road-Side Unit
SQL
Structured Query Language
TCP TRL
Transmission Control Protocol Technology Readiness Level
T-SQL
Transact SQL (Microsoft’s Dialect of SQL)
UI
User Interface
UML
Unified Modelling Language
URI
Uniform Resource Identifier
URL
Uniform Resource Locator
UX
User Experience
UDP
User Datagram Protocol
VicPol
Victoria Police
VIN
Vehicle Identification Number
XML
Extensible Mark-up Language
36
La Trobe University
Assistive Technologies and Public Transport
37
La Trobe University