Emerging Technologies in HCI

Oct 25, 2017

Human Computer Interaction has changed dramatically in the last 43 years, since the introduction of the first Graphical User Interface for a computer released by Xerox in 1973 through to todays’ GUI’s on systems like Microsoft’s Windows 10, Apple’s OSX and Ubuntu’s Xenial Xerus. Introducing a GUI for computers meant that they were now accessible to, and useable by, regular people and not just educated computer programmers.

But Human Computer interaction has come even further in the las few years with the rise in popularity of such technologies as Augmented Reality, Virtual Reality, Motion sensing technologies, Geolocation Technologies and Eye tracking technologies to name a few.

These technologies have advanced to the point where they have opened up a whole new world of possibilities, not just for the businesswoman viewing a new building concept via AR or the teenager playing VR games but also to provide assistive technologies for those with disabilities or special needs, providing a better quality of life.

Augmented Reality

Augmented Reality, or AR, which mixes the real world with the digital world, allowing a user to view digital content overlaid on real world objects. For example, this year Niantic released their AR game Pokémon GO which allows players to walk around different locations to capture hidden Pokémon creatures. They use a smartphone as a viewport and when in the correct Geo-location, a digital monster appears on the screen and the player uses swiping motions on the screen to interact with the game. Augmented Reality applications are readily available on most smart phones and tablets as well as on games consoles and computers.

This technology can be used to assist adults and children with autism.

Playing pretend as a child—whether using a paintbrush as a wand or imagining a large cardboard box to be a castle—is more than just for fun. It is also an essential developmental activity that teaches children social and emotional skills and builds their self-esteem. However, most children with autism—a neurodevelopmental condition that affects the ability to communicate and interact with others—are less engaged in imaginative play. And this can have a profound impact on them into their adult lives. Rozenfeld, M. (2016)

Ph.D. candidate Zhen Bai used Augmented Reality technology to create an application where a child could see themselves on screen via a camera. They were given blocks with markers on them to play with, while the blocks were simply blocks when the child looked at the screen they would take the appearance of cars, buildings and other objects. Using the AR tracking capabilities the objects would mimic the movements made by the child, for example if the child moved a block around that was a car marker then on the screen they would see the car being moved around. Using this system Zhen Bai was able to confirm that AR could engage children with autism in imaginary activities.

Virtual Reality

Virtual Reality, or VR, is reality emulation. Using a headset as a viewport, the user can enter a virtual world that they can interact with. If the user looks to the left the view port moves in a real-world manner, tracking the movement and changing the view as if the user were actually in that reality.so if the user was in a virtual forest and looked up they would see the sky, if they looked down they would see the ground and so on. Some VR systems come with controllers that let the user move and interact with their hands in the virtual world, picking up and using different objects. Other systems utilise a motion platform that lets the use walk in place, kind of like a self-propelling treadmill, but their movement on the motion platform allows the user to walk around the virtual world. While other systems use motion tracking sensors to monitor user movement which lets them interact with the VR world.

VR is also used for motor rehabilitation, it offers the opportunity to bring the complexity of the physical world into the controlled environment of the laboratory, this allows for the creation of a synthetic environment with precise control over a large number of physical variables that influence behaviour while recording physiological and kinematic responses.

Virtual Reality (VR) provides a unique medium suited to the achievement of several requirements for effective rehabilitation intervention. Specifically, therapy can be provided within a functional, purposeful and motivating context. Many VR applications present opportunities for individuals to participate in experiences, which are engaging and rewarding. In addition to the value of the rehabilitation experience for the user, both therapists and users benefit from the ability to readily grade and document the therapeutic intervention using various systems. In VR, advanced technologies are used to produce simulated, interactive and multi-dimensional environments. Visual interfaces including desktop monitors and head-mounted displays (HMDs), haptic interfaces, and real-time motion tracking devices are used to create environments allowing users to interact with images and virtual objects in real-time through multiple sensory modalities. Opportunities for object manipulation and body movement through virtual space provide frameworks that, in varying degrees, are perceived as comparable to similar opportunities in the real world. Sveistrup, H. (2004)

Motion sensing technologies

Motion sensing technologies make use of cameras, infrared technology or accelerometers to track a user’s movements so that they can perform tasks in a virtual manner on screen. One example is the Leap Motion system, this small piece of hardware can be connected to a computer which then allows the user to interact with their hands on screen, picking up and moving virtual objects. Another example is the Nintendo Wii which uses a hand held controller to track the motion of the users hand so that they can play tennis or go ten pin bowling.

There are many different technologies that can be adapted to the needs of someone with dementia. Some pieces of assistive technology have been designed specifically for people with the condition but a lot of potentially helpful technology has not.

A simple but effective way of using motion sensing technology for a dementia suffer is to have the sensors positioned in critical area which activate pre-recorded messages to help remind the user to perform specific tasks. For example, a sensor placed near the front door could tell the user to lock the door, another placed in the kitchen could remind them to check the gas is turned off.

Motion sensors are also a less intrusive alternative to having cameras in the home. Using motion sensors in various areas of the home users can have their routine monitored, creating a pattern of daily behaviour. Monitors can see how long someone has spent in bed, how long they spend in a certain room or how long they sit in a specific place. If there is a big change in routine it could be an early indicator that something is wrong.

• Geolocation is used in a number of ways, social media apps and websites often tag photographs to say where they were taken, or add a location to a status update. Lots of smartphone applications use Geolocation technology, Foursquare used to mark you at a specific spot on the planet and let you check in to a location such as a restaurant, rewarding you with virtual stickers or real world discounts. The Starbucks app alerts you when you are close to a branch of their coffee houses, encouraging you to come in to their shop.

But as with the other technologies, geolocation software can also be used for medical purposes. If you live in a rural area it can be used to track your health should you require ir

Current Assistive Technologies

There are currently a lot of excellent assistive technologies available for a wide variety of uses. People who have cognitive impairments may have problems with perception, understanding speech (receptive aphasia) or with producing written or spoken word. Others may suffer processing issues relating to attention or memory, making remembering and performing specific task extremely difficult.

Assistive technologies for cognitive impairments may include

• Wireless devices such as PDA’s with verbal prompting
• Portable devices that provide auditory step-by-step instructions
• PDA’s that use pictures and audio recordings.
• Medication reminder devices

Motion sensing technology can be used in a number of ways to assist people with visual impairments, there is a system called no Eyes Yoga, which allows the user to learn yoga with the assistance of motion capture camera from an Xbox 360 games console. The camera uses mathematic equations to judge the angle of the users limbs, torso and head and speaks to them on how to adjust their body to reach the correct pose.

Assistive technologies are there to help improve the quality of life for people things like VR, AR and Motion Sensing Technologies are now at a point where they are affordable to the general public.

REFERENCES

Lineback, N. (2016) Graphical User Interface Timeline [online]. Available from <http://toastytech.com/guis/guitimeline.html> [21st October 2016]

onvert.com. (2016) onvert.com Guide to Augmented Reality [online]. Available from <http://onvert.com/guides/what-is-augmented-reality> [21st October 2016]

Rozenfeld, M. (2016) Augmented Reality Can Help Children With Autism Tap Into Their Imaginations [online]. Available from <http://theinstitute.ieee.org/technology-topics/con...> [21st October 2016]

Sveistrup, H. (2004) 'Motor rehabilitation using virtual reality'. Journal of NeuroEngineering and Rehabilitation 1 (1), 1

Alzheimer's Society. (2015) Assistive technology - devices to help with everyday living [online]. Available from <http://theinstitute.ieee.org/technology-topics/con...> [21st October 2016]