The future world of work with the University of Auckland and UniServices

CARES: Robotics in Agriculture and Horticulture Centre for Automation and Robotic Engineering Science aims to inspire and create innovative robotic technologies that improve societal wellbeing.

New Zealand leads the world in primary production in particular horticulture and agriculture.

Multipurpose Orchard Robotics Over the last four years, our collaborative project with Plant and Food Research, University of Waikato and Robotics Plus, delivered research, developing an Autonomous Multipurpose Mobile Platform capable of navigating autonomously in orchards with vision sensing of flowers and fruit for kiwifruit and apples in orchards, arms and grippers for harvesting kiwifruit and apples, and precision targeted spraying of pollen on flowers. This research was the largest real world trial of a robotic harvester and pollinator.

Decision automation for orchards and vineyards Our research has expanded into a new and larger collaborative programme of work. In collaboration with Universities of Canterbury, Waikato, Otago, Plant and Food Research, Lincoln Agritech, and co-designed with Maori and industry, we will create new technologies that capture and convert data about plant structure, and the activities of expert humans, into smart decisions and actions in fruit growing. We will use Artificial Intelligence to interpret data and then identify and communicate actions to less experienced humans and to automated robots.

Horticulture industries suffer from high labour costs and increasing shortages of skilled labour, which reduce productivity and quality of crops. Our new technologies will assist fruit growing operations, and help NZ companies to generate premium horticultural products for the global market.

Dr. Henry Williams Professor Bruce MacDonald Professor Stuart Bradly Associate Professor Patrice Delmas Dr. Ho Seok Ahn Dr. Trevor Gee

Artisan to Automation: Robotics and platform economies for the future of manufacturing The 21st Century has been defined by the emergence of ‘digital platforms.’

Amazon, Uber and AirBnB have radically transformed sectors with digital platforms that manage the distribution of things between service providers and users. We envision a future for manufacturing where digital platforms have transformed the industry. We want to retain and grow skilled craftspeople, as well as reduce repetitive, dangerous and low-skilled/zerohour contract work through smart automation.

This project is a technology demonstrator that creates and tests a digital platform for automated design and assembly of an ‘igloo.’ When finalised, design data is automatically created for fabrication of the panels from which the igloo is constructed. To enable the assembly of building structures larger than the robot’s reach, a novel methodology has been developed enabling a robot to be shifted to new arbitrary locations, then self-relocate and continue assembly. The platform also automatically programmes an industrial robot to pick up and position these panels for precision assembly. There are numerous benefits: more time to focus on the design, refining and engineering; more precise assembly; less time on fabrication and assembly; less heavy lifting. In parallel we are using Augmented Reality to overlay structural data about each panel on the panel. This gives the worker more information at the point at which it is needed, reducing potential errors or mistakes.

Digital platforms are radically redistributing capital, value and power in these new platform economies. This ongoing research investigates what a sustainable platform economy for NZ design and manufacturing could be.

Associate Professor Mark Battley (Faculty of Engineering) Dr. Dermott McMeel (Faculty of Creative Arts and industry)

High-involvement models of working: a pathway to better jobs and performance High-involvement models of working increase worker influence over the work process through higher levels of job autonomy or greater involvement in work-related decisions. They aim to improve the quality of jobs and workplace performance through greater employee empowerment. Our studies show that genuine improvements in empowerment make work more motivating, more satisfying, and more supportive of work-life balance. Greater empowerment fosters skill utilisation and learning.

NZ workers whose skills are better utilised report higher income, better career prospects and lower intentions to quit.

Greater empowerment is not a best-practice recipe but a principle that should be adapted to specific contexts. For example, in lean production, it is typically applied in group-based problem solving rather than in individual jobs. In other situations, there is more scope for individual ‘job crafting’. But there is a dark side: when work intensification leads to role overload, it puts employee well-being at risk. Managing work processes to enhance employee involvement, while minimising the risks of work intensification, is an ongoing challenge.

How well is NZ doing in high-involvement working? We show that our workforce responds very positively to it and the OECD reports that we are the best of the Anglophone countries in terms of high-involvement work practices. Globally, only Finland, Denmark and Sweden are ahead of us.

Professor Peter Boxall, University of Auckland Business School Dr. Gayani Hewagama, NorthTec Dr. Meng-Long Huo, Monash University, Malaysia Dr. Deepika Jindal, University of Auckland Business School Dr. Keith Macky, Whitireia New Zealand Professor Jonathan Winterton, Taylor’s University, Malaysia

Embracing the future of manufacturing The Laboratory for Industry 4.0 Smart Manufacturing Systems (LISMS) at the University of Auckland aims to develop smart manufacturing technologies for businesses in New Zealand.

LISMS is made up of researchers from a wide range of disciplines and backgrounds, who are passionate about empowering engineers to adapt to the ever-changing technological landscape via developing and deploying creative manufacturing tools and applications.

LISMS aims to address three critical research and social questions: 1. How will future manufacturing look like?

2. What are key enabling technologies for the Factories of the Future?

3. How can New Zealand manufacturing companies stay competitive in the Industry 4.0 context?

LISMS researches on: 1. Defining and classifying engineering professionals’ role in the future of manufacturing 2. Developing an integrated framework for technology selection and implementation of Industry 4.0 3. Developing a Digital Twin framework for smart manufacturing applications 4. Exploring human-centered autonomous industrial robotics and human-machine collaboration framework 5. Utilizing Augmented Reality (AR) to assist shop-floor workers on machining The team believes that the future manufacturing landscape will be a fusion of creative workforce and intelligent machines and factories that creates high valueadded products and services. Humans will continue being placed in the centre of the industry and society, surrounded by empathetic and intelligent digital creatures. LISMS has a close relationship with the local industry to bring the research outcomes into transferring a business into a knowledge-intensive one.

Professor Xun Xu Dr. Yuqian Lu Dr. Jan Polzer

Psychology for a better world – Keeping what we value in play Many people recognise that our current way of life is past its use-by date.

However we continue to replicate it day after day. Why? Can the same features of human psychology that prop up the status quo be tapped to create a better, more sustainable world – one that has human and ecological flourishing at its core? This question is at the centre of a research programme led by Niki Harré in the School of Psychology.

In a series of recent studies, the researchers have examined what people value most deeply, and found this to be human connection, expression, the natural world, personal strengths, vitality and spirituality. What is more, when people realise that others also share these values they feel a sense of belonging to a human community, reassurance and hope. These feelings inspire cooperation and creativity – qualities critical to facilitating constructive responses to the issues we face.

People also have a propensity to learn by copying, either through direct imitation or by living the stories of their time. When we demonstrate and discuss ‘tales of joy’ about a positive future, we invite people to enter these, and thereby help bring that future into being. How do we promote tales of joy? One way is through values-based networks. Niki has led two sustainability-focused networks, one at Western Springs College and one in the Faculty of Science at the University of Auckland. These have resulted in waste separation and composting, reductions in resource use, talks and workshops, student run events and interdisciplinary teaching.

What would work and workplaces look like if they reflected our deepest values and attempted to be living tales of joy?

Professor Niki Harré Dr. Helen Madden Dr. Daniel Hikuroa

The Empathic Computing Laboratory The Empathic Computing Laboratory (ECL) at the Auckland Bioengineering Institute develops systems that help share understanding. Using Augmented Reality(AR), Virtual Reality(VR), physiological sensing, a​ nd other technology, the ECL staff and students are developing new ways to enhance face to face and remote collaboration. For example, using 3D cameras to scan a person’s surroundings and sharing them live with a remote collaborator, developing ways to share gaze an​d gesture cues remotely, or measuring people’s brain activity when working together.

The goal of the ECL’s research is to enable remote people to work together as easily as, or better than, if they were face to face. The impact on the future of work will be that people will be able to get expert help with tasks wherever and whenever they need it, without the expert needing to take a long and expensive trip. The AR/VR technology also enables the remote expert to look through the eyes of the local worker and so better understand and assist with the task that they are trying to perform.

The technology could be applied in many domains. For example, doctors could use it to have a remote surgeon provide them with expert help, first responders to allow a remote supervisor to better understand an emergency situation, and field service engineers to get help fixing complex machinery.

Outside the workplace, it could provide new tourism experiences, connect remote friends and family, and enable remote people to share concerts or other live experiences together.

Dr. Amit Barde Dr. Huidong Bai

CARES: HumanMachine Interaction Effective delivery of services or functions by a robot or virtual agent, requires a deep understanding of how humans perceive and interact with machines, so that scientists and engineers can design machines that are safe, useful and acceptable by humans.

In the health context, we are interested in how robots might assist patients to improve their health outcomes. We are conducting research in several areas: Our research has shown that Paro, a companion robot, can significantly reduce loneliness, increase social interactions, and improve blood pressure in older adults in rest-home settings.

We have also shown that robots can improve the management of chronic obstructive pulmonary disease. A randomised controlled trial led by Professor Elizabeth Broadbent, (funded by the HRC), found that robots significantly increased patient adherence to long-acting inhalers and pulmonary rehabilitation exercises compared to the control group.

We are also investigating whether robots can assist people with mild dementia to keep stay independent for longer and reduce burden on healthcare systems.

Interactive healthcare robots for dementia are being developed by the CARES team with Korean researchers to enhance cognitive abilities, assist with daily routines and provide companionship.

In an international collaboration, Dr Ho Seok Ahn is leading a team investigating how we can apply our understanding of the human social interactions, behaviour and emotions to create robots that behave and interact with people in socially acceptable ways.

Our research into human-robot interactions will help us to design machines of the future that will be help people in their daily lives.

Professor Elizabeth Broadbent Professor Ngaire Kerse Professor Bruce MacDonald Dr Kathy Peri, Dr Ho Seok Ahn Dr Craig Sutherland

Jong Yoon Lim Chris Lee

Assistive Augmentationinfusing humanity into technologies As technologies become more pervasive and precise, novel smart devices increasingly permeate our everyday activities. However, the interfaces exposed by these new ‘smart things’ limit both human-computer and humanhuman interaction to a very narrow and unoptimized manner. Either we spend significant time figuring out these ‘smart’ technologies or the way we use them disconnects ourselves from physical reality. The need to design intuitive technology that fits and adapts to us becomes increasingly apparent.

The Augmented Human Lab focuses on designing intuitive human-machine interfaces and envisions breaking down barriers between humans and technology, therefore creating a seamless integration to achieve optimal outcomes. This has included projects such as “FingerReader”, that allows blind users to access information simply by pointing at objects and asking questions; “Muss-Bits”, that allows deaf users to ‘feel’ music; “GymSoles”, a smart insole that enables users to perform exercises with correct body posture; and “Sparsh”, that enables users to move digital media between devices by simply grabbing them into their hands.

Our work brings a sense of the ‘humanity’ into technology. This ranges from understanding real-life contexts in which technologies function, and understanding where technologies are not only exciting or novel but have meaningful impacts on the way people live. This particularly pertains to the lab’s mission of enhancing how we live, work, and play.

Associate Professor Suranga Nanayakkara Haimo Zhang - Research Fellow Yilei Shi - PhD student Tharindu Kaluarachchi - PhD student

Understanding cancer’s complexities through augmented reality In the treatment of cancer, there are many unsolved challenges facing oncologists and their patients; cancer is an ever-changing complex disease that is still poorly understood. Towards better understanding how cancer changes with time, we have been given a rare research opportunity: a patient with a lung tumour and almost 100 metastases requested and consented to donating her cancer tissue for research, providing an opportunity to investigate the complexity of cancer in a single patient.

Our group is using genomic technologies to better understand how this patient’s cancer evolved. To understand the complexities of this disease, our approach would need strong visualisation to link our data, including mathematical models of tumour evolution, to distinct body locations and points in time, which existing cancer data visualisation tools were unable to meet.

A team spanning the Centre for eResearch (genomic data analysis and augmented reality), School of Computer Science (mathematical reconstruction of tumour evolution), School of Architecture and Planning (spatial interaction), Faculty of Medical and Health Sciences, and New Zealand eScience Infrastructure (NeSI; bioinformatic computing), together produced an interactive Augmented Reality representation of this patient’s tumours, how they develop over time, and how unique genomic patterns relate to where they are found in the body.

With this tool, we are gaining insight into how metastatic cancers change over time, facilitating communication of concepts so complex they were previously difficult to conceptualise or discuss. We see a peek into the future of oncology, where multidisciplinary teams may meet in augmented settings to amass layers of patient data enabling better treatment decisions, providing a platform for coming to a shared understanding of prognosis and treatment. Our understanding of the evolution of cancer is itself evolving, thanks to exciting novel tools like this.

Professor Mark Gahegan, Bianca Haux, Nick Young, Associate Professor Uwe Rieger, Dr Michael Davis, Jack Guo, Dr Alex Gavryushkin, Professor Alexei Drummond, Tamsin Robb, Braden Woodhouse, Dr Kate Parker, Dr Cherie Blenkiron, Peter Tsai, Dr Jane Reeve, Professor Cris Print; Dr Ben Lawrence

Using predictive analytics ethically The combination of vast bodies of searchable data and sophisticated analytic tools allows us to assess with previously unobtainable accuracy the probability that a child will be maltreated, a released prisoner will reoffend, a discharged patient will be re-admitted, how a court will decide a case, how a student will do at university.

While predictive analytics promises significant benefits it also poses significant ethical challenges: It will usually be impossible to obtain meaningful consent; Identification as highrisk may be stigmatising. Predictive tools are more accurate than alternatives but even the best make mistakes. Predictions may reflect the content of flawed data sets. The technology challenges current expectations around privacy, relies on selective information held in datasets, and may exclude human judgement. Improved capacity to identify need may exceed our ability to respond. Many of these risks bear disproportionately on minorities or already marginalised groups.

This daunting list of risk has led many to conclude that it is simply unethical to use predictive analytics in social policy contexts.

This project aims to address those concerns to develop an ethical framework for social policy uses of predictive risk modelling.

The Practical Strand The principal investigator works with government ministries and agencies building assessment frameworks.

The Privacy, Human Rights, and Ethics framework – the PHRaE – built for the Ministry of Social Development, is a detailed online assessment tool which aims to identify privacy human Rights, and ethics risks early in the design cycle of a proposal to use client data. The PHRaE has been subject to extensive internal and external consultation. We are currently following recommendations that it be further developed to become an All-of-Government resource.

The Theoretical Strand The more theoretical strand of the project addresses widespread scepticism about the political and ethical legitimacy of social policy uses of predictive analytics. Not only is it ethical to use predictive analytics in these context, it will often be unethical not to do so.

Professor Tim Dare