Graduation Thesis: Gamification and UI/UX design of an energy consumption dashboard

GRADUATION THESIS

GAMIFICATION AND UI/UX DESIGN OF AN ENERGY CONSUMPTION DASHBOARD

Anna

Munk - 463090

Saxion University of Applied Sciences

Creative Media & Game Technologies

Ambient Intelligence Research Group

463090@student.saxion.nl

Project lead(s): Javier Ferreira Gonzalez and Peter Ebben

Company supervisor: Alejandro Moreno Celleri

Graduation supervisor: Tim Roosen

Author keywords

Gamification; Dashboard; Energy; Community; Design; Solar; Intuitive design; Energy consumption; Energy production; User Interface; User Experience

Preface

The following thesis covers the development process of an energy analysis tool. The Ambient Intelligence research group provided this project for the benefit of the Vriendenerf and Aardehuis neighborhoods. This took place over a period of five months, from November 2022 to April 2023, and involved target group research, design principles, and user testing.

I am a fourth-year graduate of the Creative Media and Game Technology program at Saxion, with a strong emphasis on user interface and user experience design. My previous experience and acquired knowledge render me qualified for this project. For instance, I have previously collaborated with a team of four design students and developed a user interface for a virtual reality program implementing hand tracking.

Before I start the thesis, I'd like to thank the Ambient Intelligence research group for giving me the opportunity to write it. I would especially like to thank Alejandro Moreno Celleri for not only introducing me to the project but also guiding me through it as a company supervisor. In addition, I want to thank Javier Ferreira Gonzalez and Peter Ebben for organizing the community meetings and other behind-the-scenes work.

Finally, I'd like to express my gratitude to Tim Roosen, my graduation coach, for his amazing guidance.

I sincerely hope you enjoy this reading and discover something interesting and insightful.

Anna Munk

Enschede, 6 April 2023

Abstract

This graduation project, led by the Smart Industry research group at Saxion together with the Ambient Intelligence research group (AmI), aims to create an energy analysis tool for the SERENE/SUSTENANCE projects in the Dutch sustainable communities of Vriendenerf and Aardehuis. The goal of the project is to create an energy consumption dashboard by adding gamification components to engage users in a playful form of motivation.

The project is focused on developing an interactive dashboard to track energy production and consumption. The proposed research question is "How can gamification be incorporated into the current prototype, and what gamification strategies would be suitable for an effective energy analysis tool that meets the needs of the target group in Aardehuis and Vriendenerf?"

To answer the research question, this project involves desk research, a review of the literature, several tests using various techniques, user testing with the intended target audience, and an expert interview. The ultimate objective is to improve the energy analysis tool and encourage local communities to use sustainable energy sources.

List of Figures

List of Tables

1.0 Introduction

1.1 Company Outline

The educational institute Saxion is home to numerous research teams. This graduation project is being led by the Smart Industry research group, which includes the Ambient Intelligence research group (AmI). The Ambient Intelligence research team's goal is to make daily living smarter by working with advanced technology, including ambient technologies. AmI uses sensor technology, artificial intelligence, and augmented reality, along with professional knowledge, to develop practical solutions for safety, sports, renewable energy, and animal welfare. All of this enables AmI to respond to meaningful, demand-driven research topics on key issues.

The current Ambient Intelligence research group consists of about 26 researchers from numerous fields of expertise. Besides that, the AmI group works with other researchers from Saxion and other institutions with a focus on high-tech applied science. Additionally, AmI has structural cooperation and sponsorship with Roessingh R&D, the University of Twente, and Thales. Also, numerous European development grants frequently support these projects. This means that the research team's financial resources vary depending on the theme and subject matter (Saxion, n.d.)

Only a small percentage of the AmI researchers were directly involved in the development of this product. Javier Ferreira Gonzalez and Peter Ebben, both also working for the AmI research group, served as extra quality assessors contributed feedback, and provided communication with other parties involved and the product's final user. The direct supervisor for this project was Alejandro Moreno Celleri Weekly meetings were arranged for the supervisors for quality assurance, feedback, planning, and brainstorming.

1.2 Project Introduction

The Horizon 2020 SERENE and SUSTENANCE projects aim to demonstrate cost-effective and customer-oriented solutions for effectively integrating different energy system carriers for the sustainable development of regional communities to meet their energy needs from local renewable energy sources (Saxion, 2022).

Doing so will speed up the transition of the European electricity system to a more decentralized structure with local power and consumption. For smaller cities, villages, and local communities, some of the objectives include addressing all significant and relevant technical and non-technical challenges in establishing the best combinations of local renewable energy (solar PV, wind), energy storage (battery and thermal), flexible demand (heat pumps, HVAC, electric boiler, electric vehicles, wastewater pumping), and demand-response based technologies and solutions.

Using heat pumps, solar PV, and smart grid technology to handle load balancing problems while also encouraging the self-consumption of local renewable energy, the project's overall goal is to investigate how citizen involvement can be improved to produce 100% renewable energy on a neighborhood level. By doing this, it will replace the use of natural gas as a heat source in existing neighborhoods while reducing reliance on the national grid and preventing issues with nearby lowvoltage grids.

The current project in the Netherlands consists of the following demonstration sites: the community of Aardehuis and the neighborhood of Vriendenerf, both of which are active in the fields of sustainability, renewable energy, and self-sustaining communities. These two communities will work

collaboratively to achieve key objectives while building a sustainable neighborhood and spreading ideas and experiences across the region (Saxion, n.d.).

1.3 Previous Research

As mentioned before the SERENE/SUSTENANCE project's goal is to create a customer-centric solution. For this AmI wants to design an energy analysis tool. With this tool, users from both communities can check their energy production and consumption.

It is important to mention that two graduation students started working and writing their thesis on this project from February 2022 until June 2022.

Their project goal was to create the first baseline of this energy consumption dashboard. Together they developed the prototype on which this project is based.

Marvin Eilander focused his research on how to add elements into the prototype that will push behavior change. The research Marvin Eilander did on the phycological aspect is very in-depth and gives a lot of insights into human behavior. With that research and testing were these interventions Marvin Eilander found to be the most effective: general tips, consumption visualization, personal recommendations, and a Leaderboard (Eilander, 2022)

In collaboration with Marvin Eilander, Gerben Bornebroek worked on creating the current version of the dashboard prototype. Gerben Bornebroek researched accessible and intuitive dashboard applications and created low-/mid- and high-fidelity prototypes, which all got tested, iterated, and improved. The final version of Gerben Bornebroek and Marvin Eilander’s prototype will be used as a base-layer for this project (Bornebroek, 2022)

Another significant point that needs to be made is that Marvin Eilander and Gerben Bornebroek were unable to communicate and/or conduct tests with the residents of Vriendenerf and Aardehuis due to other ongoing local research projects. This means all tests Marvin Eilander and Gerben Bornebroek made were done with a substitute target group and are therefore unverified.

1.4 Target Group

For this project, the primary target audience was the residents of the SERENE/SUSTENANCE project communities, specifically the Aardehuis and Vriendenerf communities, which consist of 24 and 12 residential buildings, respectively. These communities are in the village of Olst in the province of Overijssel in the Netherlands (Eilander, 2022) (Bornebroek, 2022)

Marvin Eilander and Gerben Bornebroek previously conducted thorough research to identify the project's target audience. They utilized a mind-mapping technique to identify key characteristics and points of interest.

One area of focus was the location and demographic information of the communities, including the age group and size of the communities, as well as differences between these two communities.

Additionally, the research delved into the personalities of the residents to understand their behavior, with key characteristics such as energy consciousness, community-mindedness, empathy, and tolerance being identified. This information was utilized to anticipate how the residents would interact with the created product.

Lastly, this research examined the residents' needs and desires, with key points including a desire for connection with nature, self-expression, and a commitment to creating a sustainable future.

As a result of these findings, Marvin Eilander and Gerben Bornebroek developed personas for both communities, considering the differences in age, skills, needs, and pain points. These personas can be found in Appendix 1

2. Problem Definition

The problem that the Horizon 2020 SERENE and SUSTENANCE projects aim to solve is the lack of an easy way to track all their energy production and consumption. The Ambient Intelligence research group at Saxion is working on designing an energy analysis tool to meet the needs of the target user group in the demonstration communities of Aardehuis and Vriendenerf.

However, the previous research on this topic was limited and based on testing with a substitute target group, so there is a lack of understanding of the specific needs. Therefore, the problem to be addressed in this project is determining the requirements and design of an effective energy analysis tool that will meet the needs of the target user group in Aardehuis and Vriendenerf and successfully encourage behavior change towards sustainable energy use in these communities.

2.1 Research Objectives

The following objectives are set to determine when this project phase has been completed.

• A functional prototype in Figma was created.

• New gamification features will be added.

• The prototype was tested throughout different iterations and improved.

• The prototype's design makes it simple to adjust, add or remove features.

2.2 Main question

How can gamification be incorporated into the current prototype, and what gamification strategies would be suitable for an effective energy analysis tool that meets the needs of the target group in Aardehuis and Vriendenerf?

2.3 Sub questions

1. How can the current prototype be improved based on feedback from the target user group?

2. How can gamification be incorporated into the energy analysis tool to effectively facilitate a shift toward sustainable energy use?

3. How can the energy analysis tool be designed to be intuitive and accessible for the target user group?

2.4 Methodology

The Design Thinking Methodology was applied to this project. Empathy and iteration are prioritized in the problem-solving method known as "design thinking" to produce user-centered solutions. It is a technique that is frequently applied in the fields of innovation, business, and design. An overview of the design thinking procedure is provided below:

Empathize: Understand the user's needs, wants, and pain points.

Define: Clearly articulate the problem you are trying to solve.

Ideate: Generate as many ideas as possible, without judging or evaluating them.

Prototype: Create a physical or digital representation of the idea.

Test: Gather feedback from users and use it to refine the idea.

Design Thinking is frequently used to find innovative solutions to complex, ambiguous problems. It is a user-centered strategy that emphasizes iteration and empathy. Designers are able to develop solutions that are specialized to the needs of the user by comprehending their needs.

The project began with research on gamification and energy analysis tools, followed by a problem statement, goal, and project deliverables. Ideas from Gerben Bornebroek and Marvin Eilander that were not incorporated into their final prototype due to a lack of time or that were removed during development were revisited and discussed with Alejandro Moreno Celleri.

New ideas to incorporate gamification were ideated by brainstorming, creating mind maps, and talking to colleagues about it. The results from the ideating process were sketched out.

On paper, the prototype sketches were created; these were then converted into the online wireframing tool Figma. All the following iterations were completed in Figma.

The testing was done with the Figma prototypes, and testing methodologies, like questionnaires, card sorting, A/B Testing, and usability tests were used to get the best and most diverse results from our end users.

2.5 Project Relevance

As the importance of generating clean energy has grown over time, the SERENE and SUSTENANCE projects have been developed to accelerate the shift towards a more decentralized European electricity system, with an emphasis on local power generation and consumption. The goal of these projects is to make this transition as efficient and cost-effective as possible, with a focus on meeting the needs of the customer. To aid in the management of their own power and energy resources, the local communities participating in these projects will require tools for energy analysis.

3. Theoretical Research

3.1 Competitors analyze

3.1.1 Previous Research

For his thesis, Gerben Bornebroek conducted a comprehensive analysis of both direct and indirect competitors. Then, all competitors were evaluated again to see if there had been any changes to their functionality, key selling points, or visual appeal. (Bornebroek, 2022)

A visualization as well as a comparison of the design in the beginning of 2022 and the beginning of 2023 of the following competitors can be found in Appendix 2.

3.1.1.1 Conclusion

After re-evaluating the four competitors previously analyzed by Gerben Bornebroek, the following changes were found:

The competitors' Sense and Kostal stayed the same since the beginning of 2022 and no new functionalities or key points were added.

MySolarEdge and Tesla both changed their design. MySolarEdge changed its design to a more minimalistic and modern look. They removed the colorful illustration on the starting page and swapped it for a picture. Tesla added an illustration of a house, to show the energy flow between the solar panels, the grid, the Powerwall, and energy usage.

A more detailed overview can be found in Appendix 3.

3.2 New Competitor Research

After studying the competitor analysis previously done by Gerben Bornebroek it was important to check the market for changes or new competitors since the beginning of 2022. In the following, three competitors were analyzed for their look and their features.

3.2.1 Home Assistant Energy Management

Home Assistant is a popular home automation platform that allows users to control and automate their smart home devices. It is open source which means that it allows the worldwide community to create their own dashboards and designs. Most users let it run on a Raspberry Pi or a local server.

For this project, Home Assistant Energy Management was useful to look at to gather ideas and dashboard layouts. The design is very user-need-based, that way each user can customize it to their requirements and features needed. (Home Assistant, n.d.)

3.2.2 RegalGrid

RegalGrid is a mobile app that allows users to control and automate their smart home devices. Users can utilize their smartphone or tablet to not only monitor consumption and system efficiency, but also remotely control devices. For installers, the app allows the configuration and monitoring of all necessary systems.

The review of the RegalGrid design was beneficial due to its clear and detailed data presentation. The simplicity of the design ensures that even non-technical users are able to understand it. Additionally, the integration of a point collection feature adds an element of gamification to the energy dashboard.

Features: Monitoring, Active Control, Maintenance, Customer Management, Collecting Points (RegalGrid, n.d.)

Figure 1 Home Assistant Energy Management Interface by Home Assistant, n.d.

Figure 2 RegalGrid Interface by RegalGrid, n.d.

3.2.3 Energyflip

Energyflip is a company that provides energy management solutions to homeowners and businesses. EnergyFlip can monitor the user's standby consumption, gives energy-saving tips, and provides the user with the smartest way to use the generated energy to save money.

Features: Monitoring, Active Control, Energy saving tips, money-saving tips, and comparison (Energyflip, n.d.)

3.2.4 Conclusion

Analyzing these three competitors lead to the following conclusion:

Features like monitoring the current energy consumption and production should be directly visible when opening the app. All three keep the design of the data presentation clear and concise, that way, users without technical knowledge are able to understand it.

RegalGrid and EnergyFlip both implemented features that gamified the application. Collecting points, comparisons, and giving tips, these features may encourage users to use the app longer.

3.3 Gamification

"Gamification is design that places the most emphasis on human motivation in the process. In essence, it is Human-Focused Design (as opposed to “function-focused design”) Gamification is the craft of deriving all the fun and engaging elements found in games and applying them to real-world or productive activities." - Chou, n.d.

To effectively incorporate gamification into the current prototype, a thorough analysis of relevant gamification frameworks was conducted. Three frameworks were identified that shared some similarities but also had distinct differences. These frameworks were then summarized and compared, ultimately leading to the selection of one framework to use in the current project.

3.3 1 Gamification Research

3.3.1.1 Gamification: What It Is and Why It Matters to Digital Health Behavior Change Developers

In 2013, Brian Cugelman published a paper on the topic of digital health and the use of gamification as a means of behavior change. He identified seven core elements that contribute to the persuasive nature of gamification.

• Goal setting: The user can make the decisions to work toward a goal

• Capacity to overcome challenges: The user can grow, learn and develop themself

• Performance feedback: The user gets constant feedback through the experience

• Reinforcement: The user gains rewards and avoids punishments to enhance the experience

• Progress comparison: The user can compare themself with others

Figure 3 Energyflip Interface by Energyflip, n.d.

• Social connectivity: The user can connect and interact with other people

• Fun and playfulness: The user is focusing on an alternate reality

These elements work together to facilitate growth, learning, and development, and to provide rewards and punishments as motivation. By monitoring progress and interacting with others, individuals can engage in an alternate reality that is both enjoyable and effective at fostering change.

(Cugelman, 2013)

3.3.2.2 Octalysis Framework

The Octalysis Framework by Yu-kai Chou is widely known within the game development industry. Developed by Yu-kai Chou following extensive research on gamification, this framework outlines eight core drives that are essential for successful gamification efforts.

• Epic meaning and Calling: The users believe that they were “chosen”

• Development and Accomplishment: The users want to overcome challenges and make progress

• Creativity and Feedback: The users express creativity and get feedback to respond on

• Ownership and Possession: The users feel like they own something and feel ownership towards it

• Social influence and Relatedness: The users get a sense of being included and can connect with others

• Scarcity and impatience: The users are shown something they cannot get to trigger interest

• Unpredictability and Curiosity: The users want to know what is happening next

• Loss and Avoidance: The users get the sense that they could lose progress if they don’t act immediately

This framework is mostly used to create games with a good gamified system. To prove if a game has good gamification, a score can be calculated based on how each subject of the framework was used in the end. (Chou, n.d.)

3.3.2.3 Gamification mechanics for behavioral change: A systematic review and proposed taxonomy

This paper, published in 2017, was authored by Ramon Hervas, David Ruiz-Carrasco, Tania Mondejar, and Jose Bravo. The authors conducted a systematic review of literature that covers the psychological aspect of behavior change and identified seven main psychological fundamentals that are related to behavior change.

Psychological concept Definition

Self-efficacy

Cognitive restructurings

Social influence

Vicarious learning

Shaping

Strength of one’s belief in personal aptitudes to reach goals

Learning process to identify and confront ineffective and disruptive behaviors

Social power to change behavior of others in a particular direction

Learning that is derived from indirect sources, rather than direct instruction

Gradually molding to perform a specific response by reinforcing responses that are like the desired response

Behavioral momentum Relation between resistance to change and the rate of reinforcement obtained

Nudge theory Reinforcement and indirect suggestions to try to achieve non-forced expected behavior

Through a comprehensive review of relevant literature and analysis of multiple publications, the researchers identified six gamification mechanics that can be utilized to facilitate behavior change: goals, status, randomness, appointment, scoring, and immersion. Each of these mechanics was further divided into subcategories and accompanied by definitions and psychological fundamentals. (Hervas, Ruiz-Carrasco, Mondejar, Bravo, 2017)

Table 1 Definition of Psychological Concepts by Hervas, Ruiz-Carrasco, Mondejar, Bravo, 2017

Figure 4 Gamification Mechanics by Hervas, Ruiz-Carrasco, Mondejar, Bravo, 2017

3.3.2 Conclusion

This analysis of the frameworks was done to answer the sub-question two. The frameworks were compared to identify similarities and differences.

After analyzing these three frameworks, the conclusion was made to base this project mainly on the Taxonomy of gamification mechanics from the paper “Gamification mechanics for behavioral change: a systematic review and proposed taxonomy” However, it is important to note that the theories and insights of all three frameworks overlap.

The decision to utilize this framework was based on the researchers' focus on the psychological aspects of behavior change and the fact that it was designed to target groups at different stages of behavior change. Given that the target group for this project is already engaged in environmentally friendly behavior and open to new technologies, it was determined that this framework would be appropriate to use in this project as it does not need to address the initial stages of behavior change.

4. Ideating

4.1 Gamification Features

Following the review of gamification frameworks, competitor analysis, and a thorough examination of Marvin Eilander’s previous findings on gamification, the selection of the following three features was made. These features were chosen due to their potential to gamify the dashboard, while still setting the focus on the app's primary function.

4.1.1 Leaderboard

Many social dashboard apps, such as fitness apps, utilize dashboards to motivate users through competition. These dashboards often include leaderboards on which users could compare their progress with others in their social circle. Being at the top of the leaderboard provided a sense of accomplishment and social influence. The incorporation of this feature into the dashboard was previously discussed but ultimately scrapped due to concerns about the use of personal data. An answer to the privacy problem was found through desk research on leaderboards and discussions with colleagues. The goal was to design a leaderboard that is more generic and does not list specific numbers or names. Instead, it only displays a user's standing on the leaderboard, increasing their level of privacy.

Figure 5 Leaderboard Design by Maéva Laurent 2021

4.1.2 Goals

The psychological aspects of goal setting are broad, encompassing factors such as social influence, shaping, and behavioral momentum. Allowing users to create and set their own goals or challenges provides them with a sense of purpose and something to work towards. This, when paired with a schedule or countdown feature, serves as a powerful motivator for users to maintain and improve their performance. Personal goal setting is a key trigger for users to act and strive for selfimprovement. The ability to create personal goals can help users to focus on what they wish to achieve and provide them with a sense of direction. Additionally, a personal goal-setting feature can foster a sense of self-efficacy, encouraging users to believe in their abilities to achieve their objectives.

4.1.3 Achievements

Incorporating achievements or badges into the design of the project serves to enhance the overall UX by providing users with a sense of accomplishment and progress. The psychological principles behind achievements are like those outlined earlier about goals. An achievement can be defined as a significant milestone that marks a progression or advancement. By including these elements in the design, users can see evidence of their efforts and progress.

In this project, the idea is to include streak-based achievements, such as "saved X amount of energy in Y days," to provide users with a sense of accomplishment as they collect and complete these badges. The inclusion of this type of achievement is intended to motivate users to sustain their efforts and strive for improvement. This feature is not only adding an element of fun and excitement to the UX but also pushes users to engage more with the platform and achieve their goals.

Figure 6 Habit Mobile App design by Budiarti R. 2022

Figure 7 Duolingo Achievement App Design by John Trivelli 2020

4 2 Previous prototype

4.2.1 Analyzing

After the current prototype was analyzed and reviewed, notes were taken of any features that were effective and functioning properly as well as identifying areas for improvement, removal, or alteration based on opinion.

4 2.2 Findings

The following findings were compiled and presented:

• Suggested features included a leaderboard, back buttons, animations, new icons and navigation, and personalized predictions

• The home page could benefit from a more visually appealing design

• The usage page contained charts that could be simplified

• The goal page could be redesigned with a more visually appealing and motivating design, as well as the addition of community goal-setting features

• The community page could be redesigned, and new features could be added

• The profile page could be personalized with the inclusion of a profile picture, name, and badges/achievements

4 3 Moodboard

After conducting an analysis of the previous prototype, various dashboards were consulted to gain an understanding of potential design solutions. To aid in the conceptualization of ideas, a mood board was created to visually represent the aesthetic. During the consultation, it was noticed that many other dashboard applications use animations to mimic the flow of certain elements, such as the flow of energy from the solar panel to the battery.

Figure 8 Moodboard

4 4 Sketches

Once the initial research and analysis had been completed, the next step was to begin sketching out potential design ideas. Several sketches were created, including those for leaderboards, badges, the home page, a community page with community goals, a user profile page, and a usage page. These sketches were used as a foundation for further development and design iterations. All sketches can be found in Appendix 4.

Based on all the above, following features were noted as essential to add for this phase of the project to gamify the dashboard app:

• Leaderboard

• Animations / Illustrated look

• Achievements

• Community goals

4.5 Conclusion Ideation

During the project's ideation phase, it became clear what features needed to be included in the dashboard prototype to effectively address the main research question. A thorough mood board, research process and initial sketches were used to establish the prototype's appearance and feel. However, it became clear during the analysis of the earlier prototype and related research papers that the initial step should be to carry out testing with the actual target group. This was deemed necessary to confirm the accuracy and reliability of the findings, which Gerben Bornebroek and Marvin Eilander established. In order to make sure that the research question was sufficiently addressed and that the prototype delivered the intended value proposition, it was decided to prioritize testing with the target audience.

Figure 9 Sketches

5. Testing Session 1

5.1 Objectives

This testing session was very necessary since the prototype was not tested with the actual target group in this state. The whole testing document can be found in Appendix 5.

An usability test was performed with two testers from the communities in Aardehuis and Vriendenerf with the following objectives:

1. Test if the current prototype is fulfilling the needs of the user

2. Gather information about new Ideas

For the first objective, five scenarios were created for the user to go through.

1. Create a tip for the community

• Task: “You read a good tip about how to save energy during the winter and want to share that tip with your community.”

• Home page -> Side menu -> Community -> Community Tip plus button

2. View the information on an individual solar panel

• Task: “The sun is shining a lot today and you are curious to see more information about your individual solar panels.”

• Home page -> Side Menu -> Solar panels -> View button

3. Find a chart that shows how much energy is used per day and where.

• Task: “You have charged your car at home today and now want to know how much of your daily use went towards that.”

• Home page -> Right icon bottom menu -> Select day

4. Compare Energy production

Figure 10 Previous Prototype

• Task: “You would like to compare how much energy you produced this and last week.”

• Home page -> Right icon bottom menu -> Compare button

5. Goal setting

• Task: “To motivate yourself, you would like to set a goal to consume 100 kWh over a period of 14 days.”

• Home page -> Side menu -> Goals –> Set button

In order to achieve the second objective, an open conversation was held in which four ideas were proposed and discussed. To provide a better understanding of each concept, wireframes were presented as visual aids. These wireframes were designed in Figma to illustrate how the proposed ideas could be implemented within the application and provided a clear image of the proposed design.

The concepts that were discussed during this conversation were the implementation of leaderboards, community goals, achievements, and the overall design of the application. Each of these ideas aimed to increase the level of engagement and motivation for the users.

The participants were instructed to "think-out-loud" before they began the test. This means to speak as soon as an idea enters your head without first weighing its value or usefulness.

5.2 Results

The results of each participant's test have been organized down into a feedback grid, which can be found in Appendix 6.

Figure 11 New Ideas Wireframe

Tester 1 Results

The most important findings from the first testing session were that the navigation does not work as intended and that features like goals and the community page are not interesting to everyone, as the first tester mentioned he would never use them

On the new ideas, the tester gave the following feedback:

The leaderboards were a “nice-to-have” feature if they would not show any personalized data. All challenge-orientated features like goals or leaderboards were not interesting at all.

Tester 2 Results

The most important finding from the second testing session was again that the navigation system created a lot of problems. The tester gave the idea to keep all the charts simple, he had a technical background but believes that general people would understand it better.

When talking about the new features the tester gave a lot of ideas and feedback, since he liked being competitive So, a leaderboard and goals would be good for motivation

5.3 Testing Conclusion

The usability test was performed to gather feedback and results for these two objectives:

• Test if the current prototype is fulfilling the needs of the user

• Gather information about new Ideas

The testing session delivered very insightful results, with two different perspectives emerging as a result of utilizing two different testers in the process. One tester, who likes being competitive, expressed a strong preference for features such as leaderboards and goals as they provided a sense of competition and motivation. On the other hand, the other tester did not express any interest in these types of features. The two testers, despite their differences, were able to provide a comprehensive analysis of the product under test and highlight different key areas that need improvement. Both testers agreed that adding a more illustrated look will make finding and understanding data easier for people without technical knowledge.

The results of this testing session revealed several key findings regarding the functionality of the prototype's navigation system. Specifically, it was determined that the utilization of two separate menus caused confusion, as it was not clear where certain pages could be located. Both testers looked for a back button during the tasks multiple times when they got stuck on a page. Additionally, the testers encountered difficulty in finding and understanding certain buttons and their functionality within the prototype. These issues may have been partially attributed to the color scheme used within the prototype, as there was no assigned action color that was only used for interactive buttons and icons.

Further consideration needs to be given to the navigation system and color scheme in order to improve the overall usability of the prototype before moving on to adding the new features

6. Iteration 1

The testing with the two members from the community showed what needed to be done to improve the following iteration. The main problem was the navigation of the prototype together with the fact that users did not understand what was clickable due to the buttons and the color scheme.

6.1. Navigation

To solve the problem with the navigation that occurred during the testing session, a tree diagram of the current navigation through the app was created.

When looking at the tree diagram, it is noticeable that nearly every page can be reached from the home page. But that gave the testers too many options to choose from. That navigation together with the missing back button, left the testers stuck and frustrated.

Figure 12 Previous Navigation

After doing a thorough analysis of the tree diagram and test outcomes, it became clear that changes needed to be made. After further review, it was determined that the implementation of back buttons and changes to the navigation were necessary. As a result, the hamburger menu, which is the button that opens up into a side menu, located in the top left corner was removed and replaced with the addition of two pages with icons in the bottom menu. Furthermore, the profile page can now be accessed directly from every page through an icon in the upper right corner. Additionally, the hamburger menu's settings and log-out options have been consolidated under the profile page.

During user testing, it was noted that while an overview of their solar panels would be a good addition, it was not a frequently used feature. As a result, this page has been moved to the production page in order to make the data available to users who may wish to access it.

6.1.1 Testing Navigation

To ensure the optimal functionality of the new navigation system, a so called card sorting test was executed.

Card sorting is a user-experience research method used to understand how users categorize and organize information. Participants are presented with a series of cards, each representing a piece of content or function, and asked to group them in a way that makes sense to them. The resulting groupings and labels can provide insight into how users think about and understand the information and can influence the design of navigation and information architecture.

The tree diagram was utilized as a means to conduct this testing, focusing specifically on the main cards. This approach was taken as many of the cards within the system serve to present data in a consistent manner throughout the application, such as the usage and production dashboard's capability to display daily, weekly, monthly, and yearly overviews.

The following cards were provided to the tester for sorting.

Figure 13 New Navigation

Figure 14 Card Sorting Test

The testers received a brief overview of the project and the purpose of the testing. The three cards, the home page, usage, and production were positioned for them, and the testers were instructed to categorize the remaining cards according to what they thought made sense. They were informed that the cards could be placed under the existing categories of production or usage, but also given the discretion to create new main categories if they deemed them to be important.

Again, the testers got asked to think out loud to give insights about the intention when sorting the cards.

6.1.2 Testing Navigation Conclusion

This testing phase was conducted with a sample group of six testers. An overview of all results can be found in Appendix 7

All participants sorted the same set of cards into their respective usage categories. Regarding the production category, four testers sorted the cards according to the previously established tree diagram, while two testers suggested a new subcategory specifically for solar panels, with the individual panels being listed underneath. Most testers created their own categories for community and goals. A common feedback point for this section was the suggestion that the card labeled "community goal" could be placed under both the goals and community categories. Apart from one tester, all participants created a separate category for profiles, but with varying and sometimes no additional cards placed next to it.

6.2 Buttons and the color scheme

During the testing phase, a problem was identified in which testers were not able to clearly identify the function of certain buttons or that items were clickable at all. To address this issue, another step that was taken was to adjust the color scheme. By implementing a new navigation system and adhering to the so called 60-30-10 rule, most items are now more visible and easily understood by users. (Adelugba, 2020)

The use of color can play a significant role in this, as it can help to guide the user's attention, create a sense of hierarchy, and ultimately improve overall usability. The 60-30-10 rule is a widely accepted guide for app design, and it helps to create a balanced and cohesive color scheme that is easy for users to navigate.

The 60% represents the primary or neutral color, which serves as the base color for the design, often as the background. This color should be used consistently throughout the app and should be easy on the eyes.

The 30% represents the secondary color, which is used for medium-emphasis elements such as cards, carousels, and other interface components.

The 10% represents the accent color, which is used to create focal points and draw the user's attention to important elements, such as CTA (call to action) buttons, pop-ups, and other significant interface elements. This color should be used sparingly and should be distinct from the primary and secondary colors.

By considering the user experience and the perception of the interface, designers can ensure that buttons and other interface elements are clear and easily understood by users.

Since both participants of the previous testing mentioned that the buttons in the navigation were not clear to them, regarding color, new icons for the new navigation were added as well.

The idea of adding a dark version of the dashboard was considered as part of the color scheme research. After a discussion with Alejandro Moreno Celleri, it was decided to proceed forward with making this version so that users could select between a light and dark mode which in turn will give the user a more customizable experience with this modification to the dashboard design.

6.3 Comparison Page

One page that required substantial modification was the comparison page. Multiple iterations of the page were created, building upon the original design.

Figure 16 Comparison Page Iterations

Figure 15 Before and After Color Adjustment

Ultimately, two designs were chosen for A/B testing: a bar graph and a line graph. The objective of this test was to not only evaluate user preference for the designs but also to ascertain which of the two graphs had superior legibility.

The participants for this test were adults from the age 35 and up, that have some technical knowledge.

A total of five testers performed two tasks in alternating order and were subsequently prompted to answer two questions.

Tasks:

1. You want to compare four weeks in a month to find out on which Friday you have used the most electricity (Bar Graph) (this week)

a. How much did you use that Friday?

2. You want to compare 4 months to find out on which 1st of each month you used the least electricity (Line Graph) (Oct 22)

a. How much did you use that day?

The results show that most testers liked the bar graph more since it was easier to read.

Figure 17 A/B Testing

6 4Testing Iteration 1

Based on the findings above a new iteration of the prototype was created. This iteration had a new navigation system with a new color scheme and an updated comparison page

The updated prototype was tested on the following objectives. The whole testing document can be found in Appendix 8

6.4.1 Objectives

A usability test with the following objectives was performed: Test the new Navigation, Features, and Design

For these objectives, five scenarios were created for the user to go through.

1. Ranking Usage Leaderboard

• You want to check the ranking and look up on which position you are on the usage leaderboard

2. Set up a new Goal

• To give yourself more motivation you want to set up a new goal

3. View Badges

• You saw something about achievements on the home page yesterday and now want to check them

4. Compare Production

• You want to compare four fridays in a month to find out on which you have used the most electricity

5. Amount of energy stored in

• You want to know how much energy you have stored in your battery

6 4.2 Results

The test was conducted by a group of six testers, all of whom have some level of technical knowledge and were within the age range of 30-64. To facilitate a comprehensive understanding of

Figure 18 Dashboard Iteration 1

the test results, another a feedback grid was developed which included all the key points mentioned during the testing process.

Worked Problems

- Ranking Task

- Goals and setting up goals

- The compare page once the tester got the hang of it

- Navigation

- Battery not found, needs to be placed on a different page

- Ranking name needs to be bigger

- Some problems with the compare page, due to prototype functionalities

Questions Ideas

- Why is the battery there?

- What are achievements?

- Why can't I click on this?

- Where is my car charging station?

Table 3 Feedback Grid Testing Iteration 1

6.4.3 Testing Conclusion

- Add Tooltip Icons for Ranking, Community Goals, Badges

- Move Battery to Production or Usage

- Add a wall box to the home page for charging e-cars

- Add a weather prediction to the home page

The testing was conducted with the following objectives: Test the new Navigation, Features, and Design

The process showed that the updated navigation system has already improved the user experience. Test subjects were able to complete tasks faster and with fewer problems. The first two tasks were completed without any issues, and as users grew accustomed to the new system, later tasks displayed improvements. However, the third task revealed some naming issues with the badges that confused some testers. After some first obstacles, task four went without any issues as soon as the user understood the graph. Finally, it became clear that moving the battery to a different page would make the task's battery search more user-friendly. Overall, the testing process provided valuable insights and ideas and specific areas for improvement, like adding a wall box to the home page overview or production prediction, that can be addressed to further enhance the user experience.

7. Iteration 2

7.1 Expert Review

Iteration one of the dashboard was reviewed during a meeting with Danique Hofstee, a UXUI Designer who also works for the AmI research group, and several improvements that could be made were noted. The expert noted that the design had improved significantly since the previous iteration and liked the new navigation layout. However, Danique suggested that adding information text, tooltips, or a beginner's tutorial to help users understand the functionality of the application. Danique also suggested adding a border to the menu and adding more information to the compare page.

Danique was generally pleased with the design and suggested further testing with the intended audience to get more feedback. To enhance the overall user experience, these recommendations will be considered and then implemented.

7.2 Implementing Feedback

A new version of the dashboard was created after a thorough analysis of the test results and professional review.

The new iteration includes several significant updates, such as moving the battery feature to the production page and including a prediction option for energy production. Motivational messages were integrated at the top of the page as well as within the carousel slider, and a wall box was added to the home page's house animation. In addition, some minor design changes were made to create a more consistent look and feel. The dark version of the prototype was adjusted to these changes as well. The links to the light and dark versions of the dashboard prototype can be found in Appendix 9.

8. Final Results

8.1 Final Testing

A thorough final testing session was conducted toward the end of the project to collect insightful feedback. The results of this final user testing session are further discussed in the section below

Residents from the Aardehuis and Vriendenerf were chosen to carry out this final test. Six households, three from each community, were chosen to participate out of the total of eleven households who expressed interest in testing the dashboard prototype. All participants signed a consent form and agreed to audio recording. Appendix 10 shows the entire testing document.

Figure 19 Final Prototype

8.2 Objectives and Questionnaire

The following objectives were set for a usability test that was conducted:

• Navigation

• Understanding

• Ease of use

• Usefulness

• Gamification Elements

Six scenarios were developed for the user to experience to accomplish these goals:

1. Energy Score

• A 5-second test of the Home Page

• Show user Home Page

2. Compare

• You want to compare the energy usage of the last days with each other

• Home -> Usage-> Compare -> Add days

3. Prediction

• You want to know approximately how much energy you will produce tomorrow

• Home -> Production -> Arrow to tomorrow

4. Achievements

• You got a notification that you earned the achievement “15-day streak” and now want to check that out

• Home -> Goal -> Click on Achievement

5. Ranking

• This dashboard is going to have some social features like a leaderboard system. You want to know what place you are in the ‘Saving Ranking’

• Home -> Community -> Saving

6. Goals

• To motivate the user more to use less and less energy, this dashboard is going to have a feature to set your own goal. You want to set a new goal to motivate yourself

• Home -> Goals -> Add

After each scenario/task, the tester answered three questions about navigation, understanding, and ease of use on a Likert scale, for example: How do you rate the navigation to this screen? With the answers being from 1 easy to 5 difficult.

Scenarios four – six also had a question about if they would use that new gamification element. A post-testing questionnaire was also given out to get general feedback on the look and feel of the dashboard prototype

The following questions were asked:

1. From 1 (Liked it) to 5 (Didn’t like it) how did you like the look of the dashboard?

2. From 1 (Liked it) to 5 (Didn’t like it) how did you like the look of the dark version of the dashboard?

3. Which of these two versions do you prefer and would use?

4. How easy (1)/difficult (5) was it to understand the app's menus and navigation?

5. How satisfied (1)/unsatisfied (5) were you with the app's overall functionality?

8.3 Results

As mentioned before, six households from the communities of Aardehuis and Vriendenerf participated in the test with the following objectives:

• Navigation

• Understanding

• Ease of use

• Usefulness

• Gamification Elements .

A detailed overview of all answers to the scenario questionnaire answers and the post-testing questionnaire can be found in Appendix 11.

In the following, a detailed analysis of the individual testing scenarios and their corresponding results will be presented, shedding light on the strengths, weaknesses, and potential improvements of the prototype based on user feedback.

Figure 20 Dark Version

8.3.1 Scenario 1 Energy Score

The participants were given a 5-second glimpse of the energy analysis tool prototype's home page with the goal of determining whether a quick glance is enough for users to determine whether their home now produces more energy than it consumes.

When asked if they could tell if their home was producing more energy than it was consuming based on a quick glance at the home page, 5 of the 6 participants who took part in the testing immediately replied with a "yes." One participant, on the other hand, found the screen to be overwhelming and was unable to respond because their first impression was unclear.

8.3.2 Scenario 2 Compare

The participants were given the following task: “You want to compare the energy usage of the last days with each other.”

Only 2 of the 6 testers were successful in navigating to the right screen by clicking the "compare" button on the usage page. The other 4 testers tried to compare the days on the page for the weekly usage overview but were unable to find the button that would have taken them to the compare page. The feedback gathered from all participants regarding this task indicated that they overlooked the "compare" button completely, as it was located next to the legend of the graph.

8.3.3 Scenario 3 Prediction

The task given to the participants was to estimate how much energy they would generate the following day. Five out of six users were able to find the production page after successfully navigating there, and some were able to locate the button that displayed the prediction of energy usage for the next day more quickly than others. Scenario 3 proceeded smoothly with no issues, as five out of six testers were able to complete the task without any difficulties.

Figure 21 Task 1 Pie Chart

8.3.4 Scenario 4 Achievements

The participants were tasked with checking their earned achievement of a "15-day streak" after receiving a notification. However, some participants found the language barrier to be a small challenge because they weren't all familiar with the idea of achievements. Once the concept was further explained, they were able to comprehend it in the context of the dashboard. All participants, except for one tester, eventually found the achievements on the goal page. Users were surveyed regarding their thoughts on the achievements, a new gamification component. Three of the six testers said they would use this feature but didn't see it as something they would use regularly. The three other testers said they weren't particularly interested in the feature.

8.3.5 Scenario 5 Ranking

The participants were given the following task: “This dashboard is going to have some social features like a leaderboard system. You want to know what place you are in the ‘Saving Ranking’ ” This task presented difficulties for the testers because they missed the ranking due to overlooking the headline and searching for ranking indicators on the production and usage pages. Even when some testers were able to find the ranking, they were unsure of how to determine their own ranking based on the information that was given to them. Furthermore, most users expressed a negative opinion about this feature, as they did not perceive it as something they would use in their daily interactions with the dashboard. Only one participant out of six said they would be interested to use this feature, although they said they would only use it briefly to test it out. All five of the remaining testers said they would not like to take part in such a competition.

8.3.6 Scenario 6 Goals

The testers' last assignment was to create their own energy-saving goal for the dashboard. Since this task was given after the testers had become accustomed to using the dashboard's navigation, everyone was able to find the page with ease. Similar to the ranking feature, some testers had mixed feelings about the goal-setting feature, saying they preferred the dashboard for informational purposes only and no other features.

8.3.7 Post-Testing Questionnaire

A post-testing questionnaire was given to participants after the testing was finished, and it contained five questions about the dashboard's appearance and usability. The results of the questionnaire

Figure 22 Task 4 Pie Chart

Figure 23 Task 5 Pie Chart

analysis showed that most participants preferred the dashboard's light version over its dark counterpart in terms of appearance and feeling. While most participants eventually became familiar with the dashboard's navigation, they voiced their displeasure with the order in which the content was prioritized, specifically asking that more attention be paid to the dashboard's key features.

8.4 Conclusion

The most important finding from the final testing was that the two communities had different opinions about the dashboard, probably because of age differences. Older community members in Vriendenerf who participated in the testing preferred a dashboard that was condensed to just the essential features and included an introductory lesson to explain them. They didn't think that elements like the leaderboard, objectives, and achievements were essential for the app.

In contrast, Aardehuis testers expressed a greater interest in the additional features while stating the importance of giving priority to the core functions. Goals and achievements were viewed as niceto-have but not essential features. Notably, the ranking feature received poor feedback from all users, with many of them expressing a lack of interest in competition as a motivator.

An important positive outcome of the testing was that participants generally thought that it was simpler to navigate the dashboard as they completed each task. Additionally, the objective of giving users a quick overview of the status of energy generation and usage on the home page was successfully accomplished, enabling users to evaluate the situation quickly. Another noteworthy discovery was that several participants expressed a keen interest in offering feedback. Participants in the testing sessions expressed a desire to share their thoughts and ideas to enhance the dashboard.

9. Discussion

This project can be regarded as successful based on the conclusive outcomes of the final testing carried out in section 9. This study used a variety of research methodologies, including expert interviews, card sorting, A/B testing, desk research, and user testing with participants that came close to the actual intended audience as well as with the members of the communities.

Gerben Bornebroek and Marvin Eilander laid a strong foundation for the project at the outset, but testing with the intended audience was lacking. Two representatives from each community of the target audience participated in the first round of testing sessions, and the results provided insightful information that was incorporated into the project's first iteration.

A/B testing and card sorting tests conducted later helped hone the dashboard's user-friendly layout. It is important to keep in mind that these tests weren't carried out with the target group itself, but rather with participants who closely resembled it.

Additionally, an expert review was conducted to get a different viewpoint on the dashboard from someone who is not closely associated with the user group or directly involved in the project. The expert interview was an important source of feedback, offering an unbiased evaluation of the dashboard's usability and design. The knowledge gained from this interview helped to improve the final product. For instance, the advice to include tooltips and make specific design changes resulted in a more thorough and user-friendly final product.

10. Conclusion

The primary focus of this study was to address the research question:" How can gamification be incorporated into the current prototype, and what gamification strategies would be suitable for an effective energy analysis tool that meets the needs of the target group in Aardehuis and Vriendenerf?"

Three sub-questions were created to help support the overall investigation and help answer this main research question.

The answer to sub-question one can be found in sections 6.3 and 9.4. These two sections describe the feedback that was gathered in the two testing sessions that were performed by the target user group. For instance, while it was evident from the first testing that the navigation is broken, this was resolved in the second testing with the target group.

The answer to sub-question two can be found in section 5. This section was devoted to finding and coming up with gamification ideas that were later incorporated into various dashboard prototype iterations. A leaderboard, as well as personal and group goals and achievements, were added as gamification elements for this project. Some of the target audience members stated that they would use these features as inspiration.

The answer to sub-question three can be found in sections 6.0, 7.0, 8.0, and 9.0. These sections provide an overview of the method of iterative design and testing sessions that were carried out, which together helped to develop a final iteration that is user-friendly and accessible to the target audience. For the energy analysis tool to be usable and effective for the target users, the insights and feedback gathered from these sections were essential. Based on the different technical knowledge and age of the two communities, particularly the most recent testing demonstrated that the two target groups require different designs to be intuitive and accessible.

11. Recommendation

How can gamification be incorporated into the current prototype, and what gamification strategies would be suitable for an effective energy analysis tool that meets the needs of the target group in Aardehuis and Vriendenerf?

Numerous new insights were discovered as the target group got more involved in the project, especially during the final testing session. To confirm the results of the earlier testing sessions, which were only conducted with three members from each community, it is highly recommended to carry out another round of testing with a larger number of community members. The users' desire to participate more and offer feedback was one significant result of the testing session. Because of their willingness to actively engage and participate in the research process, the target audience should get involved in the ongoing development and improvement of the dashboard.

Another recommendation is to expedite the dashboard's development, possibly by starting with the key features so that users can get a better understanding of the application. The current result of this project can be viewed as a good place to start because new features can be removed without affecting the dashboard's overall functionality.

Reference list

Adelugba, A. (2020, September 11). How the 60-30-10 rule saved the day. Medium. Retrieved February 2023, from https://uxdesign.cc/how-the-60-30-10-rule-saved-the-day-934e1ee3fdd8

Bornebroek, G. (2022). Ui/Ux Design Of An Energy Consumption Dashboard For Carbon Neutral Communities (thesis). Saxion, Enschede.

Chou, Y.-kai. (2022, June 9). Octalysis: Complete gamification framework - yu-kai chou. The Octalysis Framework for Gamification & Behavioral Design. Retrieved December 2022, from https://yukaichou.com/gamification-examples/octalysis-complete-gamification-framework/

Cugelman, B. (2013). Gamification: What it is and why it matters to Digital Health Behavior Change Developers JMIR Serious Games. Retrieved December 2022, from https://doi.org/10.2196/games.3139

Eilander, M. (2022). Motivating energy-conscious behaviour in dashboards for carbon-neutral communities. (thesis). Saxion, Enschede.

Energyflip. (n.d.). Energyflip. EnergyFlip. Retrieved January 2023, from https://www.energyflip.com/

Hervas, R., Ruiz-Carrasco, D., Mondejar, T., & Bravo, J. (2017, May 1). Gamification Mechanics for Behavioral Change: a systematic review and proposed taxonomy. Gamification mechanics for behavioral change: a systematic review and proposed taxonomy. Retrieved December 2022, from https://dl.acm.org/doi/pdf/10.1145/3154862.3154939

Home Assistant. (n.d.). Home assistant. Home Assistant. Retrieved January 2023, from https://www.homeassistant.io/

Kostal. (n.d.). Kostal Solar App KOSTAL Solar App - PV-monitoring, altijd en overal. Retrieved January 2023, from https://www.kostal-solar-electric.com/nl-nl/producten/monitoringsoftware/solar-app/

MySolarEdge. (n.d.). MySolarEdge. SolarEdge. Retrieved January 2023, from https://www.solaredge.com/en/products/software-tools/mysolaredge

Regalgrid. (n.d.). Regalgrid. Retrieved January 2023, from https://www.regalgrid.com/en/

Saxion. (2022, May 2). The H2020 Serene Project: Delivering sustainable and Integrated Energy Systems in local communities.: Saxion. Hogeschool Saxion. Retrieved December 2022, from https://www.saxion.edu/news/2022/may/the-h2020-serene-project-delivering-sustainable-andintegrated-energy-systems-in-local-communities

Saxion. (n.d.). Ambient intelligence. Hogeschool Saxion. Retrieved December 2022, from https://www.saxion.nl/onderzoek/smart-industry/ambient-intelligence

Sense. (n.d.). Sense Home Energy Monitor. Sense.com. Retrieved December 2022, from https://sense.com/sense-home-energy-monitor

Tesla. (n.d.). Mobile App Overview: Tesla. Tesla. Retrieved January 2023, from https://www.tesla.com/support/energy/powerwall/mobile-app/mobile-app-overview