Chef Bento Usability Test Report Ver 2
Table of Contents Testing Script Test Summary
1
Result Summary
2
Appendix Appendix 1. Test Script
7
Appendix 2. Answering Sheet
13
Appendix 3. Product Page
24
Test Summary Description: The usability test is conducted between November 29th-December 2st 2018 in similar settings. After each test, answers are recorded by using Microsoft Word.
Objective: Test Homepage Content: a)Does our user confuse by the layout of our homepage? b)Does our user confuse by the content of the search page? c)Did the filter page provide the right variable that the user wants to control? Test Product Page Content: a)Does our user have any preference on the page layout? b)Does our user find any of the page layout helpful for decision making? General Concern: By walking through a simple check out, does our user have any confusion?
Participants: Persona: People who desire Asian food and would like to cook for themselves but are in a lack of experience and time. Numbers of Participants: 5
Chef Bento Usability Test
1
Summary This test is considered as an agile testing given the scale of our sample. However, a few statistical conclusions wouldn’t hurt to facilitate the future development of this product. In that regard, any suggestions from our testee as well the statistical conclusions will have reference value after detailed consideration and discussion. This summary will be structured based on the sessions of the usability test script. General Information Although it is not feasible to apply this data in the following development regarding the size of the sample as well the information we are gathering, we did the following chart which could illustrate the expected cooling time and times of cooking per week in our samples.
Even Though the sample size is limited, we could still conclude that there is a universal need for white collar workers have limited time for cooking and a weekly basis need for cooking by themselves. Home Page Content This session includes my discoveries in Task 1, the usability of the home page content. 1. Navigation Due to the layout of the script, we only collect the comment from our samples. The homepage is quite straightforward according to their comment and none of our testees seems overwhelmed by our layout. Meanwhile, 80% percent of the participants have questions about the small rectangle on the top right corner of the interface. Since it is a function haven’t been fully explained in the landing page, we have to clarify that in the instruction. Chef Bento Usability Test
2
Other concerns focus on the appearance of the mobile app as well the meaning of a certain category, which would be considered rather insignificant in terms of usability. 2. Filter
The filter test session exposed a major design flaw: First, the original filter oversimplifies the dieting problem, while many of the participants mentioned their appetite and allergy scenarios. Some can’t stand spicy food, some have a certain allergy. Such information should be contained in the filter for safety reasons. Secondly, terminologies on the existing page are ambiguous as well. 2 people have questions on the special offers while the expertise level of the cooking is not stated as well. Also, 1 participant asked about the difference between lunch and dinner. Such unclear Although only one person in the sample point it out, it is important to bring up that the filter page itself could be very overwhelming to busy white-collar workers given that he/she still has to make choices after he/she inputs his/her preference in the filter. Product Page The usability of the product page, as fully tested in Task 2, is a session exploring how to make the best of the product page not only to promote the product but also help the customers to make decisions.
Chef Bento Usability Test
3
Although all people who could tell the difference between each layout, they share a similar interest of these pages. According to the results, the most appealing product page emphasis on the graphics part of the product, like the branding material and the product history. While on the other hand, the most helpful product page emphasis on the product stats. Since they are all concerned about their health condition. Additionally, one participant mentioned that the product comments from certificated purchase could also be an deciding issue in terms of their purchasing decisions. Check out Page This session is to report the usability issue we will have in the checkout process. However, The checkout page is very simple according to most people’s test. In that regard, no more description in this area. General Suggestion Other than the UX content of the test, many users express their dilemma as they have to make choices among so many options. Consequently, if a user’s diet preference is recorded, we could automatically generate a bundle of result suites their appetite and make appropriate decisions for them. Conclusion Although we produce graphs for statistical conclusion, we are still considering this more of an agile test. Therefore, the opinion of the individual counts in the optimization process. In the light of that statement, the following suggestion for future development of the app is concluded. 1. Quick Buy Function:
Chef Bento Usability Test
4
Quick buy function is loosely based on the filter function. When a customer’s diet preference is recorded, the system could source a limited number of weekly meal plan according to that preference. Currently this function won’t support text input. But we believe that in the future, with the improvement of artificial intelligence, our system could recognize any potential preference in a text box, which allows us a deeper level of customization. What’s more is that if the participants could provide feedback of this dish, the system could learn his more detailed preference, and recommend a more suitable meal plan for the customer. 2. Exclusive Recipe Page
Since the recipe is not on display in the product page anymore, a specific page where the customer could feel more intuitive cooking while following the instructions. To help the cooking process more effective, video playback, automatic timer and voice assistance may be added to this page. 3. Product Page Redesign Given the result of our usability test, the product page needs a major redesign where we emphasis on the branding material as well as the customer’s review. However, we won’t eliminate the other information we have on display during the test, specifically, the stats of different food. We will make it easy to find while we maintaining the branding material and the top customer’s review as our main body of the product page. Due to the scale of the interface, please see Appendix 3
Chef Bento Usability Test
5
Testing Improvement The test goes well as planned, and I have many interesting feedbacks. However, based on the the test we had and the question we prepared, it makes much more sense if we could conduct a remote usability test, which allows us to collect results from a larger more sample size. Meanwhile, even though the main issue of the website has been tested, there are still pages that haven’t been tested yet. If I still have time, I would also give the landing page as well the login page a try.
Chef Bento Usability Test
6
Appendix 1. Testing Script Test Session 1 Testing Homepage Content General Description: This Session design to gather general information from the participants and testing the usability of homepage and filter page. The content is shown to the participants on a prototype on a cellphone. The participant could interact with the prototype as he/she interacts with a normal mobile app.
Testing Script: Part I: Set Up Script: Hi, My name is Siyu, and I’m going to be walking you through this session today. Before we begin, I have some information for you, and I’m going to read it to make sure that I cover everything. You probably already have a good idea of why we asked you here but let me go over it again briefly. We’re asking people to try to look at series of mobile interface that we’re working on so we can see whether it works as intended. The session should take about 20 min. The first thing I want to make clear right away is that we’re testing the interface, not you. You can’t do anything wrong here. In fact, this is probably the one place today where you don’t have to worry about making mistakes. As you look at these interfaces, I’m going to ask you as much as possible to try to think out loud: to say what you’re looking at, what you’re trying to do, and what you’re thinking. This will be a big help to us. Also, please don’t worry that you’re going to hurt our feelings. We’re doing this to improve the site, so we need to hear your honest reactions. If you have any questions as we go along, just ask. I may not be able to answer them right away since we’re interested in how people do when they don’t have someone sitting next to them to help. But if you still have any questions when we’re done I’ll try to answer them then. And if you need to take a break at any point, just let me know. Do you have any questions so far?
Part II: Information Session Script: OK. Before we look at the interface, I’d like to ask you just a few quick questions.
Chef Bento Usability Test
7
Questions: 1. First, could you please tell us your occupation? 2. Thanks, how many time did you cook every week? 3. How long does it take for you to cook? Do you just do it for yourself? 4. Do you have any experience with a self-cooking product like Blue Apron? 5. If you do, could you please tell me a little bit about it?
Script: OK, great, thanks for your answer! We’re done with the questions, and we can start looking at our design.
Part III: Homepage Testing Action: Present the prototype with “Homepage” interface on the screen to the participant.
Script: First, I’m going to ask you to look at this page and tell me what you think of it. what strikes you about it, what you can do here, and what it’s for. Just look around and do a little navigation. You can scroll if you want to, but don’t click on anything yet.
Action: Ask the following questions, during the question session, participants are allowed to look at the screen for reference. Question: What’s your first impression of this layout? Do you think anything is confusing? Could you please describe what you could do from this homepage by just looking at it?
Script: Thanks for your answer. Now you could try to click anything you like, try to navigate through pages.
Action: Ask the following questions, during the question session, participants are allowed to look at the screen for reference. Question: Do you have any confusion navigating between these pages? Observation: Did the participant pause at any specific page? Chef Bento Usability Test
8
Action: Show the participant the filter page. Script: Now let's say you don’t have anything specific in mind to order. So you write Chicken in the search column, Now you run into this page to let your preferences filter a few options for you. Question: Do you have anything in mind that you want to add in the filter and it is not shown here? Do you have questions on the filter which is already on the filter page? Script: Thanks for your participation, That’s it for session 1.
Chef Bento Usability Test
9
Test Session 2 Testing Product Page Content General Description: This Session design to test the usability of product page content. The content is a series of screenshots of an interface the same product designed with a different emphasis on the content. It could be shown to the participants on a screen, on a mobile phone and even on print. The difference in content are: Emphasis on the inviting product Image; Emphasis on the product stats(Nutrient, Calorie, etc); Emphasis on product instruction; Emphasis on the story behind this specific cuisine.
Testing Script: Script: Hi, Thanks for your cooperation in the first session. Now for the next session, we are going to ask you to compare four different interfaces about the same content. After the comparison, we will ask you a series of questions based on your impression of the interface. Action: Present the different interface with the medium that is convenient to the test situation. Allow the participant to observe the material no less than 2 min. Participants won’t allow watching the content. Question: Could you please identify the difference between four different interfaces. Please describe the one that is most appealing to you. Please describe the one that is most helpful for you to make a purchase decision(Whether you buy it or not) Action: Present the interface again to the participant, explain the difference in a designer’s perspective, and ask the following question. Question: Thanks for your answers, it is very helpful to us. So for our designers, I will slightly explain the difference between these four interfaces. Could you please provide some general comment for what is necessary to be presented in a product page for you to make an informed decision? Other than what we have in the interface, what else do you think should be presented? Chef Bento Usability Test
10
Observation: Did the participant pause at any specific page? Script: Thanks for your participation, That’s it for session 2.
Chef Bento Usability Test
11
Test Session 3 General Comment General Description: This Session design to test the general usability of the prototype. The content is shown to the participants on a prototype on a cellphone. The participant could interact with the prototype as he/she interacts with a normal mobile app.
Testing Script: Script: Hi, Thanks for your cooperation in the second session. Since you are already familiar with the prototype, for the last session, we are going to give you a small task. Please try to locate the fish fillet with Cabbage, put it in the shopping cart, and check out. Action: Present the prototype with “Homepage” interface on the screen to the participant. If participants running into any difficulty navigating between pages, the tester is allowed to give hint. Question: I realize you paused at ( )Screen, do you have any difficulties in that page? Do you have any confusion in the check out page? Observation: Did the participant pause at a specific page? Script: Well, that’s pretty much about it. Thanks for your cooperation in the last session. Your feedback could really help us make better decisions in terms of design and user experience. If you have more questions, feel free to shoot me a note over email. I’m always happy to respond.
Chef Bento Usability Test
12
Appendix 2.Answering Sheet Participant 1 Test Session 1 Testing Homepage Content General Information: Could you please tell us your occupation? Design Intern How many time did you cook every week? I don’t have time to cook, but I always want to. How long does it take for you to cook? Do you just do it for yourself? If I could do it in less than 20 min, I would consider Do you have any experience with a self-cooking product like Blue Apron? No Home Screen, etc: What’s your first impression of this layout? Do you think anything is confusing? I think it is pretty clear, but I will need more context if you only show me this. What is this cursor like a rectangle on the top right corner? Could you please describe what you could do from this homepage by just looking at it? I could look at the different category of food, and browse through different ways of finding the stuff I want. Do you have any confusion navigating between these pages? Well, I think it is pretty straightforward. Observation: Did the participant pause at any specific page? No. Filter Screen: Do you have anything in mind that you want to add in the filter and it is not shown here? I don’t eat spicy food. Everytime when I bump into peppers, I don’t have the appetite to eat. Do you have questions on the filter which is already on the filter page? What’s the difference between lunch and dinner? Test Session 2 Testing Homepage Content Questions: Could you please identify the difference between four different interfaces. Well, I could tell one is like a normal branding page, one is the instruction, others have numbers or nutrient fact about food. Please describe the one that is most appealing to you. The branding one looks very cool(The Back Story one) Please describe the one that is most helpful for you to make a purchase decision(Whether you buy it or not) Obviously the nutrient fact page!
Chef Bento Usability Test
13
Could you please provide some general comment for what is necessary to be presented in a product page for you to make an informed decision? I think what you have is already very dope. But if I have to say anything, it probably doesn't have to be that long. Other than what we have in the interface, what else do you think should be presented? I can’t think of anything. Observation: Did the participant pause at any specific page? No Test Session 3 General Comment Questions: Do you have any confusion in the check out page? I think everything is very clear. Observation: Did the participant pause at a specific page? No
Chef Bento Usability Test
14
Participant 2 Test Session 1 Testing Homepage Content General Information: First, could you please tell us your occupation? Showroom Assistant Thanks, how many time did you cook every week? 30 mins a day, 5 days per week How long does it take for you to cook? Do you just do it for yourself? 30 mins for a single meal. Yes, I’m living alone. Do you have any experience with a self-cooking product like Blue Apron? Not really. If you do, could you please tell me a little bit about it? Home Screen, etc: What’s your first impression of this layout? Do you think anything is confusing? I like the contrast color which emphasizes the key stuff. Could you please describe what you could do from this homepage by just looking at it? I can search and I guess can click on one product to see its details. Do you have any confusion navigating between these pages? I don’t exactly know what will happen if I click on the navigation bar to see the “deals”. Observation: Did the participant pause at any specific page? No Filter Screen: Do you have anything in mind that you want to add in the filter and it is not shown here? Unlike many Asian people, I am allergic to different nuts. And I know that certain Asian cuisine contains a nut. Do you have questions on the filter which is already on the filter page? I am not sure what is the “Special Offer” means. Test Session 2 Testing Homepage Content Questions: Could you please identify the difference between four different interfaces. Yes, one is like the history of the product, one has instructions, one has the nutrient facts, the last one has a very beautiful picture of the product. Please describe the one that is most appealing to you. Definitely the one with the beautiful picture Please describe the one that is most helpful for you to make a purchase decision(Whether you buy it or not) The nutrient fact one. Could you please provide some general comment for what is necessary to be presented in a product page for you to make an informed decision? Chef Bento Usability Test
15
I think what you have is enough, but you probably have to figure out a way to put it together. Other than what we have in the interface, what else do you think should be presented? I think it is good enough. Observation: Did the participant pause at any specific page? No Test Session 3 General Comment Questions: Do you have any confusion in the check out page? I was thinking to add one more product after I filling out payment information. The only way I could do it was to return to the shopping cart, add one more product and redo the checkout process. I think that is not really efficient. Observation: Did the participant pause at a specific page? She spent some time trying to add one more product but she didn’t get stuck.
Chef Bento Usability Test
16
Participant 3 Test Session 1 Testing Homepage Content General Information: First, could you please tell us your occupation? Project Manager Thanks, how many time did you cook every week? About 3-4 times a week How long does it take for you to cook? Do you just do it for yourself? 70 mins for a single meal. Yes, I’m cooking for a family of 3. Do you have any experience with a self-cooking product like Blue Apron? I sometimes try that, but the recipe is too western. Home Screen, etc: What’s your first impression of this layout? Do you think anything is confusing? I think it is like a google app. Maybe I am too sensitive about that. I think everything is clear on this page. Could you please describe what you could do from this homepage by just looking at it? I can click on the product and see their details, also I could view different product using a different way of locating them.. Do you have any confusion navigating between these pages? What is the rectangle on the top right corner? Observation: Did the participant pause at any specific page? He pauses in the home page for a while. Filter Screen: Do you have anything in mind that you want to add in the filter and it is not shown here? Well, I don’t like sweet stuff for lunch or dinner. That’s too much. Do you have questions on the filter which is already on the filter page? What’s special offer? Test Session 2 Testing Homepage Content Questions: Could you please identify the difference between four different interfaces. Yes, One is about how to cook, other is about numbers, the rest two is about image. Please describe the one that is most appealing to you. I think it is the one with a lot of dish picture. Please describe the one that is most helpful for you to make a purchase decision(Whether you buy it or not) The one with clear nutrition numbers. Could you please provide some general comment for what is necessary to be presented in a product page for you to make an informed decision? I think what you have shown me is all important, but I mostly care about the weight and nutrient fact. Chef Bento Usability Test
17
Other than what we have in the interface, what else do you think should be presented? The instruction page is too long. Very tired to scroll down. Observation: Did the participant pause at any specific page? She watched the instruction page for a while. Test Session 3 General Comment Questions: Do you have any confusion in the check out page? I am not sure if I could change my address in the check out page. Observation: Did the participant pause at a specific page? No
Chef Bento Usability Test
18
Participant 4 Test Session 1 Testing Homepage Content General Information: First, could you please tell us your occupation? Data Analyst Thanks, how many time did you cook every week? 2-3 times a weeks How long does it take for you to cook? Do you just do it for yourself? 10 mins for some cold sandwiches. Yes, I’m single for now. Do you have any experience with a self-cooking product like Blue Apron? No I’m not really fond of western food while at the same time my cooking skills are bad. Home Screen, etc: What’s your first impression of this layout? Do you think anything is confusing? I like the contrast color which emphasizes the key stuff. Could you please describe what you could do from this homepage by just looking at it? I can search and I guess can click on one product to see its details. Do you have any confusion navigating between these pages? I don’t exactly know what will happen if I click on the navigation bar to see the “deals”. Observation: Did the participant pause at any specific page? No Filter Screen: Do you have anything in mind that you want to add in the filter and it is not shown here? I would say a filter showing spicy levels like mild spicy, medium spicy and extra spicy would benefit me. Do you have questions on the filter which is already on the filter page? I’m not really sure what exactly “beginner” “intermediate” “expert” level means. Because they are just relative standards and one cannot measure that 100% objectively. Test Session 2 Testing Homepage Content Questions: Could you please identify the difference between four different interfaces. Of course, there is one with a history of the food, one with very nice pictures, I assume is about how you could cook it, and there is another one with numbers about this food, as the nutrient fact, weight, the number of peoples it could serve. And there is the last one. I believe it is with the cooking instruction. Please describe the one that is most appealing to you. Definitely the one with the nice picture of the food. Please describe the one that is most helpful for you to make a purchase decision(Whether you buy it or not)
Chef Bento Usability Test
19
The cooking instruction works for me. Because I am really bad at cooking so I want to check in advance if I can handle the cooking process. Otherwise, the product makes no sense to me. Could you please provide some general comment for what is necessary to be presented in a product page for you to make an informed decision? I would like to see some top comments from other users or even my friends so I can know more about the true product instead of hearing all the advertisements. Other than what we have in the interface, what else do you think should be presented? No. Observation: Did the participant pause at any specific page? No Test Session 3 General Comment Questions: Do you have any confusion in the check out page? No. But I think the “+” “-” buttons for product amounts is a little bit tiny. Observation: Did the participant pause at a specific page? No.
Chef Bento Usability Test
20
Participant 5 Test Session 1 Testing Homepage Content General Information: First, could you please tell us your occupation? Digital Artist Thanks, how many time did you cook every week? Everyday How long does it take for you to cook? Do you just do it for yourself? Depends on my schedule, Usually, it takes about 20-50 min. Yes, I am cooking for myself, but sometimes I cook more so that I have leftovers for the next meal. Do you have any experience with a self-cooking product like Blue Apron? I heard about it, I always want to try. But the subscription stuff is killing me. Home Screen, etc: What’s your first impression of this layout? Do you think anything is confusing? I think it looks very much like the chalkboard where you put all the menus on. I think many restaurants in my hometown has the chalkboard to write the menu. Could you please describe what you could do from this homepage by just looking at it? I think I could see a lot of details of a simple dish. Do you have any confusion navigating between these pages? What’s the rectangle right next to the search column. Observation: Did the participant pause at any specific page? No Filter Screen: Do you have anything in mind that you want to add in the filter and it is not shown here? Well, I don’t think it needs anything. But if I have to say it, it is too complicated for me to select too many stuff in the filter. I like things in a simple way. Do you have questions on the filter which is already on the filter page? I think they are all very confusing. I don’t like so many choices. Test Session 2 Testing Homepage Content Questions: Could you please identify the difference between four different interfaces. Yes, there is one about the history of the food, one with nice pictures, one with nutrient facts, and the other with instructions. Please describe the one that is most appealing to you. I really like the one with nice pictures. Please describe the one that is most helpful for you to make a purchase decision(Whether you buy it or not)picture Of course the nutrient fact one. I am a very healthy person. Chef Bento Usability Test
21
Could you please provide some general comment for what is necessary to be presented in a product page for you to make an informed decision? I think that’s enough. But all the information is kind of important to me if I have to make a decision. Other than what we have in the interface, what else do you think should be presented? You already think of everything. Observation: Did the participant pause at any specific page? No Test Session 3 General Comment Questions: Do you have any confusion in the check out page? I think there is not any thing is clickable. But everything is very clear. Observation: Did the participant pause at a specific page? No
Chef Bento Usability Test
22
Appendix 2. Product Page
Chef Bento Usability Test
23