2008 UX Findings by Curt Odar

2008 Web User Experience Findings Contact Rebecca Sherrill or Curt Odar for more information about these projects

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Learn how you can apply the findings in this book to your web projects.

Topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 How can we best enable members to find information by Topic on the Middle Market websites? Project Summary & Outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 Detailed Process / User Quotes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

Taxonomy Redesign: Card Sorting . . . . . . . . . . . . . . . . . . . . 34 If members were to organize the content on our sites, how would they do it? What terminology would they use? Do our existing taxonomies make sense to our members, or should they be revised?

Home Page Redesign . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 How can we redesign our homepage to better meet the needs of our members, providing (a) navigation, (b) personalization and (c) engaging, updated content worth returning for. Project Summary & Outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .36 Detailed Process / User Quotes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

Title & Abstract Testing . . . . . . . . . . . . . . . . . . . . . . . . . . 45 How do members consume content on our websites? Based on the titles and abstracts, how do they decide which documents to download and which to skip over? How can we optimize this experience? Project Summary & Outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .45 Detailed Process / User Quotes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

Workplan Generator . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 How can we design a new interactive tool on the website to best meet the needs of our members? What features would they use, and how would they apply it to their jobs? Project Summary & Outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .79 Detailed Process / User Quotes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

Tab Labels/Navigation . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 Is our new navigation structure intuitive? What are the best labeling options? Project Summary & Outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 Detailed Process / User Quotes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

Topic Center Templates . . . . . . . . . . . . . . . . . . . . . . . . . . 98 What consistent format should be adopted by Topic Centers? Project Summary & Outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 Detailed Process / User Quotes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

New User Login Process . . . . . . . . . . . . . . . . . . . . . . . . . 103 Why do so many members not complete the initial registration process? How can we improve the situation and give them a positive first experience on the website? Project Summary & Outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 Detailed Process / User Quotes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

Next Steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125 In the past year, we have begun to implement user-centered design methods, but there are more steps that we can take in 2009 to fully benefit from this work. The Corporate Executive Board

Introduction User Experience principles and practices help us understand how our content and design decisions provide value to our web users. The projects in this book were conducted primarily for the Middle Market member-facing websites, focused on improving web user experience. Using user-centered design methods, we were able to prioritize and design enhancements according to the members’ needs and aptitudes, instead of making assumptions about what was needed or how they’d use it.

How to Use This Book The findings in this book are not specific to Middle Market websites or Middle Market users. They can be applied to other initiatives you may be working on. For example, findings in this book may be used as input to the creation or redesign of Decision Support Centers, stand alone web pages, tab naming, writing titles and abstracts, and many other initiatives.

Why use User-Centered Design Methodologies? UCD methodologies inform our judgment; they do not give us a “right” answer. However, we can have confidence in our design when we have benchmarked other sites, researched and identified user experience (UX) best practices, and our testing results have confirmed both. Our recommended process to achieve this is below.

Content is King Keep in mind that the design of a website is only part of the picture. Its success will depend on the quality of the content. A perfectly well-designed, usable website will not be successful if the quality or presentation of the content is poor. Some of the projects highlighted in this book tested both content and design, while others focused more on design attributes.

Recommended Design Process Depending on the project size or type, you may or may not want to use all of the methodologies listed below. Contact Rebecca Sherrill or Curt Odar if you would like feedback regarding your design approach. 1. Research Identify your goal and target audience. The goal of most projects or web enhancements is to solve a problem, satisfy an unmet need, or improve upon an existing design or process. Get a full picture of the current situation (and audience), using as much information as you have access to. Examples include: a.

Web analytics

Example: As a first step to changing our navigation structure (pg. 84), we checked to see which areas of the site receive the most traffic.

b. Discussions with stakeholders

Example: When redesigning our new user process (pg. 100), we talked in detail with Account Directors from Member Services to identify current challenges and get a better picture of the processes already in place.

Discussions with members

Even before usability testing, interviews with members can provide more context for an existing problem or unmet need.

d. Survey data

Example: Before redesigning the homepage (pg. 35), we looked at a combination of web analytics and past survey data to gain a better understanding of members’ main intentions when coming to the website.

Usability testing of current state

If you’re improving on an existing page or functionality, test the current state with users to identify areas of

The Corporate Executive Board

improvement and allow a “Before & After” comparison after the enhancement has been made.

2. Competitive Analysis Most of the challenges we face have been encountered by dozens, if not hundreds, before us. Invest some time in browsing websites to see what approaches have been taken by others. Example: When redesigning our new user process (pg. 92), we explored the new user processes for other websites by registering for accounts on LinkedIn, Facebook, Mint.com, and The McKinsey Quarterly. For each website, week took screenshots of the page flows and emails and did a comparison, identifying strong and weak areas.

3. UX Best Practices There is a wealth of information online about user experience best practices that have been thoroughly studied, tested, and are accepted as a standard. These best practices cover everything from writing email subject lines to designing forms to methods of displaying information. Some websites to visit are: a. User Interface Engineering Browse/search the free articles or purchase an in-depth report. b. UseIt Browse/search the free AlertBox columns or purchase an in-depth report. c.

Usability.gov: Research-Based Guidelines Usability.gov offers a 262 page book (in PDF format) covering guidelines for home pages, layout, navigation, links, text appearance, graphic design, accessibility, search, and more.

d. Boxes and Arrows Though this website primarily contains articles about the processes and theory of User Experience, a search through the archive will often return very useful recommendations. 4. Design Use steps 1, 2, and 3 to inform a design. If possible, make a variety of designs and work in a group, rather than having a single person design alone. 5. Usability Testing Once you have a design (or multiple versions of a design), start showing it to people. Examples of usability testing processes and scripts can be found throughout this book. After you’ve tested your design with 5-7 users, make adjustments based on your findings and test again. Usability testing works best as an iterative process should be repeated as many times as possible throughout your project. You can start with a very high-level design drawn on a piece of paper, or a graphic mockup, and eventually be testing with near-finished working prototypes. Note that usability testing informs the design process, but it does not provide you with “the answer” or create a completed design in itself. This is why the iterative testing is crucial. 6. Measuring Impact The final step in the process is to measure the impact of the project. a.

What defines success? Identify what exactly will indicate that your project was successful or that the goal was achieved. This will allow you to return to completed projects, identify what was successful/what wasn’t, and apply lessons learned to future projects.

b. Make it measurable If possible, build your design/pages in such a way that activity and traffic can be tracked. For more information, visit the User Experience Design boardroom: http://boardwalk2/sites/UserExperienceDesign/ The Corporate Executive Board

Topics

User Experience 2008

Topics January - March 2008

Project Summary Purpose Based on informal feedback from members and internal CEB teams regarding difficulty in finding resources efficiently through the website, we explored the concept of “Topic Centers,” which would allow members to browse content by subject or category instead of type.

Process Concept Testing: Dec 5 – 14 Concept testing allowed us to gauge the level of interest among members about the idea of Topic-based navigation and get feedback on presentation styles and terminology. • Interviewed seven MLR members with a range of experience using the website • Testing was conducted over the phone for 30 minutes each, using GoToMeeting to show comps and collect feedback • Each session began with a short interview to learn about how the members currently use the site and find information • Profiles of each member were compiled before each session based on utilization, web user profiles, and peer group membership. These profiles were used to customize our script and presentation deck for each member. Usability Testing Round 1: Dec. 27 – Jan 3rd Based on our findings in concept testing, we had solid ideas about terminology and the display of information. The next step was to test the interaction – how do the users navigate to and within the information? Are they able to find what they need? • •

Conducted 45-minute sessions with seven users (5 members and 2 non-members) Testing was done in-person, allowing users to test interactive prototypes. Users were given tasks to navigate into the Topic Center from the home page and find a specific piece of information. Additionally, they were asked to research a topic of their choice.

Two prototypes were tested but neither were intuitive for the users. They found the existing left drill-down navigation in our Research tab to be more intuitive.

Usability Testing Round 2: Feb 11 – 15  Conducted 45-minute sessions with seven users (3 members, 4 other who work in HR)  Testing was done based on content from the HR Executive Forum website  Sessions took place in person at a computer using a prototype: o Beginning at the home page, users were asked to navigate to research a specific topic. The Corporate Executive Board

Topics

User Experience 2008

Using 2 different variations of topics page, users were asked to perform tasks involving filtering and sorting.

Two prototypes were tested, containing different versions of the same content. Users paid more attention to the featured content when it was presented in a list format with shaded background (Google search results style).

Design & Implementation After testing was completed, the project continued until its launch in August as we created the visual design and interaction details. All Middle Market research teams participated in an extensive effort to tag their content to topics for this new framework. Additionally, most programs re-structured their taxonomies (See chapter on Card Sorting for more information)

Summary of Findings & Recommendations • • •

Users struggled to use original proposed Topic Center navigation ideas (3 column and drop-down menus). In the final design, we opted to re-use a standard left drill-down navigation scheme that was already familiar to them. The main challenge that users faced when using the Topic Center interfaces was caused by items being categorized or labeled differently than what they had in mind. As a result, we worked with programs conduct card sorting exercises with members and then evaluate/restructure their taxonomies. The concept of a Topics tab was very intuitive to users and they were able to successfully use our prototypes to find information in the second round of usability testing.

Outcome The most important outcome is that we did not spend time and resources building the original proposed design, which was shown to be ineffective when it was tested with members. The Topics tab was launched in August 2008 as part of a navigation redesign. 5 of 7 Middle Market programs conducted card sorting surveys with members and restructured their taxonomies according to feedback received. Measurements of Success Through analytics, we’ve seen a decrease in searching and an increase in browsing the Topics pages, which indicates that the new navigation paradigm is easier to understand than the previous one. Previously, an equal amount of users searched and browsed. Now, browsing happens twice as often as searching. The ultimate measure of a successful visit to the website is a repeat visit. In 2008 we saw repeat visitors increase by 8% compared to 2007 across all Middle Market programs. This is a measurable indication of the positive impact made by enhancements such as the Topics redesign. Overall Benefits Having our content organized by Topic provides benefits in itself: It allowed us to implement customization in later projects (Overview Page Redesign, New User Process) by having members choose topics that they’re interested in and then pushing content recommendations accordingly. A single taxonomy for all content on the website is easier for research teams to manage. The Corporate Executive Board

Topics

User Experience 2008

Round 1 Usability Testing Report January 11, 2008

Background The purpose of exploring the idea of a Topic Center is to allow members to browse website content by subject or category instead of by type. Example:

I’m a member of HREF and I want to get information on Employee Retention. I don’t know or care what format this information comes in (Research? Webinar? Tool or Template?). I don’t have time to read through hundreds of search results. Instead, I would like to quickly access the best and most relevant content that HREF has to offer on this topic in one central location.

Based on informal feedback from members and internal CEB teams regarding difficulty in finding resources efficiently through the website, we explored the concept of “Topic Centers,” destination resources that would be easy to find on the website and would contain the best content we had to offer on a particular topic.

The Corporate Executive Board

Topics

User Experience 2008

Testing Process Concept Testing (Dec. 5 – 14) Concept testing allowed us to gauge the level of interest among members about the idea of Topic-based navigation and get feedback on presentation styles and terminology. • • • • • •

Interviewed seven MLR members with a range of experience using the website Testing was conducted over the phone for 30 minutes each, using GoToMeeting to show comps and collect feedback Each session began with a short interview to learn about how the members currently use the site and find information Profiles of each member were compiled before each session based on utilization, web user profiles, and peer group membership. These profiles were used to customize our script and presentation deck for each member. See testing script: Appendix A See testing Powerpoint slides at X:\UserTesting\ConceptTesting12-07

Usability Testing (Dec. 27 – Jan. 3) Based on our findings in concept testing, we had solid ideas about terminology and the display of information. The next step was to test the interaction – how do the users navigate to and within the information? Are they able to find what they need? • • •

• •

Conducted 45-minute sessions with seven* users (5 members and 2 non-members) Testing was done in-person, allowing users to test interactive prototypes. Users were given tasks to navigate into the Topic Center from the home page and find a specific piece of information. Additionally, they were asked to research a topic of their choice. Pages tested included: o MLR home page with a list of topics (selecting a topic takes the user into the Topic Center) o Expanding drop-down menu for quick access to topics and sub-topics, which would be accessible from all pages in the website (see Amazon.com’s new menu style) o Access to the Topic Center from search results page o Topic Center 3-column interface, using short questions and bolded key terms o Landing pages for both high-level topics and sub-topics See testing script: Appendix C See testing prototypes: Appendix D

*5-8 users is the recommended amount needed to discover the majority of usability problems. See Jakob Nielsen’s Why You Only Need to Test With 5 Users

The Corporate Executive Board

Topics

User Experience 2008

Highlights from Findings Topic Center Finding

Recommendation

Users struggled to use proposed Topic Center navigation (both 3 column and drop-down menu). The navigation in the Research tab received more positive feedback.

Use a more standard method ofDo not introduce a new method of navigation, but re-use one that our users are already comfortable with, such as the left drill-down menu drill-down used in Research section. In our testing, users found this type of menu to be very intuitive, while they needed to “figure out” how to use the Topic Center menus.

The main challenge that users faced when using the Topic Center interfaces was caused by items being categorized or labeled differently than what they had in mind. 3 out of 7 could not complete Task #2 because they couldn’t figure out where the information would be categorized. Those who did complete it expressed confusion over where it was located.

Each program needs to create a method of categorizationdefine a taxonomy that resonates with their members. A card sorting survey is recommended to get input on how to organize topics. Looking at common search terms and standards (what do other similar websites and companies use?) will give insight into labeling.

Search Finding

Recommendation

Nearly all users rely on Search at some point during their visit to the website. Users who are primarily “browsers” noted that if they couldn’t find something, they would try using the search box before giving up. Search is usually the last place users look before giving up and leaving the site.*

Provide as much value as possible on the search page itself, recognizing that this is a page nearly everyone uses (and some use exclusively): • Continuously monitor top search terms and map them to most recommended documents, tools, webinars, and peer groups. Clearly mark them as recommended so that users can get the same value out of Search as they would from landing pages that highlight specific offerings. • In addition to sorting by date, allow users to sort by relevance, popularity, and recommended items (as they can within Research)

4 out of 7 actively used the “Refine your Results” options on the Search Results page Move “Refine your Results” box to left of page to make it more prominent. In the future, possibly allow “Refine by Topic” as well so that browse and search behavior could be more easily intermingled. This is a web standard (see Amazon, Yahoo! Shopping, Forrester, etc). . 3 out of 7 users performed multiple search queries in a row in order to refine their results or find a larger number of results

Explore error correction (“Did you mean to search for __?”) and suggested search terms (synonyms), especially for searches that return no results because the user uses terminology different than what is being used on the website.

The Corporate Executive Board

Topics

User Experience 2008

Other Finding

Recommendation

4 out of 7 users didn’t notice the left green subnavigation within Research pages

Update style so that the menu stands out more. Make it obvious that the items in the menu are links that expand and that they are related to content on the page.

When reading summaries or the first page of a document, 4 users had difficulty knowing if they found what they were looking for. In Concept Testing, one member said that he usually prints out multiple documents and has to go back to them later to see if any were really what he was looking for.

Each program needs to create a method of Summaries and Table of Contents need improvement. Search results should show highlighted terms from document, not just from summaries. This is especially important for long documents because users might be missing out on contact content that is deeply buried.

With the exception of 2 users who search exclusively, most users expressed interest in having the ability to browse for more high-level of information and search for more specific information.

Always provide options to search and browse. Whenever possible, incorporate the two so that the user can easily switch between them (for example, after browsing to a topic, “Search within this topic”)

*Analytics report confirms that Overview page and the Search Results page are consistently (across all programs) the last pages viewed before a user exits the site.top 2 exit pages on the websites (11% of visits end on the Overview page, and 11% of visits end on the Search Results page)

Usability Testing Note: To read the full script, see Appendix C. To view prototypes, see Appendix D.

Why do you visit the website?

User 4

Specific information or to browse webinars/meetings

User 5

90% of the time she’s interested in upcoming webinars/meetings

User 6

To research a specific topic

User 7

So far just to browse; has not been looking for something specific

When you first enter the website, what do you do to find what you’re looking for?

User 1

Searches. He always finds plenty of information but has trouble figuring out which search results are relevant. He prints out multiple documents and goes back later to review them and see if they contain relevant information.

User 2

Browses Webinars every month to see what’s relevant to her and her team. Searches when she wants to have information on a specific topic. Sometimes browses Research and Tools & Templates tab.

User 3

Primarily browses. Often looks under tabs for Webinars to see what’s coming up. Other tabs he visits are Tools & Templates and Research.

User 4

Searches for specific information; browses Webinars and Meetings

User 5

Identifies herself as more of a browser but uses search a lot for specific keywords

User 6

Searches but specifically wants Research

User 7

Browses Webinars and Tools & Templates, has never searched

The Corporate Executive Board

Topics

User Experience 2008

What do you do when you can’t find something?

User 1

Will look at the Overview page and tabs to see if anything catches his eye. At that point, might try Google.

User 2

If search is unsuccessful, will go into Tools and look for a category that’s similar for her search term.

User 4

Leaves site

User 6

Goes to Google

Thoughts on Summaries and Descriptions

User 1

“The titles don’t always grab me, so I read the summary. And if something in there grabs my eye, then I’ll download it.”

User 2

“I don't know if they know that ‘blog’ is a weblog. Do a 3 sentence description on what a blog is. There are some older marketers who can use a website but might not know what these emerging tools are. Just for the stuff that isn't obvious. Even CRM -- it's my life, but a lot of people don't know that that means Customer Relationship Management.”

User 3

“I do like how there's an explanation of what the individual pieces are and you don't have the room to do that in the first version. At this point I’m really interested, I’ve found my topic, so I want to know more about what those 3 things are. I’d probably still vote for the faster navigating down in the first one. Can you do both?”

Finding

Recommendation

Users like to see explanations and descriptions

Display concise, useful descriptions with links when possible

Descriptions vs. Pop-Ups Do you prefer to see descriptions under links within the page, or have them appear in a pop-up when you hover over the link?

User 4

Descriptions Thinks pop-ups are a good idea but drive her nuts because they cover things up

User 5

Pop-ups

User 6

Descriptions Thinks the pop-ups are too easy to miss/overlook

User 7

Pop-ups Likes that you can choose to see the descriptions only if you want to

Home Page: Users 1 – 3

Which layout do you prefer to see on the MLR home page? Option 1: List of topics (see Appendix B, Fig. 1) Option 2: Drop down menu tool Option 3: “Challenges” layout (see Appendix B, Fig. 3)

User 1

1. “I like the first one. If I’m going to search, I’ll just go to search.”

User 2

1. “I like the first one best by far. It’s very clean and I can see what the topics are and then I know what to expect because I’ve been on the site so much.” Also saw value in 3. “I almost see Top Challenges as it could be its own tab or under the Issue Center. Instead of Branding, it could be Top Marketing Challenges. Branding is a topic; Top Challenges in Marketing are also a topic.”

User 3

1. “I know I have a preference for #1. I think because when I’m coming to the website more often than not I 'm looking for something specific I need help with and 1 gets me there the fastest. I tend to like to see everything at one time vs. having to use drop downs. The #1 option is easier for me to say if I am going to look for something in Planning & Budgeting then I also see there's Organization Design without having to click anything. “#3 is probably how I should

The Corporate Executive Board

Topics

User Experience 2008

be using the site -- I should learn more. But typically my visits are less to go and explore and more to find something specifically and #1 does that the fastest.” Note: After first round of testing, comps were updated based on the unanimous feedback. Home Page: Users 4 - 7

Which layout do you prefer to see on the MLR home page? Option 1: List of topics (see Appendix B, Fig. 1) Option 2: List of topics broken up into 4 categories (see Appendix B, Fig. 2) Option 3: “Challenges” layout (see Appendix B, Fig. 3)

User 4

2. Also interested in 3 because she relates to one of the challenges. Thinks 1 and 2 are more appropriate for overview page because they’re more broad; 2 is more visually pleasing.

User 5

2. Likes the idea of them being grouped together; easier to read when broken up and not a long list. Also likes 3 visually.

User 6

2. Easier to understand

User 7

2. Easier to understand and use, “neater”

Would you access Topic Center from search results page? User 4

Probably not, would just look at search results but doesn’t feel having the option there would hurt

User 5

Probably not, would use the route she’s most accustomed to.

User 6

Yes, thinks it would help direct her to good content

User 7

Probably if it caught her eye

Finding

Recommendation

Users seem unsure if they would want to access a Topic Center from the Search Results page.

Followed up with this topic in Usability Testing

Questions vs. Statements Do you prefer to see sub-topics represented as questions or statements?

User 1

Questions “Oh yeah, now this is clear. I would have clicked on that [question], but with the previous title, I wouldn’t have drilled down any further.”

User 2

Questions “I like this better. But then again, I’m one of those people who likes words. When you put one word up there, it leads people to their own discretion as to why they'd go there. When you put a sentence, at least for me, it actually gives them a reason to go there. If it says online marketing, I may or may not go there, but if I see the question, that's a different thought process you're sending towards me. The only problem is there are quite a few words there.”

User 3

Statements “The question are clear and make sense but if I was going though quickly I’d be overwhelmed by the amount of words and feel distracted” Statements Less is more; those who are less experienced may want questions but she feels that less text = less work

User 4

User 5

Statements Prefers statements but thinks that questions would be helpful if it was a topic she didn’t know much about

The Corporate Executive Board

Topics

User Experience 2008

User 6

Questions Doesn’t have to think about it and it’s easier to understand

User 7

Questions More information

Bookmark Individual Items, Topics, or Both

User 2

Items. “From a practical standpoint, I’d probably prefer for it to be more granular because I only send people things I’ve read. If I send full sections to them, I’d be concerned they'd dismiss it as a whole book to read. I’m more likely to ask them specifically about an article.”

User 3

Items. “I’d say individual tools and documents. I probably wouldn't use that as an immediate resource and seeing the exact resources would be more helpful.”

User 4

Items

User 5

Items

User 6

Items

User 7

Both

Finding

Recommendation

5 out of 6 users thought they would only want to bookmark individual items rather than topics.

Do not add functionality to let users bookmark a topic.

Where do you go first when looking for information on the MLR website? User 1

Would go to Research Tab first, then Meetings and Webinars, then Anatomy I would use search, but first would try to come in through a tab.

User 3

Drawn to tabs, then looks at green box for new info. If I was searching something, would go to the search box.

User 4

Goes to Tools and Templates a lot. Terminology is different from his own, which is a challenge. Tends to browse and does not use search much.

User 5

User Would click on Research Tab first -- “There is a lot of stuff on here.” All tabs make sense except Develop Team expertise and Anatomy Center – don’t know what that means

User 6

Would probably go to Tools and Templates to find information first.

User 7

Would go to Search first because it’s trained behavior and he usually has an intent. He wonders why search isn’t more prominent

What do you do when you can’t find what you’re looking for?

User 1

After trying search, would then email her rep for help finding something

User 2

Might go to Google or other specific resources: grc.com such as , Pragmatic Marketing, and search within their sites.

User 3

References some basic marketing books, marketing profs, marketing sherpa, and keep a best practices folder on hard drive.

User 6

Would first go to search within the site, then would go to Google.

Task 1: Starting from the blue “Topic Center” box on the home page, find information on Launching a Product. The Corporate Executive Board

Topics

User Experience 2008

To view these pages, see Appendix D, Fig. 1, 3, 5

User 1

• Clicks New Product Development and arrives at 3 column layout. • 3 column layout - Reads the questions, then says depends on which stage she was in that would lead her to a question…reads through the three options, then goes to the development toolkit • Was expecting a different layout. ”It took me a little bit longer to grasp these questions because the location was not next to New Product Development.” • Top level topics – some confusion over how these would overlap with each other (ex: under Traditional Marketing, would “wonder what that was since other topics are within traditional marketing”) • Would like to see member-submitted recommendations and tools under Vendor Management • Submit question to peer group – Concerned that she “may or may not hear back,” but it would be great to know that account rep will be getting it as well • View all content behaves as expected

User 2

• Reads through the topics and chooses Marketing Strategy. Looks at select topic options…thought first column was an offshoot of Marketing Strategy. Reads through column 2 options… didn’t find it, and would go back to Branding, Messaging and go to market strategy…starts going one by one but still has trouble finding it. Goes to online marketing to look there…says sales and marketing integration is interesting to her…never found content on launching a new product. • Would like to see tools for the generation of collateral- none of the topics point her to that • Likes the question and answer feature…ask a question to a peer • Would go to ask a peer, then would go to search…would be great if the landing page would point to external sources as well.

User 3

• Takes a few seconds to read through the categories and sees a few different areas it could be….chooses marcomm, then says that wouldn’t be it, then chose marketing strategy, not successful, planning and budgeting, not successful, chooses branding, not successful, new product development…yes…I guess that’s where…it would be under new product development…but that seems more like the research side on whether there is a market need…thought of launch more of a campaign launch, press releaseesreleases, advertising, etc • Format is not bad, but could be stronger. I don’t think it would take me long to figure this out. • I might have a webinar or a best practice listed here…clicks into more…says that is exactly what he would expect…anything that connected back to the networking would be good.

User 4

• Looks in Marketing Strategy, doesn’t find it, picks New Product Development, clicks through the 2nd column, finds what he is looking for • It would be helpful to see how you’d see columns 1 and 2 fit together – not steps in a process, but rather a grouping…maybe visually…points to a chart on pragmatic marketing framework on his cork board. • 3rd column -- are these tools, are these best practices? Reads through each summary but unclear to him…clicks through the third column….would expect that to be a full set of content on that particular topic. • For ask a peer, is unclear which peer group it would go out to.

User 5

• 3 column layout not what was expected - Expected that she would have already been in Online Marketing – thought the first column of topics would be within Online Marketing. • Prefers topics to questions in column #2. • Expected to find search box under column #2 (if her question was not there) • Column #3 – expects to find topic papers, q&a, research, external google-esque type stuff • Landing page is what she expected. Having search on that page is great

User 6

• After taking a few seconds to read list, goes to New Product Development. Clicks through to 2nd column, chooses bringing my new product to market….goes to 3rd column. Depending

The Corporate Executive Board

Topics

User Experience 2008

on stage or project, would click on one of the items…execution tools, timelines. 3rd column would expect to find and find helpful a step by step tool on how to launch a product, or a research article or something like that…would like articles or graphs on how it would work. • Didn’t see the link to view all content on this topic. Landing page is what was expected. • “Can’t Find What You’re Looking For” boxes behave as expected. User 7

• Thinks “New Product Development” but is not sure. Clicks on “Marketing Strategy,” wonders if they’re the same • Finds rollover (top drop down) annoying • Goes back to New Product Development- Finds Launching a Product but feels the topic is mislabeled. • Content in the solution column should be common ways, checklists, top 10 things, etc • Would like to see more “lively” text. “New Product development toolkit” is boring and doesn’t say what a toolkit is (thinks it should say something like “10 great secrets”)

Finding Navigation of 3-column layout is not intuitive. 4 users expressed confusion over how the columns functioned and related to each other

Recommendation

The fact that were categorized or labeled differently than what users expected hindered their ability to use this 3-column layout because they had to backtrack and make decisions between topics that seemed to overlap.

Again, each program needs to create a taxonomy that resonates with their members in order for any navigation structure to be successful.

Use a more standard method of navigation. The 3 column layout may work well somewhere else, but it’s not effective here.

Task 2: Starting at the Topic Center orange drop-down menu, find information on “Word of Mouth Marketing.” Then search for “Word of Mouth Marketing” (Topic Center button appears in Search Results page) To view these pages, see Appendix D, Fig. 2, 4, 5

User 1

Using Drop-Down Menu: Did Not Succeed Looks under Communications, Marcom, reads through subcategory questions but can’t find it. Wants to go search instead. Using Search Looked past green Topic Center button, directly to first search result. Thought the button would take too much time. Would expect to find several topics in that topic center – not specific to what she was looking for

User 2

Using Drop-Down Menu: Succeeded Looked under Branding, External Communication, Strategies, sees many sub-topics that it could be under. Surprised to find it under Emerging Media. Using Search Noticed highlighting first, saw the Topic Center button but didn’t click it. Reads through titles first, then dates, then scrolls back to top.

User 3

Using Drop-Down Menu: Succeeded Scrolls through each topic, reads subtopics, stumbles at M&A (Mergers & Acquisitions or Marketing & Advertising?). Looks under Marketing Strategy, Communications, thinks that Word of Mouth Marketing could fit under many different areas. Finds it but would not categorize it there (thinks of Emerging Media as technology focused) Using Search Scrolls through entire page, Selects a link next to the Topic Center button. Also filters results.

The Corporate Executive Board

Topics

User 4

User Experience 2008

Using Drop-Down Menu: Did Not Succeed Scrolls through topics, chooses Communications. Sees content and notes “I would still have to search to find word of mouth” while scrolling through the page. Using Search Clicks on Topic Center button/links without really reading the content

User 5

Using Drop Down Menu: Succeeded • Diidn’tDidn’t notice the orange widget from home page…when guided to it took several seconds to search through list, but found topic page. • About the widget: liked that she didn’t get confused on the first column…likes the interaction that she doesn’t have to click on the top topic level • Clicked on the first level topic because she wasn’t sure if there was another menu that would show up by clicking – but didn’t think it made sense that clicking there would bring her to another topic page • Does not get M&A…”marketing and advertising”? Search Glanced at green Topic Center button but skipped over it because she saw it as an ad and automatically ignored it. “They must be talking at me and let me get to what I want” “Word of mouth seems completely unrelated to emerging media” (the topic center that is shown in the green box on the search page)

User 6

Using Drop-Down Menu: Did Not Succeed Scrolls over topics and reads subtopics (Communications, Marcomm, back to Communications). Gives up and goes to the search box. Using Search Doesn’t understand what the links are next to Topic Center green button and would scroll down.

User 7

Using Drop-Down Menu: Succeeded • Feels it should have a 2.5 second delay so if he scrolls over accidentally, it doesn’t appear • After 20-30 seconds of looking at the topics, he finds it. Did not expect to find it here. • Likes drop down interface more but feels people won’t see it. If users are trained to see that method, would prefer that Using Search Did not comment or click on green Topic Center button.

Finding

Recommendation

Users struggled to use proposed Topic Center navigation (both 3 column and drop-down menu)

Use a more standard method of navigation, such as the left menu drill-down used in Research section. In our testing, users found this type of menu to be very intuitive, while they needed to “figure out” how to use the Topic Center menus.

Each program needs to create a method of categorization that resonates with their members. A card sorting survey is recommended to get input on how to organize topics. Looking at common search terms and standards (what do other similar websites and companies use?) will give insight into labeling.

Only 2 out of 7 thought they might click the Topic Center button from the Search Results page. Others skipped over it because it looked like an ad, seemed like it would take "too much time," or they were more interested in seeing search results.

When incorporating Topic Center or any other feature of the website on the search results page, display it as an actual search result or users are likely to overlook it.

The Corporate Executive Board

Topics

User Experience 2008

From the Overview page, users preferred to see a list if all topics as links which they could click into. They preferred this format over drop-down menus (In Usability Testing, only 1 out of 7 used the drop-down menu after seeing both options). This layout was also strongly preferred (7 out of 7) in Concept Testing.

When topic-based structure is available, make it very visible and accessible on the home page. No content should be hidden under menus because users like to see the entire range of subjects in one glance.

Task 3, Part 1: Select a topic that you would like to research. Find information using the Topic Center, Anatomy tab, Research Tab, and Tools & Templates tab.

User 1

Topic: Group collaboration tools

Did not find what she wanted under any tab or under search. Says she would contact her rep at that point. Anatomy • Reads through items: could be brand management, media mix, or communications • Reads through the mouseover summaries to decide. “These are pretty tactical” • Looking for something higher level in terms of strategy – would email or call my rep at this point. Research • Looks through topics – choosing media mix selection…would go through results, but would have liked to see the wheel (pie chart) • Did you notice subcategories (green navigation menu)? “No, they did not stand out to me” Tools & Templates • Reads through topics, clicks marketing communications, channel selection, reads thru the documents and selects one – likes the title…skimmed the summary • If couldn’t find it, would go back to Tools & Templates overview and look through other topics…sees implementing brand strategy, and clicks into there • If couldn’t find it, would search first, then might call account rep.

User 2

Topic: Win/loss analysis

Has the most success with Search Anatomy: • More likely to not go here and instead go to the search bar • Clicks into media mix selection – lots of information – “here’s my online right here – I don’t see online as one of these items…too much information”, try to narrow down through the search bar • Noted image was too small to read or figure out • Expecting a laundry list but instead got a bucket – too much to read Research: • Read through many topics, but didn’t see what she was looking for so would go to the search bar • Doesn’t feel like figuring out how we’ve categorized stuff – I’d just type it in the bar • Likes the “sort by” feature on research tab, likes the format is the same as in search results • Likes the subcategories with the counts next to it • Likes it more than the first three column layout • If I was poking around, it would be easier to serve it up this way – research tab – having someone else classify it for you helps Tools & Templates: • Tools &Templates overview – lead generation roi calculator sounds very interesting. • Doesn’t read the words – “once I get to what I want, then I read“ • Xcelsius tool – how fun! • Confused a bit with the terminology. might be good to have definition pop ups

The Corporate Executive Board

Topics

User 3

User Experience 2008

Topic: “Lead generation.”

Finds material in Topic Center, Anatomy, and Search.

Topic Center: • Chooses widget. Looks for ‘strategy’, then for ‘competition’, doesn’t see anything so will look at questions. Not finding anything right away, Would search for “competition” at this point. Anatomy: • Starting at 1 going clockwise…sees number 2 box…clicks into it (butrbut would scan others first) • Would scroll down before looking to right hand column • Reads through discussions, tries to click on an image, notes that it’s not called the same thing as we call it, but would click into “competitive analysis” to see if it is the same • In pdfPDF, did control-F to find “win/loss” (is not successful) • Looks through table of contents in pdfPDF Research: • Reads overview first • Goes to Competitive Positioning. First notices the green navnavigation menu. Goes to the Sort By to see what’s in the drop down…would probably use Most Popular. • Scans through list of bold titles, looking for something that would imply competitive strategies and tactics. • Can’t find what she’s looking for, so goes to title “Competitive Positioning” but may not be the area where she wants to be. Tools & Templates: • Looking at titles on overview page; not paying attention to images. • Not seeing anything right away, but would go to segmentation next (noted that that is a different priority for their firm). User 4

Topic: “Sales Tools” Anatomy: • Finds and clicks on Lead Generation. Scrolls down, then clicks a document. Doc loads 20+ seconds….reads through table of contents, finds something interesting, looks promising, success • Best practices on definition of a lead • With the peer groups, posting something could take time. Looking for immediate gratification. Research: • Reads over research tab overview and notes that the taxonomy on the current overview page is better than what is in the prototype • Clicks into lead generation…goes immediately to the green navnavigation box…reads the sub topics…clicks on lead qualification….Reads through titles, trying to find one centered around definitions or qualification. Not really what he’s finding at the moment. • Gravitates to the word B2B…because there is so much B2C…don’t trust generalized marketing strategy Tools & Templates: • Finds Lead Generation ROI Calculators - not what he is looking for. • Finds Sales Support, Lead Generation, and clicks first item (McKesson scorecard). “Yeah, this would be something I would be interested in.” • Confusing that Lead Generation ROI and Lead Generation under Sales support are in different places.

User 5

Topic: Online Marketing

Has the most success with Research and Search. Topic Center: • Goes to blue box and then to 3 column layout. Finds sales-marketing integration, then goes to new product development.

The Corporate Executive Board

Topics

User Experience 2008

• Customer insights….what is that supposed to mean…sounds like insights on a prioductproduct instead of research on a market. Would think of it more as customer needs… Anatomy: • Scanning the outside boxes for a few seconds…clicks on sales support • Can’t make sense of the landing page….scrolls around finally clicks on further research link…reads through results from research tab… Research: • Didn’t realize the green box navnavigation was part of the results, or that you could see sub-topics • Clicks on sales-collateral tools, Clicks on sales force effectiveness, Clicks on solution marketing…. • “Not sure what front line support is compared to those other things” Tools & Templates: • Scrolls through overview page…after a few seconds, finds sales support…clicks • What happened to the categories I was looking at before? (on the research tab) • Very unclear on the trade show and event planning topic. User 6

Topic: Partner Branding

Has the most success with Research and Search. Finds other interesting things in Tools & Templates but wasn’t quite what she was looking for. Topic Center: • Would start in branding…clicks on brand identity…would click top link (fake link)…if didn’t like what she found, would search partner marketing. Anatomy: • Goes to message development…don’t see anything she likes, would go to search • Would like to see the subcategories under message development…what goes into creating the message, vetting the message, etc • Would like a faster way to see all the resources and search the resources within this topic Research: • Likes the Overview page, likes the clusters and the organization, it draws her eyes to the larger categories • Goes to brand development and scrolls through results list. Views first 2 pages of search results, then would skip to search. • Did not notice the green navigation menu to the left…now that she sees it, clicks on brand planning, then clicks on brand vision. She likes the second item and wants to click the title, but choseschooses view study…after doing that, says she would click view summary Tools & Templates: • Tries marketing communications • uses green box…finds plan development…looks through to see if there was anything interesting…kind of likes the first one, but doesn’t think it was really what she was looking for.

User 7

Topic: Online Marketing (B2C)

Is not successful in finding any material specific to B2C for Online Marketing. Anatomy: • More fun, but not as scan-able- can’t find online marketing • Clicks on “mixed media”- page is “awful”- too much content. Doesn’t know where the eye needs to go Research: • Online marketing- hard time finding • Campaign management-doesn’t think that’s correct • Doesn’t notice green left navigation

Finding The Corporate Executive Board

Recommendation 18

Topics

User Experience 2008

4 out of 7 users didn’t notice the left green subnavigation within Research pages

Update style so that the menu stands out more. Make it obvious that the items in the menu are links that expand and that they are related to content on the page.

When reading summaries or the first page of a document, 4 users had difficulty knowing if they found what they were looking for. In Concept Testing, one member explained that he usually prints out multiple documents and has to go back to them later to see if any of them were really what he was looking for.

Summaries and Table of Contents need improvement. Search results should show highlighted terms from document, not just from summaries. This is especially important for long documents because users might be missing out on contact that is deeply buried.

4 out of 7 users expressed frustration over the layout of the Anatomy landing pages (“Expecting a laundry list but instead got a bucket – too much to read”)

Instead of having landing pages with only a few pieces of content, provide a list of items that users can dig deeper into. Work on general formatting of these pages to keep them simple, clean, and easy to scan through.

Using all of the tabs, only a handful of users found and opened a document that was interesting to them: 2 in Anatomy, 1 in Research, and 1in Tools and Templates. For the same topics, 5 out of 7 found and opened document through Search (see below).

This implies that documents did exist which the users were interested in, but they couldn’t locate them within the tabs and were able to find them with search. Consider providing search boxes within tabs.

Task 3, Part 2: Search for the topic that you chose to research User 1

• After searching, reads through summaries to see if any contain another keyword she’s looking for • Notes the date of the documents as important for relevancy • First reads the title, if it looks interesting reads the summary, then would click view the summary. but if something looks right on, will go right to view study. • Did not notice the refine results box – understood how it would work, and thought she would use it if she saw it. She saw content, and preferred to view content over refining results.

User 2

• Likes and uses the googleGoogle suggestions (selects “competitive analysis” ) • Uses Refine Results for Tools and Templates, reads through those 8 and finds something interesting. • Immediately saw highlighting first …likes the links below each search result. Reads titles and dates. • Clicks view summary first. • Add to top priorities…thinks it is required reading, add to favorites is like your folder

User 3

• Uses googleGoogle suggest. • Clicks the research filter and likes the first two items • Scrolls through entire page….probably would not drill it down on the left hand side….would trust the site…. • Nothing tells him how these results are organized…relevance, popularity, date • Would click right to view study after reading summary, date, title, and date • Assumes Top Priorities is within favorites

User 4

• Uses Refine Results (for Research) • Clicks view summary, reads summary, then clicks view study and starts reading through the first page • First page does not help explain document. He reads through it again but continues to wince reading through it and does not feel it summarizes it well • Plays with the date filter…would expect an order by instead of a filter here • If wanted to refine his search, would type new search (add industry, company, type), but would like to see in filters

User 5

• Reads through the summaries on search results, doesn’t find what she’s looking for, so searches again with different search term • After finding something, goes directly to view study • Scans through pages, doesn’t think it’s what she’s looking for, but as she continues to scan, she finds what she was looking for.

The Corporate Executive Board

Topics

User Experience 2008

• Would like to see Search by keyword, search by topic near the search box in header • Likes the highlights in the search box • Recommend, add to favorites (thinks this adds to browser bookmarks) – does not care for these • Sort of notices the filter box, but wanted to see the results to see if she got what she wanted on the first try – always had bad luck on filtering results…would have gone back to re-search if she didn’t find what she wanted • Likes the headline, date, summary, and alternating color background • So used to Google, goes to search bar and redoes the search – will go three times and give up User 6

• On first search, only gets 1 result, so searches again for different phrase. • Clicks view summary, but summary is not 100% clear to her • She keeps browsing and uses the refine results filter. • Tries to click the title but finds she can’t.

User 7

• Concerned that some results are out of date • Made a mistake in search (typo) and was surprised that the search didn’t correct it • Clicks View Study. 80 page PDF is a turn-off because it’s difficult to find information within it “Eww” • Good that it is in a new window • Expects titles should be clickable and underlined. • Refine Results: Not sure why filters are in prime real estate- thinks they should be on the right. Needs to know services to know if it’s useful or not. Could also be horizontal under search instead of dominating the screen. • Should be able to view summary without going to another page

Finding

Recommendation

4 out of 7 users specifically looked for the date of Search Results, expressing interest in seeing results listed from newest to oldest.

Allow sort by date

4 out of 7 actively used the “Refine your Results” options on the Search Results page

Move “Refine your Results” box to left of page to make it more prominent. In the future, possibly allow “Refine by Topic” as well.

2 people tried to click the title of the Search Results, which is plain text and not a link

Add links to Search Results titles. This is a standard used across the internet (see Google or any major search engine) and is expected behavior. While it does not prevent the user from accessing the search result, it can cause frustration, especially for those who are new to the site.

3 out of 7 users performed multiple search queries in a row in order to refine their results or find a larger number of results

Explore error correction (“Did you mean to search for __?”) and suggestion search terms, especially for searches that return no results because the user uses terminology different than what is being used on the website.

After finding an interesting search result, 3 users clicked “View Summary” and 3 clicked “View Study.”

Throughout any changes or enhancements made to the Search Results page, be sure that both of these links remain on the page.

Through search, 5 out of 7 users successfully found and opened a document that they were interested in.

Again, continue to place emphasis on enhancements to search, since users had more success with search than with any of the tabs. They are more likely to keep returning to search based on this positive experience.

If you returned to this website for information, where would you go to find it? User 1

With all options, after seeing everything, would go to search first…otherwise, would go to topic

The Corporate Executive Board

Topics

User Experience 2008

box User 2

Would go to search first because it was the fastest way to find something.

User 3

If I wanted to learn about a topic, I’d go through the tabs and browse through the topics.

User 4

If I was looking for a tool and template, would go to the tab, if looking for general research would go to topic centers. Would use Tools & Templates tab or Meetings and Webinars tab otherwise would hit the topic centers pretty quickly. Very important that he understands the entire process, which is why he browses.

User 5

Topic center because Tools and Templates and Research fit within the topic centers. Looking at the anatomy….looking for a way to get to tools… It’s nice to have a framework so you understand how things fit Once you’re in a particular area, you may want to search within that area. If I saw strategic planning in the topic box, I’d go there, but I don’t see it so I’ll go to search

User 6

Would go to topic center (blue box), or if I had a specific topic in mind, I’d go to search

User 7

Search. For sites like MLR, he always searches. Expects to receive newsletters on more detailed feature items.

Finding

Recommendation

Provide as much value as possible on the search page itself, recognizing that this is a page nearly everyone uses (and some use exclusively):

•

With the exception of 2 users who search exclusively, most users expressed interest in having the ability to browse for more high-level of information and search for more specific information.

Continuously monitor top search terms and map them to most recommended documents, tools, webinars, and peer groups. Clearly mark them as recommended so that users can get the same value out of Search as they would from landing pages that highlight specific offerings. In addition to sorting by date, allow users to sort by relevance, popularity, and recommended items (as they can within Research)

*Analytics report confirms that Overview page and the Search Results page are consistently (across all programs) the top 2 exit pages on the websites (11% of visits end on the Overview page, and 11% of visits end on the Search Results page) *Analytics report confirms that Overview page and the Search Results page are consistently (across all programs) the last pages viewed before a user exits the site.

Next Steps Website Enhancements Based on our findings, a number of small enhancements can be made to areas of the website: • • • •

Adjusting style of left green menus to make them more noticeable Adding a “Sort By:” menu to the Search Results page Adding links to the titles on the Search Results page Adding a “Search within this topic:” feature to Research pages

The Corporate Executive Board

Topics

User Experience 2008

The feasibility of other more complicated recommendations will considered for future enhancements.

Determine Topic Taxonomy and Labeling The most common concern that users had about a topic-based method of navigation was the categories and labeling would not correspond to their way of thinking and that they would therefore have trouble finding what they need. Further effort and research will be needed as each program refines their method of categorizing content: • • •

Perform card-sorting survey internally and externally to determine how topics should be categorized. Continually analyze search terms (top search terms and terms with no results) to gain insight into how users label things vs. how we label them. This should provide input to card-sorting survey. Ensure that once categories are chosen, they are used consistently throughout the website and don’t change from tab to tab.

Build an Intuitive Topic-Based Navigation Users struggled with both methods of topic-based navigation presented in the usability testing sessions. Navigation of the existing Research and Search Results pages was more intuitive to them than either the 3column topic layout or the Amazon-style drop down menu. As we move forward, consider using layout from Research pages combined with listings that are structured like Search Results.

Assigning Topics to Existing Content Before offering any method of topic-based navigation, content that currently exists on the websites will need to be assigned to topics through Admin accounts. This is already done under the Research tab but will also need to be applied to items such as Webinars and Peer Groups. Functionality needs to be added to the website to allow Admin users to do this.

Appendix A: Concept Testing Script Introduction 2 Minutes

Thank you for participating in this session! We want your feedback as we evaluate an idea for a new feature on the MLR website. Our hope is that this new tool will allow you to more easily find what you’re looking for when you visit the website. I’m going to begin by asking you a few questions about how you currently use the website. After that, we’ll look at some screen mock-ups which I’d like to get your feedback on. Any questions before we begin?

Preliminary Interview 5 minutes

What is your role? Why do you visit the website? How often do you use it? When you first enter the website, where do you go to find what you’re looking for? (For example, going to a certain tab, or using the search box) What do you do when you can’t find something?

Discussion Topic 1: How to Access Information 5-10 minutes

Now we’re going to look at some mock-ups of webpages. I’ll be asking you to “think aloud” as you’re looking at these, so say anything that comes to mind. For example, if you don’t know what something is for, you can say, “I don’t know what this is for.” Show Option 1, Step 1/3 What is your initial impression of this page? What would you expect to be able to do here? What action would you take? Show Option 1, Step 2/3

The Corporate Executive Board

Topics

User Experience 2008

Show Option 1, Step 3/3 Is this what you expected to find? Is there anything missing from this that you would like to see? What do you think of the categories? Show “Question” Version This is a slightly different version of the page we just saw. Column 2 now contains questions. Do you find this more or less helpful than the previous version? (Show Version 1, State 3 again). Why? Show Option 2 Here is another variation, which works the same way but is presented in a different format. Show Option 3, followed by Landing Page Rather than access all of the information on one page, we can also view it split into two different pages. Show all on one page We just looked at 3 different ways to access the same information Which of the 3 would be most helpful in finding the information you need? Discussion Topic 2: Overview Page 5 minutes

Now we’re going to take a look at the homepage, with this new tool integrated into it. Show Home Page Version 1, Version 2, Version 3 Show all on one page Of the variations we just looked at, which would you find most useful?

Discussion Topic 3: Terminology 3 minutes

What should this tool be called? (Show options) What should the categories be called? (Show options) 3 options for phrasing (Show options)

Closing Questions 2 minutes

Would you use this tool if it were available today? Why or why not?

Total duration: 20 – 30 minutes

The Corporate Executive Board

Topics

User Experience 2008

Appendix B: Concept Testing Comps Fig. 1 Homepage layout: List of topics

Fig. 2 Homepage layout: List of topics, broken into categories.

Fig. 3 Homepage layout: â&#x20AC;&#x153;Top Challengesâ&#x20AC;?

The Corporate Executive Board

Topics

User Experience 2008

Appendix C: Usability Testing Script Introduction

Thank you for participating in this session! We want your feedback as we evaluate an idea for a new feature on the website. As part of the process we’re asking people to attempt various tasks to see what elements of the tool may be challenging. I’ll be asking you to ‘think aloud’ as you use the website. For example, if you don’t know what something is for, please say ‘I don’t know what this is for.’ I may also prompt you from time to time to ask you what you are thinking. We’re going to be looking at a prototype and not a finished product, so there will be pages that don’t have content yet or contain filler content in latinLatin. Do you have any questions before we get started?

Background Questions

• Where do you go first when looking for information on the MLR website? • What do you do when you can’t find what you’re looking for? (Where do you go after you leave the site?) • What stands out to you on the home page? Do you often read the content here or do you immediately begin navigating further into the site?

Task 1

Now I’m going to ask you to complete a task using the website. You’re about to launch a product and want to see what’s available related to this topic – maybe a toolkit or research, but you’re not sure. Use the blue box labeled “Topic Centers” to find information about launching a product. • Is this what you expected to find after you selected a topic? • There’s a link to “View all content on this topic.” Where would you expect that to take you? How about the search box on the bottom of the third column? (Once on the landing page, ask how they’d go back)

Task 2

Let’s say that you want to research word of mouth marketing. Use the “Topic Centers” drop-down menu (upper left) to find information on this topic. Where would you go if you couldn’t find the topic you were looking for? On search page: • What would you expect to happen when you select these items? • Refine results – what do you expect to happen when you select the checkboxes? Would you use these?

Task 3

Think of a topic or problem relevant to your day to day work right now. What is it? How would you research it? (Have them begin at the home page) After returning to the home page: Where would you go to change your email preferences? There’s a “Request Assistance” link in the upper left of the page. What do you think this does? Would you ever use it?

General Feedback

Which method of navigation was the easiest for you to use? (Options: Clicking a link on the home page, using the drop down menu, using the 3-column layout) Now that you’ve seen this tool, where is the first place you’d go to look for information on your next visit to the website? (Topic Centers, Search, Tabs?)

The Corporate Executive Board

Topics

User Experience 2008

Appendix D: Usability Testing Prototype Fig. 1 Homepage with â&#x20AC;&#x153;Blue Boxâ&#x20AC;? used for Task 1

Fig. 2 Drop-down Topic Center menu (Amazon.com-style) used in Task #2

Fig. 3 3-column Topic Center layout used in Task 1

The Corporate Executive Board

Topics

User Experience 2008

Fig. 4 Topic Center button incorporated into Search Results page, used in Task #2

Fig. 5 Topic Center Landing Page used in Task #1 and #2

The Corporate Executive Board

Topics

User Experience 2008

Usability Testing: Round 2 February 11 – 15, 2008

Methodology   

Conducted sessions with seven users (3 members, 4 other who work in HR) Testing was done based on content from the HR Executive Forum website Sessions took place in person at a computer: o Beginning at the home page, users were asked to navigate to research a specific topic. o Using 2 different variations of topics page, users were asked to perform tasks involving filtering and sorting.

Highlights from Findings Finding

Recommendation

Topics tab is intuitive to users. From the home page, when asked to find information on recruiting, 6 out of 8 people clicked the Topics tab. 2 clicked “Recruiting” from list of topics on the home page.

Provide Topics tab as a first priority. If choosing between adding a Topics tab and adding a list of topics to the home page, add the Topics tab. Note that having a list of Topics is still helpful and should be considered in the Overview page redesign.

All users noticed the Recommended Items (both style variations), except in one situation where an item in the column layout was overlooked. When specifically asked, more preferred the yellow background layout.

Use Recommended Items on all Topics page, with a style similar to the yellow background style used in prototypes.

8 out of 8 clicked the title to open a document rather than the “View Study” link.

All document titles should be links. “View study” link can be removed.

8 out of 8 understood how the “Search within this Topic” box would work

“Search within this Topic” can be included throughout the website.

7 out of 8 successfully used both the Filters and Sort By drop-down menu when asked to find something on the page.

Filter and sorting functionality should be included in the Topics pages to help users quickly find what they are searching for.

Detailed Findings and Recommendations Note: To read the full script, see Appendix A. To view prototypes, see Appendix B and C.

. Participants

Level of Experience with HREF

Type of User

User 1

None

Non-member

User 2

None

Internal CEB employee

User 3

None

Internal CEB employee

User 4

None

Internal CEB employee

User 5

None

Internal CEB employee

User 6

Low

Member

User 7

Medium

Member

The Corporate Executive Board

Topics

User Experience 2008

User 8

Medium

Member

Where would you go to find information about recruiting? (asked from the Overview Page, Appendix B Figure 1)

User 1

Clicks Recruiting link from list of topics

User 2

Clicks Topics tab

User 3

Clicks Topics tab

User 4

Clicks Topics tab

User 5

Clicks Recruiting link from list of topics

User 6

Clicks Topics tab

User 7

Clicks Topics tab

User 8

Clicks Topics tab

Finding

Recommendation

Where would you go to find information about sourcing? (asked from the Landing Page, Appendix B Figure 4)

User 1

Clicks “Sourcing” from left green menu

User 2

Scrolled down to Sourcing link in content of page. Did not notice green menu at first.

User 3

Clicks “Sourcing” from left green menu

User 4

Scrolled down to Sourcing link in content of page

User 5

Scrolled down to Sourcing link in content of page. Did not notice green menu.

User 6

Clicks “Sourcing” from left green menu

User 7

Searches. Did not notice green menu.

User 8

Clicks “Sourcing” from left green menu. Notes that the text is too small.

Finding

Recommendation

Every user who noticed the green menu used it successfully. Those who did not notice it either searched or used a link within the page.

As mentioned in previous testing, the left menu is intuitive to use but hard to notice on a user’s first visit. The style of the menu needs to be updated to be more noticeable and include larger text links.

Recommended Items Two style variations: yellow background and 3 columns (see Appendix B Fig. 1-2) 

All noticed the recommended items, except one situation where an item in the columns was overlooked.



There were requests to make the “Recommended” text larger in the yellow background version.



When specifically asked: o 4 preferred yellow background

The Corporate Executive Board

Topics

User Experience 2008

o o

2 preferred columns 2 didn’t care

Finding

Recommendation

Use Recommended Items on all Topics page, with a style similar to the yellow background style used in prototypes.

Filtering Two style variations: Right filter box and center filter row (see Appendix B Fig. 1-2) 

During tasks: o 2 out of 8 people used filter box on the right o 7 out of 8 people used filter box in the center



When specifically asked: o 4 preferred center filter row o 3 preferred right filter box o 1 didn’t specify Finding

Recommendation

7 out of 8 successfully used Filters when asked to find something on the page. Although preferences were split when asked, more users actually found and used the horizontal menu bar.

Filter functionality should be included in the Topics pages to help users quickly find what they are searching for. Use the horizontal design.

Favorites/Recommend  4 out of 5 non-members thought the “Favorites” button would add to browser favorites. 2 did not see the button at first.  All members understood how both the Favorites and Recommend button would work. Finding

Recommendation

Members understand what Favorites and Recommend mean, but new users do not.

Evaluate this further in future testing.

Other behavior observed while performing tasks   

8 out of 8 click the title to open a document (rather than ‘View Study’); they generally expected the link to open the document and not a summary, but 2 users specifically said that they were not displeased by the fact that a summary was shown instead of the document itself. 8 out of 8 understood how the “Search within this topic” box would work 7 out of 8 used the sorting drop-down menu at some point Finding

Recommendation

All users preferred to follow a linked title instead of a “View Study” link

All document titles should be links. “View Study” links can then be removed.

8 out of 8 understood how the “Search within this Topic” box would work

“Search within this Topic” can be included throughout the website and users will understand how to use it.

7 out of 8 successfully used both the Filters and Sort By drop-down menu when asked to find something on the page.

Filter and sorting functionality should be included in the Topics pages to help users quickly find what they are searching for.

The Corporate Executive Board

Topics

User Experience 2008

Appendix A: User Testing Script Background questions

1. What research sites (if any) do you use? What HR sites do you visit and why? Where do you go to find information? 2. Do you use any social networking sites (ie Facebook, LinkedIn)? Which ones? Why do you use them? For members – show home page: 1. Why do you visit HREF? 2. Where do you go to find information on the website? 3. What do you do when you can’t find what you’re looking for?

Task 1 10 minutes

Begin at home page Where would you go to find information about recruiting? On Landing Page Where would you go to find information about sourcing? Show topic page Look around the page. Where are we in the website? What do you expect to find on this page? Find and open the most popular research document about Sourcing. How would you add this document to your favorites? Find the research document titled “Sourcing Passive Candidates.” Refer to highlighted items Did you notice these? Why are they at the top?

Task 2 5 minutes

Find and open the most recent Webinar about Sourcing. Find the tool titled “Talent Attraction Drivers.” Between interface 1 and 2, which was more helpful in finding information?

Tabs

What would you expect to happen when selecting each of the tabs along the top? (Go through one by one)

The Corporate Executive Board

Topics

User Experience 2008

Appendix B: Usability Testing Prototype Figure 1: Overview Page With list of topics and â&#x20AC;&#x153;Topicsâ&#x20AC;? tab

Figure 2: Topic Page Version 1 Recommended items highlighted with different background color and filters on the right

The Corporate Executive Board

Topics

User Experience 2008

Figure 3: Topic Page Version 2 Recommended items in a grid along the top and filters in a horizontal menu along the top

Figure 4: Landing Page for a Top-Level Topic

The Corporate Executive Board

Taxonomy Redesign: Card Sorting

User Experience 2008

Taxonomy Redesign: Card Sorting April – May 2008

Purpose During the Topics usability testing, we discovered that members often struggled to find information more because of the content than the UI. The way information was categorized was not intuitive to them, or labeled differently than they would expect. As a result, we suggested that programs reach out to members to complete a card sorting exercise and re-evaluate their taxonomies.

Process •

• • • • •

5 programs participated in card sorting throughout April and May: o Sales Leadership Roundtable o IT Leadership Exchange o Operations Leadership Exchange o Finance Leadership Exchange o HR Leadership Council Each program sent out an online survey to members (and often to CEB employees who were subject matter experts, as well). The number surveyed ranged from 17 in HRLC to over 80 in FLEx. In the survey, members were given 30-60 pieces of information in the form of virtual “cards.” They were then asked to organize the cards into groups that made sense to them. They labeled the groups, as well. The website used to implement the survey was OptimalSort.com Search Term analytics were used to ensure the terminology used by our members matched how we labeled our taxonomies Results of the survey were exported to Excel and analyzed. The results informed the revised taxonomies for each program.

Instructions for the card sorting survey at OptimalSort.com

Summary of Findings & Recommendations For detailed results of how the taxonomies changed, contact Rebecca Sherrill or Curt Odar. •

Card sorting results varied heavily from program to program: o Some gave clear indications of how members would organize content o Others were much more scattered and required significant analysis or multiple rounds of surveys

•

Changes to taxonomies also varied: o Some programs completely overhauled their taxonomies, both in organization and in labeling o Others made smaller tweaks to their existing taxonomies by adding a few extra nodes, renaming labels, etc.

Outcome 5 programs restructured their taxonomies and terminology. This was included in the May 2008 release. The Corporate Executive Board

Taxonomy Redesign: Card Sorting

User Experience 2008

Measurements of Success Through analytics, weâ&#x20AC;&#x2122;ve seen a decrease in searching and an increase in browsing the Topics pages, which indicates that the new navigation paradigm is easier to understand than the previous one. Previously, an equal amount of users searched and browsed. Now, browsing happens twice as often as searching.

The Corporate Executive Board

Home Page Redesign

User Experience 2008

Home Page Redesign February – August 2008

Project Summary Purpose Through member and stakeholder feedback we found our home page needed to achieve 3 things more successfully: Provide navigation, feature content or announcements, and provide personalization and/or segmentation. Since most traffic to our websites comes through the homepage, we want it to provide a positive experience that is worth returning for.

Process Step 1: Research Before beginning testing, we researched methods for homepage redesign: • Revising a Homepage: Applying Usability Methods that Guarantee Success Presentation by Robert Bailey, Cari Wolfman, Janice Nall at the 2008 UPA Conference •

Usability Tools Podcast: Home Page Design By Jared Spool, User Interface Engineering (UIE.com)

•

Lifestyles of the Link-Rich Home Pages By Jared Spool, User Interface Engineering (UIE.com)

We also visited and collected examples of websites that we could draw from as examples. These ranged from research-oriented websites (like CEB) to websites in other industries that provide clear, usable navigation (Examples: Geico, Southwest). Lastly, we looked at activity on our current homepage and member survey responses about why they visit the website. This data combined with best practices led us to define 3 goals of the redesigned homepage: Provide navigation, featured/new content, and personalization/customization. Step 2: User Testing In February, we had 8 users (half members, half Middle Market stakeholders) complete a ‘Boxes’ exercise. Participants were given 16 small pieces of paper, each containing a piece of content. They were asked to select and arrange (prioritize) 8 pieces on a print out of a blank home page. Step 3: More User Testing In May/June, based on the results of Step 2, we created 3 different wireframes and did task-based usability testing with 7 members.

Step 4: And Even More User Testing In July/August, based on results from Step 3, we created 2 new wireframes and tested with 6 members. Testing was task-based and then members were asked for general preferences. The Corporate Executive Board

Home Page Redesign

User Experience 2008

Summary of Findings & Recommendations • • • • •

The “Site Map” at the bottom of the page was used heavily by members during testing. For user-defined tasks, “Topics” was the most common starting point. When testing the wireframes, the elements listed as most important were: Topics, tabs, search, and “My” boxes (personalized content). This corresponded with the most popular elements in the initial ‘Boxes’ exercise (see Step 1). Strong preference for Topics list along the left of the page. Overall, every design decision for the new Overview page could be traced back to our discoveries from research and testing.

Outcome The redesigned homepage was launched in the October 2008 release. Through analytics, we see that exit/bounce rate has decreased dramatically for the homepage. This indicates that the content is more relevant to individual users and they continue to explore the site rather than leave. In the long term, this will contribute towards our goal of having members visit the website more often. Hundreds of members have proactively selected topics of interest through the new “Recommendations” box. We are able to customize offline interactions with the member using this information, in addition to customizing their web experience.

The Corporate Executive Board

Home Page Redesign

User Experience 2008

Usability Testing: Round 1 Users were given 16 small pieces of paper, each containing a piece of content. They were then asked to select and arrange 8 of those pieces on a print-out of a blank home page. In the below charts, points were assigned based on the number of users who chose that element and where they placed it on the page. Trends  My Items and Events were included by everyone (4 chose My Items in #1 spot)  7/8 included Peer Groups  6/8 included Topics, Research, Tools,  5/8 included My Company  4/8 included Services, Benchmarking In order of most to least popular: My Items

News

Topics

Benchmarking

Peer Groups

Memberships

Tools & Templates

Ad Msg

Events

Top searches

Research

Newsletters

My Company

Quick poll

Services

Full Results:

User # Topics Services Research T&T My Company My Items Events Peer Groups News Newsletters Benchmarking Memberships Ad Msg Quick Poll Top Searches

Points Assigned by Participants 1 2 3 4 5 6 7 8 8 6 8 6 8 5 3 4 3 6 6 5 6 5 1 3 7 3 4 5 4 4 2 7 1 7 8 5 8 2 8 8 2 8 2 4 1 1 3 7 6 2 3 1 5 7 4 4 5 7 7 2 5 2 1 3 4 2 3 6 1 7 1 6

The Corporate Executive Board

Home Page Redesign

User Experience 2008

Additionally, the exercise was completed by program leads internally so that we could compare the results to the preferences of our users. The results (with similarities bolded): Users: 1. My Items 3. Peer Groups 5. Events 7. My Company

Us: 2. Topics 4. Tools & Templates 6. Research 8. Services

1. Ad Message 3. Peer Groups 5. My Items 7. News

2. Topics 4. Quick Poll 6. Top Searches 8. Events

Set up materials on table This is the HREF home page. There are 16 different modules to choose from – please arrange them on the home page as you would like to see them. There’s also a blank module if you’d like to create your own. When you’re done, make any notes about the modules or the page in general. Feel free to write on any of the pieces of paper. For example, if you don’t know what something is, you can write “What is this?” Walk through each chosen module, asking why they chose it and what it does The pieces that users were asked to choose from are below:

The Corporate Executive Board

Home Page Redesign

The Corporate Executive Board

User Experience 2008

Home Page Redesign

User Experience 2008

Usability Testing: Round 2 Methodology We interviewed 7 members over the phone (6 HREF and 1 SLR). Users were shown 3 different variations of wireframes. Before viewing the pages, users were asked to think of a specific issue they have dealt with recently or are currently facing. For each version: Users were asked how they would go about solving the issue they thought of at the beginning of the session. Users were given a series of generic tasks (“How would you edit your favorites? How would you submit a question to a Peer Group?”) Users were given a series of specific tasks (“We have a job description library. Where would you go from here to find it?”) Users were asked to identify the 3 most important elements of the page After viewing all 3, we asked the users to identify their preferences. High-Level Findings:  Users did not particularly struggle with tasks in any one design.  In the Sitemap design, they used the links at the bottom more than any of the links elsewhere on the page.  Value of “News” seems to vary based on program.  Almost no one saw the “Add to Team” section (They were more likely to see it within the sitemap menu than in its own bigger box on the page)  Preference is for Sitemap version, but still pushing for some of the “fresh”/personalized content (Peer Groups and Event Calendar) as well.  Multiple people felt that quick access to the sections under My Account was a priority.  For the user-defined task, Topics was the most common starting point. Recommendations: Use some version of the “Site Map” on the home page Topics must be very prominent – most were drawn to it for the user-defined tasks. My Account sub-sections should be accessible from the home page. Users knew what My Account meant and expected to be able to find many things there (favorites, recent, add team member, etc.). News section should be optional by program

Overall Preference User 1: User 2: User 3: User 4:

Southwest + Personalized Content Sitemap + “Recommended for You” Sitemap + Latest Content + “taste of dynamism” from personalized page Sitemap + Recent Content and My Calendar. Ideally would be a less busy page so it’s a middle ground between sitemap and southwest designs. Personalized + details on bottom of Sitemap Personalized + Sitemap menu and Toolkits

User 5: User 6:

User 1: Chris 

User-defined task: What are other companies’ policies about tuition reimbursement and how long the employee must stay with the company afterwards. Sitemap Layout: Ask Peers Southwest Layout: Ask Peers Personalized Layout: Ask Peers



Most important components: o In Sitemap: Peer Network, Toolkits, My Account o In Southwest: Find a Topic, Ask a Peer, Manage my Account

The Corporate Executive Board

Home Page Redesign o 

User Experience 2008

In Personalized: My Event, My Peer Groups, My Favorites

Overall Preference: Liked simplicity of Southwest layout, but liked the personalized features of the other layout. Noted that the top menu stood out more in Southwest layout.

User 2: Marty 

User-defined task: Revamp set of questions that they ask references during the interview process Sitemap Layout: Topics  Recruiting Southwest Layout: Topics  Recruiting Personalized Layout: Recommended for You  Research



News: Likes it. One of the headlines in the example is something that was recently posted to a peer group and her company is taking action because of it.



Most important components: o In Sitemap: Topics, Peer Network, Prof. Development, My Account/My Favorites o In Southwest: Services Overview, Topics  Recruiting, Favorites o In Personalized: Recommended for You, Peer Groups, Event Calendar



Overall Preference: Liked the sitemap layout and felt that the titles and bottom categories were broken out in a way that seems “easy.” Would also take the “Recommended for You” section from the Personalized layout.

User 3: Angie 

User-defined task: Competitive benchmarking regarding compensation Sitemap Layout: Maybe Topics Southwest Layout: Find by Topic  Compensation Personalized Layout: Research



Most important components: o In Personalized: Recommended for You, Event Calendar, Peer Groups o In Southwest: Topics, Sign Up for Webinar, Ask a Peer o In Sitemap: Toolkits, Events & Webinars, Topics, Peer Groups  Says Toolkits and Topics are actually very similar. The difference is that under Topics, she would expect to find research and context for a topic with references to the toolkits. So she would actually go to Topics first unless she saw a Toolkit listed that was specific to a template she was looking for.



Overall Preference: Disliked Southwest layout because she felt it would always make her dig deeper and nd everything is at least 2 clicks away. Liked Sitemap version because it felt streamlined and organized. 2 favorite was the personalized version because it has a “taste of dynamism.” Feels all version are missing something important – Latest Content.

User 4: Penney 

User-defined task: (1) Employee engagement survey or (2) comparing benefits with competitors Sitemap Layout: Topics at bottom for both Southwest Layout: (1) Topics  Employee Engagement (2) Topics  Benefits Personalized Layout: (1) don’t know, would want to use search engine. (2) Topics  Benefits



Most important components: o In Personalized: My Event Calendar, Recommended (but wants to know how), Tabs and Topics o In Southwest: My Account, Find by Topic. Likes that Find By Topic is “more upfront” o In Sitemap: Bottom menu. Likes that there are 2 places to do everything.



Overall Preference: Site map = first choice. Ideally would like a simplified version of the site map design, so that it’s a middle ground between Southwest and Site map. Recent Content and Calendar are must-haves.

User 5: Kimberley 

User-defined task: Applying metrics to training curriculum Sitemap Layout: Topics tab

The Corporate Executive Board

Home Page Redesign Southwest Layout: Personalized Layout:

User Experience 2008

Topics  Measurement and Metrics Topics  Measurement and Metrics



Most important components: o In Personalized: That it’s personalized. Everything relevant to her on one page and then she can drill down. o In Southwest: Manage my Account. Topics in your face. Search. o In Sitemap: My Favorites, Recently Viewed, the bottom menu, and Search.



Overall Preference: Don’t like Southwest. Loves the Personalized one, but it could also be a page that you follow a nd link to. 2 choice is Sitemap because she likes the details on the bottom (saves time).

User 6: Melissa 

User-defined task: E-verification and what peers are saying about it Sitemap Layout: HR News. If it wasn’t there, I would search or look in Recent Additions Southwest Layout: Find by Topic Personalized Layout: Search. Page is too busy.



Most important components: o In Personalized: Company Info, Peer Groups, Calendar with integrated news o In Southwest: Peers, Manage My Account, Topics. But this page is not categorized enough. o In Sitemap: My Account, Toolkits, Peer Network. And News is good.



Overall Preference: Dislikes Southwest (“yuck”). Likes how the personalized one is boxed out, which makes it easier to look at. From the Sitemap page, likes that it’s detailed and feels that she doesn’t have to guess. Would like the page to include Toolkits.

The Corporate Executive Board

Title & Abstract Testing

User Experience 2008

Title & Abstract Testing November 2008

Project Summary Purpose Members express an inability to find information on the CEB member websites. Analysis of these complaints suggests that relevant content exists on the site, but it is not clear to members what documents hold the relevant the information. The initiative focused on how Titles and Abstracts could help or hurt the chances for member to find the content they were looking for.

Process Research Before beginning testing, we researched the idea of doing content-focused usability testing: • Optimizing Search by Showing Results in Context http://research.microsoft.com/en-us/um/people/sdumais/chi2001.pdf

•

Reading Online News: A Comparison of Three Presentation Formats

•

How Users Read on the Web

http://web.archive.org/web/20080211105001/http://psychology.wichita.edu/surl/usabilitynews/42/depth.htm http://www.useit.com/alertbox/9710a.html

Internal Testing Round 1: Format • Seven executive-level CEB employees were asked to complete six tasks on prototypes of websites. • Four abstract formats were tested: 1) Titles only, 2) Titles + Intro Sentence, 3) Titles + Intro Sentence with Three Bullets and 4) Titles + Paragraph.

List View

Abstract View

Internal Testing Round 2: Content • Conducted with five participants from CEB • Participants were asked to compare different types of abstracts. This time, abstracts were formatted similarly but contained different pieces of content. Member Testing The Corporate Executive Board

Title & Abstract Testing

•

User Experience 2008

tested recommendations about content and format from rounds 1 and 2, and explored the utility of Executive Summaries on the website. Members were used for testing.

Summary of Findings & Recommendations Recommendations from Round 1 1. In list views, show a short bulleted summary. Abstracts formatted with a title and one sentence followed by three bulleted sentences that fit on one line each are both the most read and reported by users to be most successful in a list view. Round Two of testing will be more prescriptive regarding the content of these bullets for the list view, as well as the length and content of the full abstract. 2. Prioritize re-titling and abstracts for list views before focusing on full abstract rewrites. Testing found a percentage of users will immediately download a document though abstracts are presented in a list - a behavior confirmed by the CEB web metrics. Re-writing titles and abstracts displayed in the list should be prioritized over the re-writing of our full abstracts. 3. Use keywords in titles and abstracts. Users scan abstracts for the keywords related to the issue they are trying to solve. Member’s keywords (i.e., how they describe problems) should be included in titles and abstracts. 4. Remove “Recommend to Colleagues” and “Alert Me of Similar Content” buttons. These buttons do not fit within the context of the list view. On the list view, present two buttons: “View Abstract” and “Download.” 5. Remove the dates and page numbers from the list view. Older studies and long documents turn off users, regardless of relevancy. As such, show the date and page number at the bottom of the full abstract to encourage users to read for relevancy before making a snap judgment. 6. Present more information in the abstract view than in the list view. With each click, or investment, we should delight the user by providing more value than the page before. Users are frustrated when they’re taken to an abstract page that doesn’t provide additional value.

Recommendations from Round 2 1. In the list view, two types of content were more helpful than others: a. action-oriented bullets, describing how the user can take action b. bullets outlining sections or chapters of the document, giving a roadmap 2. Do not make assumptions about the reader. Do not describe who a report is ideal for. Do not use “You will learn how to”, as it implies that the user doesn’t already know something about a topic. Instead, tailor wording to make the reader feel empowered. 3. Meet the user’s progressive investment with progressively more information at each click. The abstract should contain more content than the list view description, but not so much that the user is overwhelmed. Do not include extra content that belongs in the document itself (such as Methodology, Distribute To). Information presented should grow as we move down the path: List ViewAbstractExec SumDocument. 4. Use headings and bullets in abstracts to break up content. As users scan, they read the headings to determine if the bullets are worth reading. If the heading is poorly worded or abrasive to the user (e.g. “This research is ideal for”) the bullets are less likely to be The Corporate Executive Board

Title & Abstract Testing

User Experience 2008

read. 5. Hyperlinking directly to portions of a document are valuable when done correctly, but can be problematic. Only provide hyperlinks to sections of document if two criteria are met: a. The title of the page in the document matches the title of the link, or is extremely similar b. The page in the document can stand alone, and the user does not need to scroll through the document to gain the context needed to understand it.

Outcome Recommendations were delivered to research teams in Spring/Summer of 2008 and were the focus for the firmwide training on standards for titles and abstracts. Re-writing titles and abstracts will enable us to take content originally written for books and printed materials and make it consumable for the web. This must be a priority in order to compete with the wide range of information sources online. We are not able to measure the impact through web analytics, as adoption of these standards varied throughout the firm. We are, however, confident that adhering to these standards improves the findability of our content online as indicated by the behaviors observed through all 3 rounds of user testing.

The Corporate Executive Board

Title & Abstract Testing

User Experience 2008

Usability Testing Round 1: Format Executive Summary Overview The Corporate Executive Board’s (CEB) Core and Middle Market member websites are document centric, i.e., most of the information on these sites is offered as downloads, such as PDFs, MS Word, etc. In nearly all cases on the Core and Middle Market websites, the document abstract is presented on the site and is the way members determine whether or not to download a document. Despite providing abstracts to nearly all documents, members express an inability to find information on the CEB member websites. Analysis of these complaints suggests that members cannot find the information because the relevant content is not on the site, but rather because it is not clear to members what documents hold the relevant the information The goal of this project, sponsored by Glenn Tobin, is to identify the most effective abstract for the CEB to present its research and resources on its core member websites. By improving the quality of abstracts, CEB will enable more members to find the information on its sites. Testing Usability research has demonstrated that readability and findability of information on websites is impacted both by format and content presented. To isolate the impact of content and format, the testing was broken up into two rounds: Round One-format and Round Two-content. The following report only overviews Round One of the testing. Round Two will be included in subsequent updates to this document in June 2008. [Member testing, which will test findings from Rounds One and Two will occur in mid-June.] Round One testing was conducted with seven subjects from CEB being asked to complete six tasks. Four abstract formats were tested: 1) Titles only, 2) Titles + Intro Sentence, 3) Titles + Intro Sentence with Three Bullets and 4) Titles + Paragraph. Please see the Appendix for full details on the test methodology. Highlighted Recommendations Strong patterns of behavior and preference emerged, despite the variation in the Round One test group. Four key recommendations emerged from Round One which are discussed in detail in the report. 1. Create a version of abstract for when it appears in a list on the site. Testing found a large percentage of users will go immediately to download when abstracts are presented in a list (a behavior verified by the web metrics on CEB member sites). As such, testing would recommend that abstracts displayed in the list should be prioritized over the re-writing of our full abstracts. Abstracts formatted with a title and one sentence followed by three bulleted sentences that fit on one line each are both the most read and reported by users to be most successful in a list view. Currently, the website automatically takes the first 400 characters of the abstract, which must be changed to enact this recommendation. Round Two of testing will be more prescriptive regarding the content of these bullets for the list view, as well as the length and content of the full abstract. 2. Use keywords in titles and abstracts. Users scan abstracts for the keywords related to the issue they are trying to solve. Member’s keywords (i.e., how they describe problems) should be included in titles and abstracts whenever possible. 3. Remove “Recommend to Colleagues” and “Alert Me of Similar Content” buttons. On the list view version of abstract, only present two buttons: “View Full Abstract” and “Download.” 4. Remove the dates and page numbers from the list view.

The Corporate Executive Board

Title & Abstract Testing

User Experience 2008

Older studies and long documents turn off users, regardless of how promising the document may be. As such, show the date and page number at the bottom of the full abstract to encourage users to read for relevancy before making a snap judgment.

The Corporate Executive Board

Title & Abstract Testing

User Experience 2008

Note on Terminology Titles and abstracts were tested on two pages, which we call “list view” and “abstract view.” List View: Page members see when browsing the taxonomy on the current core websites.

Example: https://www.clc.executiveboard.com/Members/ResearchAndTools/Browse.aspx?tid=/2272/2321 Abstract View: Pages members see when browsing the taxonomy on the current core websites.

Example: https://www.clc.executiveboard.com/Members/ResearchAndTools/Abstract.aspx?cid=100063372&tid=/2272/232 1

The Corporate Executive Board

Title & Abstract Testing

User Experience 2008

List View | Formatting Findings What We Learned

Supporting Quotes

Prefer information formatted as bullets (7 of 7)

“Bullet point is always easier for me. I think the world is starting to think in bullet points.”

Want bullets to explain: • Document’s contents • Document’s major sections • Business problem being addressed

I like the bullet points or the numbers so that I can see these are the three areas that I’m going to look at.

It’s showing me the things that I would want to see. It’s telling me that what the business problem is, it’s telling me what the case profile is going to show, it’s telling me what I should expect if I do this and what else would be of interest to me. So this one is answering questions before I’ve had to sit back and read everything to ask. It’s setting my expectation about what I’m going to get when I open this up.

This one’s easier than the other ones that I’ve read. It’s laid out in a more concise manner. I like the fact that there’s not as much paragraph form. I like the bullet points or the numbers so that I can see these are the three areas that I’m going to look at.

What I do like about this one is the way that the abstract is laid out for me with bullet points. It’s more it’s just more concise to me I understand why I would read it “this case profile shows this this this.”

I like the bulleted format better.

My eye goes straight to the bullets. To the key questions.

I like that the abstracts are immediately available instead of having to click into an abstract page. That it gives me that information.

I’m reading more. I’m reading down into the bullets.

I like the bullets. It’s like it’s a quick hit as opposed to having to read.

“I would never do titles without abstracts. Language is such a funny thing. If you don’t have some level of a “what this is” I wouldn’t do it.”

The titles could be broad and weren’t necessarily issue based and so I don’t want to get into the content of it. It just doesn’t feel as organized as me. It feels like a hodgepodge and it’s a little frustrating.

This is not helpful because it’s just the title.

“If it’s four years old, I’m not going to bother. Even if it says good stuff, even if it looks like what I want, if it’s four years old I’m probably not going to go to it. Because I’m going to presume that 4 years ago is not best practice today.”

I would look at this and I would immediately go, “2005?” I’m in a 2008 decision, and you’re giving me something, when I’m trying to figure out what to do as a best practice as part of today’s needs and you’re showing me something that’s got a date stamp, that’s dated, I most likely would not go on. I most likely would dismiss this immediately.

Instantly I look at the date and go, “Are we kidding ourselves?” so I’m date-sensitive. I realize that they might have done amazing work but we’re in 2008. Have you guys

Do not prefer titles only (7 of 7)

Reference publish date in deciding whether to download a document (7 of 7) • Documents that appeared promising to users were dismissed because of dates

The Corporate Executive Board

Title & Abstract Testing

User Experience 2008 not done anything to update? »

I’m seeing one study broken out into a fair amount of depth. Pretty old content.

There’s too much change between 05 and 2008 for it to be relevant today.

Facilitator: “What captures your attention on this page?” Participant: “That it’s old. September 2006.”

I can tell you that if it didn’t have the page numbers on it, I would be more prone to click on it. Because that gets into my intimidation of lots of paper. Because that insinuates that I’d have to print it, id’ have to look, I’d have to work for it. It’s like, ugh, I just want it on the page.

I wouldn’t print more than 20 pages. When you think 40, you’re thinking wow, that’s a bundle.

I think when somebody’s coming or when I’m looking for an answer, I don’t want to pick up a book, I want to get information in consumable bites. I want something that’s not going to overwhelm me, I want something that I can print out and read on the metro on the way home. I want something that I can throw in my bag if I were going on a trip. And if I look at something and I think, “wow, it’s 40 pages,” the odds of me printing this, throwing it in my bag to go on a trip and reading it on the plane are very low.

Now I know I have to surf in a downloadable document, 192 pages. Talk about a personal productivity to find one thing.

Click directly on “Download” button to skip the abstract • Some percentage of users will click download skipping the full abstract entirely.

I never do that by the way [click ‘Abstract’], just for your information. An abstract is never very helpful to me and I would always get annoyed when I’m on here and see that.

“When I click through here [List Page] if it’s a great title then I click through here [Abstract Page] and, yeah, the bullets look right then you click into the document, ahh, it still wasn’t right. I almost want to get to the document quicker rather than having the stage set.”

Express confusion with “Recommend to Colleagues” and “Alert me of Similar Content” buttons

I’m not sure that these buttons over here are helpful at all…not while I’m doing my search anyway.

‘I like the alter me of similar content’ I could even see that as a separate function, instead of being a part of the same bar.

Express confusion with shading separating entries in the list • Some percentage of users thought the shaded entries were being called out as better

Is the shading for emphasis or visual distinction? Cause I would probably guess its featured research that’s being shaded.

Reference number of pages in deciding whether to download a document (5 of 7) • Documents that appeared promising to users were dismissed because of size

Recommendations 1. Present a title, single sentence with bullets in all abstracts for the list view. Abstracts in the list view formatted with a title and a one sentence followed by three bulleted sentences that fit on one line each was the most read in testing and reported by users to be most successful. To implement, the development team may need to create a new meta-data element for this abstract format for the list view [refer to “short abstract” in rest of report]. Our current system, which automatically takes the first 400 characters of the document’s full abstract, might not consistently present this information for every document. The Corporate Executive Board

Title & Abstract Testing

User Experience 2008

2. Remove “Recommend to Colleagues” and “Alert Me of Similar Content” buttons from the list view. On the list view entry of abstract, only present users with two buttons: “View Full Abstract” and “Download.” 3. Remove dates in the list view. An older date immediately turns off some users, regardless of how promising the document may be. Instead, show the date at the bottom of the abstract view to encourage them to read for relevancy before making a snap judgment. 4. Remove “Zebra Striping”. The alternating gray and white background confuses some users, as they think the gray indicates highlighting, suggesting importance. Instead, show all abstracts with white background and distinguish between different abstracts with light grey dashed lines used in the V2 style sheet. 5. Prioritize title and bullet rewriting efforts first. This user test found that a large percentage of users will always go immediately to download from the list view. Web analytics data from both Middle Market and Core web platforms confirm this behavior: in the list view, more than half of users click “Download” rather than navigating to the full abstract. [The only exception to this rule is on the Search pages of the Core websites - users almost always click the title, which leads to the full abstract page, as opposed to the “Download” button. This difference is most likely due to the position of the download button, which is different from the list view used in this test.] As such, titles and bulleted are the only components that will be seen by every user and thus they should be prioritized over the re-writing of full abstracts.

The Corporate Executive Board

Title & Abstract Testing

User Experience 2008

Titles | Content Findings What We Learned

Supporting Quotes »

Just the titles. I’m just reading the blue titles.

Again, totally commands my attention, the blue titles.

The first thing that I do is I screen the titles and none of them jumped out at me, so then I thought well, some of them are more broad than others, so what I’m looking at now is the abstract to see is there something that I might be able to apply or closely apply to what I’m working on.

I’d probably go to the ‘Sourcing Resource Center’ because I like that as a title.

I like the title it’s right on with the language for what I’m looking for.

For something as specific as ‘quality of hire’ I’d expect something a bit more specific.

“I had been working on…job descriptions for other jobs, so this one piqued my interest.”

I’m looking at titles for the most part and very quickly scanning companies and dates.

Its interesting, when I see this next set of cases I’m jumping to the GE, Oracle before I even get to the other sections (i.e., titles)

In this instance, I’m not interested. Unless the other part of the title is spot on, ‘how GE opens doors’ or ‘what it takes to open doors: GE’s view’ that’s great. Ut if it’s ‘GE Voice of the Customer Prioritization Tool’ It’s the combination of the two that’s most helpful.

I’d scroll through Victoria Secret, just because it’s Victoria Secret. He he.

It’s American Express so I would read it further.

I’m not even sure I know what that last word is in parenthesis…so I’d skip that one.

Express confusion with “CEBisms” in titles

What does “Glossy Book” mean to you? “I found it to be irrelevant. I guess it’s telling me where it came from.”

Express confidence with indicators of content are specific (Checklist, job description, template) • Word “tool” too general

This was an observed behavior more than quoteable.

Use titles as primary data point in deciding whether to download a document (7 of 7) • If they don’t read the title, or don’t find it useful, then will not read the abstract.

Scan for keywords in deciding whether to download a document (7 of 7) • More often this was an observed behavior – this was less quotable, but the behavior was noted across all participants. Scan for company names in deciding whether to download a document (7 of 7) • Company names secondary after keywords • Caution: they can also be problematic if the user doesn’t know the company

Recommendations 1. Use member keywords in titles. Do not use “CEBisms”. Our titles often include CEB keywords, such as full study names, chapter headings, product types, and other titling conventions which are appropriate for a document as a part of a book, a presentation deck, or internal initiative, but not helpful for members understanding whether a document answers a specific question. “Glossy Book”, “Decision Support Center”, and other CEBisms should be left out of any title. The Corporate Executive Board

Title & Abstract Testing

User Experience 2008

As such, use member keywords in titles. Applying recommendation will also improve search results of these documents. 2. Use profiled company names with caution. Well-recognized company names can add value, but if the user does not recognize a company name it can be confusing. Confusion consistently discourages users from reading more of an abstract or downloading. 3. Use descriptors of content types. Including terms like “Checklist”, “Job Description”, “Template”, etc, helps sets users expectations about the document. Less specific (often jargon filled) titles, such as “Business Alignment Tool” or “Member Generated Diagnostic” tend to be confusing. When using these generic titles, it is important that the rest of the abstract communicates plainly and clearly the purpose of the document or tool. 4. Prioritize titling work. Every user reads titles. The area where we can have the most immediate and substantial impact, is fixing bad titles.

The Corporate Executive Board

Title & Abstract Testing

User Experience 2008

Abstract View | Formatting Findings What We Learned

Supporting Quotes »

All of these would be hyperlinks that would allow me to not have to read in order to start absorbing.... I expect hyperlinks almost everywhere. I hate downloading. I’m a little bit allergic to it.

The first thing that I looked at after the title was “This research profiles:”

If it was a little more specific in terms of bullets…ideally if those were links to much more specific things within the document.

They’re just a lot more scanable. There’s the immediate action…I can scan these three bullets very quickly. I know that if it merited a bullet it’s probably more relevant than a paragraph for scanning. It’s more accessible to me.

The way the decision support center [abstract] was set-up was great. With two or three bullets describing what I’m going to see in here. This description here is just a description. Its not a table of contents.

Ignore full abstracts and go straight to the download or leave (2 of 7) • Some users prefer going straight to the document download • Other users get frustrated when receiving a full abstract and would leave rather than read a full abstract

“When I’ve gone into abstracts, I don’t find them that helpful. It’s more helpful for me to see the entire document so that I can pick and choose what I want to look at.”

“I’m almost kinds willing to commit up front you know, I can tell [value] by opening the document…I want to get to the document quicker.”

Use progressive investment • Leading user down the path to view a full abstract, there must be more value on the abstract view than the list view

I’m looking at what the case study says and it just basically says what I already saw. It’s just re-presenting it to me. […] I wouldn’t expect to have to click twice.

To me what helps is getting to successive specificity. Do you see enough that’s spot on that makes you go to that next layer. And do you see enough in the next layer that’s spot on that makes you go further.

“I hate coming to the download button every time. What a pain…that’s another step – seems like there’s so many steps to get the information…I’m giving up.”

Scan multiple sets of bullets within an abstract • Only text in abstract consistently read, other than title • Express interest in having the bullets to link to content within the document (3 of 7)

Recommendations 6. Use bullets to break up ideas within a full abstract. Of the users who read any part of the abstract, all read bulleted text. 7. Link the bullets to sections of content. While it should be investigated as to whether this could be done for a PDF or other downloadable document, this could be immediately implemented for Decision Support Centers. 8. Ensure abstract view have a longer, more complete abstract than the information presented in the list view. The Corporate Executive Board

Title & Abstract Testing

User Experience 2008

The summary they see on the abstract view cannot be the same they see of what they see in the list view. Users are frustrated when theyâ&#x20AC;&#x2122;re taken to an abstract page that doesnâ&#x20AC;&#x2122;t provide additional value.

The Corporate Executive Board

Title & Abstract Testing

User Experience 2008

Abstract | Content Findings What We Learned

Supporting Quotes

Scan for keywords within the abstract in deciding whether to download a document (7 of 7) • If no keyword seen, users will rely more heavily on the title

Scan for specific in the content • Business problem this doc can help solve • How this will help me with my problem

“It’s still not talking about the business problem and I believe that’s why anybody’s using this. There’s something that they’re either trying to prove, validate, or get under, and that would help.”

I think the more specific you can have the bullet be what is going to be in the content vs something that is more broad and generic…if there are specific things within the document, then the abstract tends to add value.

Scan bullets and bold text • When asked what they were reading, users would consistently note words that were in bullets or bolded headings.

This was more of an observed behavior than a quotable event.

Skip background information when presented first (5 of 7)

“When I look at this, I have to really read to understand that this is something that can relate to me, and so I read this and I see, “Okay, this is intended for broad distribution,” not really relevant, “Council’s research drawing on a variety of secondary sources,” – okay, so I know where the information’s coming from… and then I look at the topics. That’s what’s going to tell me whether or not I’m going to read it.”

I would probably expect to see that at the end, not at the beginning. I’m the type of person – I want to hone in on what I’m looking for.

“I can tell you that if it didn’t have the page numbers on it, I would be more prone to click on it. Because that gets into my intimidation of lots of paper. Because that insinuates that I’d have to print it, id’ have to look, I’d have to work for it. It’s like, ugh, I just want it on the page.”

181 pages….I’m not reading that one.

Reference number of pages in deciding whether to download a document (5 of 7) • Documents that appeared promising to users were dismissed because of size

This was more of an observed behavior than a quotable event. All users tended toward scanning for keywords. “It would be the title probably. It would have to be something that’s catchy. […] It’s a zoned out sort of approach.”

Recommendations 1. Use member keywords in titles. Like titles, users scan abstracts for the keywords related to the issue they are trying to solve. Member’s keywords (i.e., how they describe problems) should be included in abstracts whenever possible. Including more keywords in our abstracts will not only enhance users scanning documents, but it will also help our search functionality. 2. Be specific in the description of the document to be downloaded. The Corporate Executive Board

Title & Abstract Testing

User Experience 2008

The abstract must outline what value the user can expect to get from the document and how the document will provide that value. (ex. ‘Build a Disaster Recovery framework using these templates’). Although it might take a change to our backend systems and how files are uploaded to the site, but when possible, ideally the items or sections of a document referenced in the abstract should be linked directly to those related sections in the document download. 3. Remove background information from the opening lines of abstracts. The beginning of each abstract should describe what’s in the document and how they can use it. The format should be a title, one sentence then three bullets. [NOTE: ROUND 2 OF TESTING WILL EXPLORE THIS FURTHER]. 4. Put document length and date published at the bottom of the abstract. Users reference document length and published dates in determining whether or not to download a document. Long and old documents often dissuade users from downloading relevant documents. Placing the document length and date at the bottom will encourage members to dig into relevant documents that are older.

The Corporate Executive Board

Title & Abstract Testing

User Experience 2008

Document Downloads Findings What We Learned Express disappointment when there is a disconnect between the abstract and the download

Scroll through documents to find relevant information • Often, then sought out the TOC immediately

Supporting Quotes »

It’s somewhat, I don’t want to say misleading but it changed the way I expected to receive the information

The document doesn’t deliver on the promise of the abstract. A lot of times the abstracts are probably right and the cases are probably right, but it’s back to the higher order change of writing in a different way…some of the abstracts probably aren’t bad for what people are looking for, but when they get there, oh my gosh, its just not written in a way to get you the information quickly.”

Commenting on the utility of a diagnostic: “Ok, so you grade me? But, alright, so what?”

Well I would see if there’s a table of contents. […] If I could, I would probably -- because I’ve got that meeting tomorrow morning -- go to Chapter 3 or 5 for the key findings because I don’t have a lot of time.

I would look at the table contents to figure out if I really want to read it.

Recommendations 1. Indicate clearly to users that older studies still have relevance today. In both the list view and abstract view, users dismiss documents with an older date. If the study is still relevant, review or update it so the date can be changed to a recent date. Anything more than a year old would draw a comment from those folks who were date sensitive. 2. Format documents to be consumed on the web. No matter how good we make the title and summary, we have an opportunity to disappoint if the document does not fulfill the promise. Some documents and tools looked half-baked or pulled out of context. This is a much larger issue, completely outside the scope of title and abstract rework.

The Corporate Executive Board

Title & Abstract Testing

User Experience 2008

Appendix Testing Method Details | Round One The Round One test was modeled after a methodology used in a published study on the abstracts and documents on the New York Times website 1. In this testing, each subject, seven in total, was asked to perform the same six tasks on working prototypes of the Core member sites. Each task started on the browse page of the Core member site and was considered completed when the test subject had downloaded and evaluated a document. After each task, the test facilitator returned the test screen to the browse window to begin the next task. Test subjects were asked to talk aloud about their thinking as they completed the tasks and a series of questions were asked. At the end of each task, users were asked to fill out a brief four questionnaire on their experience [the questionnaire did not reveal consistent results, so they were not used]. Overview of Tasks and What Format Was Tested Task Task Description 1 Subject asked to browse for topic using current member site via the taxonomy and current abstracts 2 Subject asked to identify a specific issue within a defined topic and browse to find a relevant document using the prototype. 3 Subject asked to identify a specific issue within a defined topic and browse to find a relevant document using the prototype. 4 Subject asked to identify a specific issue within a defined topic and browse to find a relevant document using the prototype. 5 Subject asked to identify a specific issue within a defined topic and browse to find a relevant document using the prototype.

What Format Was Tested Current Site Titles Only

Titles, One Sentence

Titles, Sentence Two Bullets

Titles, Paragraph

Test subjects were interacting with the test website via a 22-inch widescreen monitor set to ?x? in 32-bit resolution attached to a Dell Latitude x400 laptop with an external mouse. The prototype site was being served locally for tasks 2-5, while task 1 used the live core member site. The user testing was recorded using Morae, which was installed on the Dell laptop. One facilitator was responsible for guiding the subject through the testing protocol and asking questions, while one or two observers watched from the Dell Latitude monitor, which was set to mirror the test subject’s view. To test any differences between the programs, we mocked up and tested users in three different domains (no differences were found). Note on Test Variances: In first three of the tests, we noticed we had an additional task, which tried to measure the difference between the content and length of titles but test subjects were not able to tell the differences between the different formats.

Test Subjects | Round One To test for any differences between the program members, we conducted Round One of testing with seven people from the CEB with domain experience for each of the three programs prototyped. Of these seven people: two from CEB’s Marketing department, three from CEB’s Human Resources department (TMT), and two from CEB’s Information Technology Function (ITG). All of the test subjects had some familiarity with core member websites, and three test subjects had used them had used the site to help them in their jobs.

“Reading Online News: A Comparison of Three Presentation Formats.” Ryan Baker, Michael Bernard, & Shannon Riley

http://www.hci.gr/files/surl/Reading_%20Online_%20News.doc The Corporate Executive Board

Title & Abstract Testing

User Experience 2008

Test Scripts | Round One Round 1 CLC

Introduction

Thank you for participating in this session! We’re getting feedback on how information is presented on our websites. As part of the process, we’re asking people to attempt a series of tasks to see what types of information may be challenging to use. There are six tasks that I’ll give you to complete. As you use the site, I’ll be asking you to ‘think aloud.’ For example, if you don’t know what something is for, please say ‘I don’t know what this is for.’ I may also prompt you from time to time to ask you what you are thinking. We’re going to be looking at a prototype and not a finished product, so there will be some nonfunctioning components of the site, which we might bump into over the course of the session. To help us get as much value from our time together today as we can, we’ll be recording our conversation, as well as our movements on the screen. Any information from this test will be kept anonymous and aggregated with the other subjects in this study. Are you comfortable with us recording? Do you have any questions before we get started?

Background Questions

• What are some sites that you use to help you do your job? (open one of the sites) • Thinking about this site: • Why do you come to this site? • Show me how you locate relevant information. (search/browse) • How do you determine whether or not to 1) download a file and 2) follow a link? • What do you do when you can’t find what you’re looking for on this site? • Have you ever used a CEB member website? If so, have you ever used it to address a problem you were facing at work?

Task 1

[Go to clc.executiveboard.com and select Research & Tools]

Self-Directed Browse

To begin our first task, think of a problem relevant to your day-to-day work. What is it? Using the left navigation on this page, locate and download a document or other resource that you think would be helpful. [On Browse Page]

A. What draws your attention on this page? B. What information are you looking at? C. What do you see that helps make your decision to click on something? [On Abstract Page]

Don’t click on anything yet…. D. What draws your attention on this page? What helps you to determine if you’re going to download this document? E. Do you think this document is going to contain the information you are looking for? How can you tell? F. What do expect is behind this download button? Describe the content and format. [On Download]

G. Was there anything surprising about the download? H. Do you think this document contains information that would be helpful? ADMINISTER QUESTIONNAIRE

The Corporate Executive Board

Title & Abstract Testing

Task 2 Existing Titles

User Experience 2008

For the next few tasks, I’ll be giving you topics. It may not be something you’re working on right now, but think about an issue you may have dealt with or may have to deal with and find a document that you think best addresses the issue. [Select Employee Engagement & Retention from the left navigation] [On Browse Page]

A. What draws your attention on this page? B. What information are you looking at? C. What do you see that helps make your decision to click on something? [On Abstract Page]

G. Was there anything surprising about the download? H. Do you think this document contains information that would be helpful? ADMINISTER QUESTIONNAIRE

Task 3 New Titles

For the next few tasks, I’ll be giving you topics. It may not be something you’re working on right now, but think about an issue you may have dealt with or may have to deal with and find a document that you think best addresses the issue. [Select Recruiting from the left navigation] [On Browse Page]

A. What draws your attention on this page? B. What information are you looking at? C. What do you see that helps make your decision to click on something? [On Abstract Page]

G. Was there anything surprising about the download? H. Do you think this document contains information that would be helpful? [Display 2 sets of titles] Looking at these two sets of titles: I. Is one set more effective or clear than the other? Why? ADMINISTER QUESTIONNAIRE

Task 4 Titles, One Sentence

The Corporate Executive Board

Title & Abstract Testing

User Experience 2008 [On Browse Page]

A. What draws your attention on this page? B. What information are you looking at? C. What do you see that helps make your decision to click on something? [On Abstract Page]

G. Was there anything surprising about the download? H. Do you think this document contains information that would be helpful? ADMINISTER QUESTIONNAIRE

Task 5 Titles, Sentence Two Bullets

For the next few tasks, I’ll be giving you topics. It may not be something you’re working on right now, but think about an issue you may have dealt with or may have to deal with and find a document that you think best addresses the issue. [Select Recruiting from the left navigation] [On Browse Page]

A. What draws your attention on this page? B. What information are you looking at? C. What do you see that helps make your decision to click on something? [On Abstract Page]

G. Was there anything surprising about the download? H. Do you think this document contains information that would be helpful? ADMINISTER QUESTIONNAIRE

Task 6 Titles, Paragraph

For the next few tasks, I’ll be giving you topics. It may not be something you’re working on right now, but think about an issue you may have dealt with or may have to deal with and find a document that you think best addresses the issue. [Select Employee Engagement and Retention from the left navigation] [On Browse Page]

A. What draws your attention on this page? B. What information are you looking at? C. What do you see that helps make your decision to click on something? [On Abstract Page]

Don’t click on anything yet…. The Corporate Executive Board

Title & Abstract Testing

User Experience 2008

D. What draws your attention on this page? What helps you to determine if you’re going to download this document? E. Do you think this document is going to contain the information you are looking for? How can you tell? F. What do expect is behind this download button? Describe the content and format. [On Download]

G. Was there anything surprising about the download? H. Do you think this document contains information that would be helpful? ADMINISTER QUESTIONNAIRE How could we format the information to be more readable and understandable?

General Feedback

[Show print outs] What are your general impressions of the site?

Round 1 CIO

Introduction

Background Questions

Task 1

[Go to cio.executiveboard.com and select Research & Tools]

Self-Directed Browse

The Corporate Executive Board

Title & Abstract Testing

User Experience 2008

A. What draws your attention on this page? B. What information are you looking at? C. What do you see that helps make your decision to click on something? [On Abstract Page]

G. Was there anything surprising about the download? H. Do you think this document contains information that would be helpful? ADMINISTER QUESTIONNAIRE

Task 2 Existing Titles

For the next few tasks, I’ll be giving you topics. It may not be something you’re working on right now, but think about an issue you may have dealt with or may have to deal with and find a document that you think best addresses the issue. [Select Project Management from the left navigation] [On Browse Page]

A. What draws your attention on this page? B. What information are you looking at? C. What do you see that helps make your decision to click on something? [On Abstract Page]

G. Was there anything surprising about the download? H. Do you think this document contains information that would be helpful? ADMINISTER QUESTIONNAIRE

Task 3 New Titles

For the next few tasks, I’ll be giving you topics. It may not be something you’re working on right now, but think about an issue you may have dealt with or may have to deal with and find a document that you think best addresses the issue. [Select IT-Business Engagement from the left navigation] [On Browse Page]

A. What draws your attention on this page? B. What information are you looking at? C. What do you see that helps make your decision to click on something? [On Abstract Page]

Don’t click on anything yet…. D. What draws your attention on this page? What helps you to determine if you’re going to download this document? The Corporate Executive Board

Title & Abstract Testing

User Experience 2008

E. F.

Do you think this document is going to contain the information you are looking for? How can you tell? What do expect is behind this download button? Describe the content and format.

[On Download]

G. Was there anything surprising about the download? H. Do you think this document contains information that would be helpful? Display 2 sets of titles Looking at these two sets of titles: I. Is one set more effective or clear than the other? Why? ADMINISTER QUESTIONNAIRE

Task 4 Titles, One Sentence

For the next few tasks, I’ll be giving you topics. It may not be something you’re working on right now, but think about an issue you may have dealt with or may have to deal with and find a document that you think best addresses the issue. [Select IT-Business Engagement from the left navigation] [On Browse Page]

A. What draws your attention on this page? B. What information are you looking at? C. What do you see that helps make your decision to click on something? [On Abstract Page]

G. Was there anything surprising about the download? H. Do you think this document contains information that would be helpful? ADMINISTER QUESTIONNAIRE

Task 5 Titles, Sentence Two Bullets

For the next few tasks, I’ll be giving you topics. It may not be something you’re working on right now, but think about an issue you may have dealt with or may have to deal with and find a document that you think best addresses the issue. [Select IT-Business Engagement from the left navigation] [On Browse Page]

A. What draws your attention on this page? B. What information are you looking at? C. What do you see that helps make your decision to click on something? [On Abstract Page]

Title & Abstract Testing

User Experience 2008

G. Was there anything surprising about the download? H. Do you think this document contains information that would be helpful?

ADMINISTER QUESTIONNAIRE

Task 6 Titles, Paragraph

For the next few tasks, I’ll be giving you topics. It may not be something you’re working on right now, but think about an issue you may have dealt with or may have to deal with and find a document that you think best addresses the issue. [Select Project Management from the left navigation] [On Browse Page]

I. J. K.

What draws your attention on this page? What information are you looking at? What do you see that helps make your decision to click on something?

[On Abstract Page]

Don’t click on anything yet…. L. What draws your attention on this page? What helps you to determine if you’re going to download this document? M. Do you think this document is going to contain the information you are looking for? How can you tell? N. What do expect is behind this download button? Describe the content and format. [On Download]

O. Was there anything surprising about the download? P. Do you think this document contains information that would be helpful? ADMINISTER QUESTIONNAIRE How could we format the information to be more readable and understandable?

General Feedback

[Show print outs] What are your general impressions of the site?

The Corporate Executive Board

Title & Abstract Testing

User Experience 2008

Usability Testing Round 2: Content Executive Summary Overview The Corporate Executive Board’s (CEB) Core and Middle Market member websites are document centric, i.e., most of the information on these sites is offered as downloads, such as PDFs, MS Word, etc. In nearly all cases on the Core and Middle Market websites, the document abstract is presented on the site and is the way members determine whether or not to download a document. Despite providing abstracts to nearly all documents, members express an inability to find information on the CEB member websites. Analysis of these complaints suggests that members cannot find the information because the relevant content is not on the site, but rather because it is not clear to members what documents hold the relevant the information The goal of this project, sponsored by Glenn Tobin, is to identify the most effective abstract for the CEB to present its research and resources on its core member websites. By improving the quality of abstracts, CEB will enable more members to find the information on its sites. Testing Usability research has demonstrated that readability and findability of information on websites is impacted both by format and content presented. To isolate the impact of content and format, the testing was broken up into two rounds: Round One-format and Round Two-content. The following report only overviews Round Two of the testing. Round Two testing was conducted with five participants from CEB being asked to compare different types of abstracts. Please see the Appendix for full details on the test methodology. Highlighted Recommendations Five key recommendations emerged from Round Two which are discussed in detail in the report. 9. In the list view, two types of content were more helpful than others: a. action-oriented bullets, describing how the user can take action b. bullets outlining sections or chapters of the document, giving a roadmap 10. Do not make assumptions about the reader. Do not describe who a report is ideal for. Do not use “You will learn how to”, as it implies that the user doesn’t already know something about a topic. Instead, tailor wording to make the reader feel empowered. 11. Meet the user’s progressive investment with progressively more information at each click. The abstract should contain more content than the list view description, but not so much that the user is overwhelmed. Do not include extra content that belongs in the document itself (such as Methodology, Distribute To). Information presented should grow as we move down the path: List ViewAbstractExec SumDocument. 12. Use headings and bullets in abstracts to break up content. As users scan, they read the headings to determine if the bullets are worth reading. If the heading is poorly worded or abrasive to the user (e.g. “This research is ideal for”) the bullets are less likely to be read. 13. Hyperlinking directly to portions of a document are valuable when done correctly, but can be problematic. Only provide hyperlinks to sections of document if two criteria are met: a. The title of the page in the document matches the title of the link, or is extremely similar The Corporate Executive Board

Title & Abstract Testing

User Experience 2008

b. The page in the document can stand alone, and the user does not need to scroll through the document to gain the context needed to understand it.

The Corporate Executive Board

Title & Abstract Testing

User Experience 2008

Example: https://www.clc.executiveboard.com/Members/ResearchAndTools/Browse.aspx?tid=/2272/2321 Abstract View: Pages members see when browsing the taxonomy on the current core websites.

Example: https://www.clc.executiveboard.com/Members/ResearchAndTools/Abstract.aspx?cid=100063372&tid=/2272/232 1

The Corporate Executive Board

Title & Abstract Testing

User Experience 2008

List View | Content Findings What We Learned Mixed views on section/chapter titles: Most found them to be valuable, while others felt they were too vague.

Supporting Quotes »

“I like a roadmap in general, I like to know where I am, so there’s

“It’s very clear. I will know exactly what to get if I click and download. All the others are a lot of extraneous words, not very clear, I have to think “now wait a minute, let me parse out these long phrases.” Here I actually get to choose what I’m looking for because it’s the broad topic area.”

If I view the abstract or download, I know exactly what I’m going to find, I know exactly the topics I’m going to read about. And I can tell at a glance if those topics are going to be helpful to the questions that’s in my head.

I expect to drill down a little bit further, I appreciate something that gives me a little bit of a road map to what I’m about to drill into

They’re too broad, too vague, and it could go in a million different directions. I want something a little bit more tactical, a little more specific

If I’m going after an information resource I’d like something to be fairly broad and I’m confident in my ability to drill down to what I need. if I need something comprehensive, I can go there, and if I don’t, I just need something targeted, at least I can go to that portion of it and know exactly what I’m going to get. Wordy and lofty. Skims across the top without going into any specific details.

» »

Action oriented and “how to” descriptions resonated with users.

Audience-based descriptions are not preferred.

The Corporate Executive Board

something nice about outlining how I could expect to see it, then I would be able to jump into the report I suppose.”

I like the solutions or what I would get out of it. I like the fact that these are actions that I could take and improve or make my problem go away.

I know what’s going to be here. It looks to be solution oriented rather than statement of fact and teaching solutions.

None of these is hitting it for me. A lot of these talk about the what, but it doesn’t tell me about the how.

Something that’s a little bit more prescriptive would give me confidence that I’m going to get something out of it that I can really use by the end of the read.

If I’m doing this type of search, I’m probably trying to answer the question “What should I do?” And there was nothing in any of them that actually just said, based on the study, companies should do this.

I’m on the website, I’m looking at stuff, I don’t need you to tell me who it’s ideal for. It’s really not your concern if it’s ideal for me or not. That’s my decision to make.

I don’t think in terms of the type of company that I am; I’m just looking for the best ideas. It may be for a company that’s similar to me or it may be a totally different company but there are shareable tactics there. It doesn’t matter to me if it’s ideal for one type of 71

Title & Abstract Testing

User Experience 2008

company or another. I’m looking for the idea, not who it’s ideal for.

Terminology Preferences

Not what types of individuals or companies this would be ideal for, but what types of scenarios or what types of situations that you would be going through. So if you’re doing one of these things, take a look at this. That’s the thing that’s a little bit nuanced than what I we looked at before.

Terminology around “you will learn how to” suggests I don’t know how to do stuff like that and I would like to think I do. I’m looking for language around designing, developing, problem/solution, helps. It strikes me as part of a curriculum rather than giving me a way of solving. It may be a visceral reaction to “Gain an edge over your competition” – it sounds salesy and shticky. It comes across as less credible to me in terms of an information resource

Users’ top priority is time investment:  Must be able to quickly determine if they should invest more time in the document or move to another  If clicking into an abstract, must get more content than was originally on the list view (progressive investment)

I may not care if I gain an edge over my competition – that’s for me to decide

It feels very narrow but ultimately that can improve my efficiency so I know not to click on it if I’m not interesting in learning in 2 things. If your goal is to get people to click on things and explore, that hinders that. On the flip side, if I’m looking to be very targeted and I need something quick, I appreciate that I’m not clicking on that. » If this isn’t what I need, I still have time to look at another abstract. » If what was on the front page was enough o make me think this is worth drilling in to, I’d be more appreciative to get this information right off the bat rather than having to click through to another page that’s going to give me the information to help me make that decision. »

» The high level summary is key. The thing that we’ve been really bad at is understanding how time-compressed people are.

Recommendations 6. In the list view, two types of bullets were more popular than the others and should be used moving forward: a. action-oriented bullets, describing how the user can take action b. bullets outlining sections or chapters of the document, giving a roadmap 7. Terminology and writing style should be determined according to the audience. HR users disliked wording that was sales or marketing oriented (ex: “Gain an edge over your competition…”) and tended to skip over it. 8. Do not describe who a report is ideal for. Instead, describe the situation it is ideal for. 9. Do not make assumptions about the reader. For example, stating “You will learn how to” implies that the user doesn’t already know something about a topic. Instead, tailor wording to make the reader feel empowered. Next steps: In the next round of testing, we will focus on two types of abstracts in the list view: Sections/Chapters and How Tos.

The Corporate Executive Board

Title & Abstract Testing

User Experience 2008

Abstract View Findings What We Learned Amount of content is key.  Too little content frustrates users (progressive investment)  Too much content is overwhelming

Supporting Quotes »

“You’re making me do a lot of work… it’s really dense

“I see this and I all of a sudden get scared, because I have 10 spare minutes and I’m trying to see if it’s worth clicking through, worth printing out a hard copy that I can take home with me… but now I’m scared and annoyed and I just go away. I don’t want visual clutter, I don’t want to have to keep reading, I want to know in those 3 bullets if this study will help me answer my questions. I felt like I just had to click twice to get to the information I was looking for in the first place.”

and way too many words on the page”

» If what was on the front page was enough to make me

think this is worth drilling in to, I’d be more appreciative to get this information right off the bat rather than having to click through to another page that’s going to give me the information to help me make that decision.

4 out of 5 users did not notice the bulleted links to sections of the document All users noticed link to Executive Summary and knew what to expect when clicking it.

Disconnect between the link and document

“I’m drawn to the Read Executive Summary. It’s an action, it’s short, you’ve highlighted that.”

“I like the link to Executive Summary. I expect a page, limited text, chunked, easy to read, and gives me the highlights.”

“I expect a page and a half of text. Very simple, not dense, but an executive summary – what are the key things I’m going to get, what are the highlights.”

“Oh, no no no! Nooo! Oh dear god. I don’t even know

“This look great as part of a study but I want it to tell me information I can use or discard. I don’t want to have to interpret.” I think you’re making me think too hard…I’m looking at this and I don’t get an instinctive tie back to the link I clicked. I feel I want to go up or down the page to contextualize it. Any other website I use – I don’t have to contextualize anything. When I click on a link, it’s exactly what I want and what I expect and I don’t have to work out how I got taken to this page.

Mixed feelings about linking directly to executive summary:  3 preferred this method The Corporate Executive Board

where to begin. I was not expecting a chart with 27 choices across the bottom. This is so CEB. No, not this. Almost the anti-this.”

It takes me a little while to figure out what I should be looking at on the page.

“It’s not sold as an executive summary. Click on the link

“My preference would be going straight to the Executive

– my expectations have not been matched.”

Title & Abstract Testing

User Experience 2008

Summary page. I felt like I just had to click twice to get to the information I was looking for in the first place.”

 1 had no preference  1 felt it would be misleading

Bullets and clear headings make or break the page

“Interim step seems unnecessary to me.”

Chunking is good. The bullet points make it easy to read

If it wasn’t signposted as well as it is and it wasn’t bulleted out, it was just text and cumbersome to get through, that would annoy me. The layout is actually really good.

Buttons carry a lot of weight on the page

» »

Anything offered in the format of a webpage:  Should be re-written for the web  Should contain links if appropriate

» » » » »

“Visually my eyes are drawn to the buttons at the bottom of the page, not the text.” “First thing I looked at was the Download button.” “I get to the end of a paragraph – and what now? Give me more. Where do I go to from here to get more on that thing? ‘More information’ link.” “The option to dig deeper on the specifics would be great, assuming there’s content that answers the how and not just the what.” “It’s the right amount of information, but the language is written for the books. Stinted for the website. Feels like a book on a website.” “The left column would be links.” “It’s all static. I suppose there are some things at the bottom that give you actions that you can take – I assume those are in the study or research, so you could just link from here to those sections.”

Recommendations 14. Abstract must contain more content than the list view description, but not so much that the user is overwhelmed. Do not include extra content that belongs in the document itself (such as Methodology, Distribute To). 15. Always user headings and bullets to break up content. 16. Clearly indicate that links are clickable so that they aren’t overlooked by users. 17. Only provide hyperlinks to sections of document if two criteria are met: a. The title of the page in the document matches the title of the link, or is extremely similar b. The page in the document can stand alone, and the user does not need to scroll through the document to gain the context needed to understand it. 18. Any content offered in the format of a webpage needs to be re-written for the web and should contain links if possible, to meet users’ expectations. Next Steps: In the next round of testing, we will:  determine preferences between linking directly to a lengthy, more interactive Executive Summary or to a shorter, more static abstract  explore the ways in which people would use the Executive Summary page, and the actions they expect to take (such as email, recommend, or print)  identify the best way to indicate that links are clickable, so that users don’t overlook them

The Corporate Executive Board

Title & Abstract Testing

User Experience 2008

Appendix Testing Method Details | Round Two In this testing, each subject, five in total, was asked review a series of printed and interactive prototypes of the CLC website. Overview of Tasks and What Format Was Tested Task 1 2 3 4

Task Description The user read through 4 different list view descriptions for a single study, then identified likes and dislikes. The user read through 4 different list view descriptions for a single study, then identified likes and dislikes. The user read through 4 different list view descriptions for a single study, then identified likes and dislikes. The user read through 4 different list view descriptions for a single study, then identified likes and dislikes.

5 6 7 8

Comparison of Abstract Views

What Format Was Tested 4 variations of List View 4 variations of List View 4 variations of List View Abstract View: 3 sets of bullets Abstract View: Bulleted links Abstract View: Link to Executive Summary Abstract View: Executive Summary All Abstract Views

Test Subjects | Round Two We conducted Round Two of testing with five people from CEB’s Human Resources department (TMT). All of the test subjects had some familiarity with core member websites, and four participants had used them to help them in their jobs.

Test Script | Round Two Introduction

Thank you for participating in this session! We’re getting feedback on how information is presented on our websites. As part of the process, we’re asking people to attempt a series of tasks to see what types of information may be challenging to use. We’re going to be showing you some print-outs and having you view webpages, as well. As you look at what we’re presenting, I’ll be asking you to ‘think aloud.’ For example, if you don’t know what something is for, please say ‘I don’t know what this is for.’ I may also prompt you from time to time to ask you what you are thinking. We’re going to be looking at a prototype and not a finished product, so there will be some nonfunctioning components of the site, which we might bump into over the course of the session. To help us get as much value from our time together today as we can, we’ll be recording our conversation, as well as our movements on the screen. Any information from this test will be kept

The Corporate Executive Board

Title & Abstract Testing

User Experience 2008

anonymous and aggregated with the other subjects in this study. Are you comfortable with us recording? Do you have any questions before we get started?

Background Questions

• Have you ever used a CEB member website? • If so, have you ever used it to address a problem you were facing at work? • What did you do on the site?

Set up

For the purposes of this testing, we’re going to pretend you’ve recently been asked to look into [some topic]. In looking for this topic on the site, you’ve run into a page that looks like this [show list view on the screen, preferably with relevant content]

Task 1

[Put printed page on table: list view for document 1 w/3 types of bullets]

Document #1 List View (Paper)

Q. All three of these descriptions are for the same document. Take a minute to read through them. Which one do you prefer? Why? R. What do you like/dislike about each one? What information is the most helpful in determining if you’re going to follow the link?

Task 2

[Put printed page on table: list view for document 2 w/3 types of bullets]

Document #2 List View (Paper)

This is the same idea, but for a different document. A. B.

All three of these descriptions are for the same document. Take a minute to read through them. Which one do you prefer? Why? What do you like/dislike about each one? What information is the most helpful in determining if you’re going to follow the link?

Task 3

[Put printed page on table: list view for document 3 w/3 types of bullets]

Document #3 List View (Paper)

This is the same idea, but for a 3rd document. A. B.

Task 4

[Display list view for Document 1 in browser]

Document #1 Abstract View #1 (Computer Screen)

[On List View]

You might recognize this – it’s one of the print outs we were just looking at. I.

What do you expect to happen if you select one of these links?

You can go ahead and click on the first link now. [On Abstract Page]

J. K. L. The Corporate Executive Board

What draws your attention on this page? What part of this is the most helpful? Was there anything surprising about this page? Did it meet your expectations? Would you want to download this document? Why? 76

Title & Abstract Testing

User Experience 2008

Task 5

[Display list view for Document 1 in browser]

Document #1 Abstract View #2 (Computer Screen)

[On List View]

What do you expect to happen if you select one of these links?

You can go ahead and click on the first link now. [On Abstract Page]

B. What draws your attention on this page? What part of this is the most helpful? C. Was there anything surprising about this page? Did it meet your expectations? D. Would you want to download this document? Why?

Task 6

[Display list view for Document 1 in browser]

Document #1 Abstract View #3 (Computer Screen)

[On List View]

What do you expect to happen if you select one of these links?

You can go ahead and click on the first link now. [On Abstract Page]

Task 7

[Display list view for Document 1 in browser]

Document #1 Abstract View #4 (Computer Screen)

[On List View]

What do you expect to happen if you select one of these links?

You can go ahead and click on the first link now. [On Abstract Page]

General Feedback

How could we format the information to be more readable and understandable? What information is the most valuable to you as you use our websites to find information? [Show print outs of 4 abstract page variations]

The Corporate Executive Board

Title & Abstract Testing

The Corporate Executive Board

User Experience 2008

Workplan Generator

User Experience 2008

Workplan Generator April – July 2008

Project Summary Purpose Before building this new tool on the website, we wanted to gauge the level of member interest and find out if and how they would use it.

Process • • •

Series of phone interviews with FLEx members Using GoToMeeting to project our screen, members were asked to use a prototype of the tool Prototypes varied as we got further along in the process and completed more interviews

Example of early prototype

Example of late prototype

Summary of Findings & Recommendations • • •

Nearly every person we talked to could identify multiple projects that they could use a Workplan Generator for. The most positive responses surrounded the concept of the tool, having our resources organized in the form of a process which could then be personalized and saved for later. Opinions varied on the more sophisticated features. Specific requirements for design: o Process must be illustrated clearly on the entrance page, so that users understand what they’re about to do. o Members must be able to access their workplan on the website. They want the ability to have a link to it in an email rather than including all of the content (which could be large) o Workplan Generator must be highly customizable in adding steps or assigning tasks o A survey used to gather information so we can recommend steps, if included, must be optional. Opinions varied greatly about whether or not it was wanted. Some said it would depend on the number of steps and their level of familiarity with the topic.

Outcome The Workplan Generator tool was launched in the October 2008 release. Prioritization of which features to build were based largely on member feedback. While the concept was well tested, the success of each Workplan Generator created is primarily dependent on the value of the content. The framework will do nothing to add value to members without good content. Research teams should also interview members to determine the value of the content. The Corporate Executive Board

Workplan Generator

User Experience 2008

In the coming months weâ&#x20AC;&#x2122;ll need to loop back with the research teams to understand the adoption of these tools by our members and how it is providing value.

The Corporate Executive Board

Workplan Generator

User Experience 2008

Usability Testing Methodology Tested with 6 FLEX members Each session was conducted over the phone for 30 minutes, showing a Powerpoint prototype of the Work Plan Generator. After the first 2-3 sessions, the prototypes were slightly edited based on feedback we received so far. Members were asked to provide feedback on 2 variations of the Work Plan Generator: a simple list format and a project management format.

   

Summary of Findings Finding

Supporting Quotes

Users want the ability to return to the workplan on the webpage and manage it from there.

⇒

“If I’m going to be looking at something often, I prefer a link to a webpage, if not, everything in an email.”

With the exception of one person, no one was interested in adding their own steps.

⇒

“I don’t want to add in my own steps.”

⇒

“I don’t want the ability to add steps or upload attachments because if I’m using a tool like this, I wouldn’t know what to add or upload. We’re going here for research and we’re looking for the information.”

⇒

“I like web-based because it’s all here and doesn’t need to be maintained” “I’d like to manage projects on the website”

⇒ “I would only add in steps if my needs aren’t met. There could be a

message, “If no recommendation seems relevant, go back to questions or list of other potential recommendations that didn’t get chosen.”

Members were indecisive about whether they preferred a questionnaire or laundry list of steps, and said it would depend on two factors: 

Length of list of steps



Familiarity with subject

Everyone could identify other areas that they would like to see the Work Plan Generator tool applied to.

The Corporate Executive Board

⇒

“I’d prefer to jump straight into this, but want the options: answer questions for a customized solution OR go straight in. “

⇒

“It would depend on how comfortable I am with the topic. In this case, I’d want the questionnaire. “

⇒

“I want to be able to jump right in. Concerned that many people won’t understand the questions in the questionnaire, might make the wrong decisions, and as a result the steps might be incomplete.”

⇒

“It would depend on the length of the list of steps. I like the approach of asking questions to hone down. If it’s specific enough, then I like the questionnaire approach, but otherwise it’s easier to go through the steps.”

⇒

“I want the option to get more specific information so that I can think about it myself and prioritize. I personally prefer a laundry list, but if the list is long, minimize it with questions. No way to know what the reader might want to see.”

⇒

“FAS 144, Acquisitions, Annual audit (getting ready for quarterly and annual closes/audits), research on business in other countries (taxes)”

⇒

“FIN 48 dashboard, competent metrics to gauge effectiveness of tax department (a dashboard that is easy to digest and could be given to non-tax people), independent contractor policies.”

⇒

“A ton. We’re going through an acquisition and want to make sure they merge in with us correctly. Recently it would have been nice to have something like this to see where we were in the process. “

⇒

“Year end tax provisioning, FIN 48, income tax compliance process, developing a tax risk policy, risk strategy”

⇒

“FIN 48, questionnaires and stuff that other people use. “

Workplan Generator

Before viewing the Work Plan Generator, most had an accurate idea of what to expect.

In terms of emails, users expressed interest in receiving two types: ▪ ▪

Link to workplan/documents Editable Word or Excel file containing workplan and links to documents

Users preferred the second “project management style” version shown

Users saw a need to contact co-workers directly regarding the work plan, instead of having it automated through the website. They want to add specific instructions or notes to any emails that are sent out to coworkers.

The Corporate Executive Board

User Experience 2008 ⇒

“I expect it to lead me through step by step, starting with the top level and getting more detailed. Expecting MS Project or Excel list of steps to consider on quarterly or annual basis. Would like to see it tailored for public or non-public companies. “

⇒

“I expect a list of information related to the topic -- all associated resources, checklists, Excels and PDFs, quickly and succinctly, also link to questions about each topic. I’m always looking to be more efficient. “

⇒

“I expect some kind of web-based data entry sheet that allows me to input date or structure.”

⇒

I’d prefer to get links in the email because I won’t have to comb through information.

⇒ “I prefer links since documents can be very large.” ⇒

“I’d like a bulleted word document or an excel file that takes me through the work flow, step-by-step; I’d like to manage projects on the website.”

⇒

“The email would give me access back to this page”

⇒

“From a visual standpoint the steps are better. I don’t need checkboxes. I like the status and sending related documents and due date to a person.”

⇒

“I like this better because it’s more tabular. I would definitely use the extra features but would prefer to distribute it myself.”

⇒

“I like the information of owner/status, ability to add/delete step. This is great.”

⇒

“If this was emailed to me, I’d want more than this information. There are 12 ways to do this; how do you want me to do this? Needs context. Am I supposed to do this? Are you working on this already and this is just an FYI? Would like the option to add details before emailing this to a team member. To notice an email, it must come from a manager. If I’m really busy, I look very quickly and then file into a folder to go back when I have time.“

⇒

“I would definitely use the extra features but would prefer to distribute it myself. But if we got bigger as a company, I’d like to use an automated workflow. It has to be the right setting. I would not go this route myself and would follow up on my own.”

⇒

“Need to be able to edit and pass around for feedback/proofreading”

Workplan Generator

User Experience 2008

Script and Prototypes As we look through the slides, try to think out loud and imagine how you would be viewing this on the website. Please share anything that comes to mind. We’re going to be looking at a prototype, not a finished product, which means that you don’t have to pay too much attention to the text content. Slide 1  What is a work plan generator? Where would you expect these links to take you?  What do you think that it will help you accomplish?  What features do you think it will have?

Slide 2  What do you think this is? What do you expect to be able to do here?  What do you expect to happen when you click “Begin”?

Slide 3

The Corporate Executive Board

Workplan Generator



User Experience 2008

Is this what you expected? What is your impression of this page?

Slide 4  What can you do on this page?  Did using a questionnaire to determine your needs seem like the best approach? Would you rather dive straight into this page?  Would you want to add you own steps and attachments to this page?

Slide 5  What do you expect to be able to do here? What would you want to be able to do here?  How would you expect the two buttons on the bottom to behave? What would you expect to be contained in the email that you receive? Are there any other actions you would want to take at this point?  Is there anything else useful that could be shown on this page?  What would your next step be once you have emailed yourself the links? What would you do with the plan?  Delegating items? Is this the best approach? Completion dates?  Where would you look for this on our website?  What would happen if you selected a Step?  What would happen if you selected one of the items listed below a Step? Let’s say you selected Template: Steering Committee Meeting Invitation. Let’s go ahead and select that.

The Corporate Executive Board

Workplan Generator

User Experience 2008

Slide 6 Here’s a different idea, a more complex approach. Which would you find more useful for your day-to-day work, this version or the previous more simplified one?

Closing Questions  Have you ever used a tool like this before? Where/when/why? Did you find it to be useful?  Would you be interesting in using this specific tool? Why or why not? In this example, we looked at a Work Plan for “Reducing Tax Service Provider Expenses.” What other topics would you like to see covered by a tool like this?  Are you working on anything right now that could benefit from this type of tool?

The Corporate Executive Board

Tab Labels/Navigation

User Experience 2008

Tab Labels/Navigation June 2008

Project Summary Purpose To ensure that members would be able to navigate the site after creating the new “Topics” tab, we explored the labeling, UI, and number of navigation items to have along the top of the website.

Process Research We started by viewing current traffic statistics for the website and identifying the tabs that received the most or fewest visits. To identify the most common intentions for visiting the website, we conducted an email survey with members and also referenced past surveys that were conducted by individual programs. When considering terminology, we looked at other websites to identify trends in naming conventions. All of this information was used to create variations in navigation and terminology to be used for testing (below). User Testing Participants: 3 members, 2 internal CEB employees, and 2 external users (unfamiliar with CEB) For each version below, users were asked, “What do you expect to happen when you click each of these tabs?” Then we asked more specific questions about finding things on the site. “Where would you go to…” (benchmark your company’s DR strategy, find a list of services that ITLEx offers, edit your account settings)

Summary of Findings & Recommendations • • • • •

Keep the amount of tabs as low as possible. This is a priority to users when navigating Use “Professional Development” over “Training & Development.” Consider removing “Anatomy” to just have “Issue Diagnostic Center” If creating a Services tab, include wording on that page that clearly defines what the user is viewing. Under the new tab labeling system, users were able to successfully predict what would be found under each section and indicate where they would expect to find specific content. Test the usage of mouseovers to see if they have a large impact on usability by allowing users to dig one level deeper into each section.

Outcome New tab structure was launched in the August 2008 release. We are now able to have more confidence that members understand the labeling of the tabs and what can be found within each one. With new members joining regularly and visiting the website for the first time, this is especially important and ensures that our websites are selfservice products. The Corporate Executive Board

Tab Labels/Navigation

User Experience 2008

Through analytics, weâ&#x20AC;&#x2122;ve seen a decrease in searching and an increase in browsing the Topics pages, which indicates that the new navigation paradigm is easier to understand than the previous one. Previously, an equal amount of users searched and browsed. Now, browsing happens twice as often as searching.

The Corporate Executive Board

Tab Labels/Navigation

User Experience 2008

Usability Testing Tabs and Services Page

Tabs: Findings & Recommendations 1. Preference between 3 versions   

4 out of 7 liked the version with the least amount of tabs. o 2 members, 1 internal user, 1 external user 1 out of 7 liked the version that we have on our website right now o External user st 2 out of 7 liked a variation of the 1 version shown o 1 member, 1 internal user

Recommendation: Keep the amount of tabs as low as possible. Clearly this is a priority to users when navigating. 2. Terminology  

2 people mentioned a specific preference for “Professional Development” instead of “Training & Development” 2 people specifically didn’t like or expect to see the term “Anatomy”

Recommendation: Use “Professional Development.” Consider removing “Anatomy” to have just “Issue Diagnostic Center.” Check survey results to see if users recognized the branding of Anatomy. 3. Services Tab  

2 out of 7 people (1 member, 1 internal) were confused or thought Services meant something else 5 out of 7 had an accurate idea about the content that the Services tab would contain

Recommendation: If creating a Services tab, include wording on that page that clearly defines what the user is viewing. 4. Mouseovers or Sub-Tabs  

2 people mentioned that they would like to have mouseovers or sub-tabs in order to determine what was under a tab Those who disliked the shortened version of the tabs said that they did not have enough details about what was underneath them.

Recommendation: Re-test with mouseovers to see if they have a large impact on the usability, as these comments would suggest.

The Corporate Executive Board

Tab Labels/Navigation

User Experience 2008

Services Page: Findings & Recommendations 1. Content 

3 out of 7 expressed confusion over the items listed in the Services tab  Why are some items listed both in Services and in other tabs?  Do the services listed only include what’s on the website, or what’s part of the membership as a whole?

Recommendation: Again, if creating a Services tab, include wording on that page that clearly defines what the user is viewing. 2. Layout 5 out of 7 had a preference for the “list” layout  1 out of 7 had a preference for the “pinwheel” layout  1 out of 7 had no preference Recommendation: Use a design that is more similar to the “list” layout than the “pinwheel” layout. 3. Premium Services 7 out of 7 understood what Premium Services meant, but all were confused about whether or not they already had access to the Premium Service or would have to order it after clicking “Access Now” Recommendation: Terminology of Premium Services is fine, but experiment with different ways to indicate if the user has access or not. Test this again in Round 2.

The Corporate Executive Board

Tab Labels/Navigation

User Experience 2008

Detailed Findings Background Information User 1

Member. Has used the site.

User 2

Internal

User 3

Member. Has used “quite a bit” Use the site for research, like change management or PM process. I’ve posted to a peer group. I use webinars.

User 4

Non-member. External. Lobbyist.

User 5

Internal

User 6

Non-member. IT

User 7

Member. Has used the site.

Slide 1: Tab Version 1

What do you expect to happen when you click each of these tabs? We’ll go through one by one:

User 1

User 2

User 3

User 4

User 5

User 6

User 7

Member #1

Internal #1

Member #2

External #1

Internal #2

External #2

Member #3

Topics

Various IT topics such as change management

A list of topics and the content associated with each.

Summary level info, look for summary options to drill down. Unsure of content. Wouldn’t know until I saw the second tier.

Broad category of services

Information about what is leadership, being a leader in a company, how can CEB help

List of topics or categories to navigate through

Menu of different areas

Events & Webinars

Events online or onsite, a listing with categories and search capabilities for events in my area.

Calendar of events and webinars with ability to register

Selfexplanatory. A list of summary links. I usually get an email with all of them where I register

List of different events categorized and by date. Links to them.

Selfexplanatory

Webinars and other events that are offered. Categorized in some way. Maybe a calendar.

Current events coming up

Peer Groups

I’ve never used them but would expect a chat, bulletin board, place to provide recommendations.

Discussion board, peer networking of some kind

You can enroll into groups, your items of interest. It would be cool to merge with LinkedIn.

Good question... Way to interact with like-minded colleagues.

Information about peer groups. Improve leadership qualities, peers communicatin g

Some kind of chat or bulletin board

Different areas of interest, list of groups and you can drill down

The Corporate Executive Board

Tab Labels/Navigation

User Experience 2008

Benchmarking

Hmm. Haven’t used it. Best practices, survey results.

Enter data about my company, data returned. Tools to do this.

Metrics, reporting, benchmark applications, configuration information in order to support x. (ie Mercury and Loadrunner, ITIL, management, scorecards)

Track… production… Outline steps to meet goals.

Benchmarkin g tools, reporting tools, surveys, forms, questionnaire s.

Statistics or standards that can be used for benchmarking. Not sure exactly how it would be accessed or downloaded… It would be nice if this was interactive in some way.

Find the area I need a benchmark for, example: ITIL. Industry standards or by size of IT department.

Professional Development

No idea

Courses, CLE, white papers on topics such as management and IDP, possibly job openings.

Nice. You could tie in webinars with credits for PMPs self study courses, other events in our area, maybe a relationship with PMI.

References, resources, tutorials, online classes.

Improve on skills, professional or academic format. Start with questions and then take them to personalized content.

Training materials

Would want to spend time here. Hone skills by area. Example: PM.

Manage Your Membership

My profile, preferences, email notifications, adding people to my account

Change password, see contact information, when subscription expires, account rep, would like to see history of site usage/what I’ve consumed

Self-service. Change your password, filter system for your favorites

Membership with CEB. Cancel, edit, add member options.

Straightforward. If you want other memberships .

Editing your account, password, contact information, anything like that

Info about myself – profile. Unsure what else.

Anatomy: Issue Diagnostic Center

You have topics. This is another way to search for what I’m looking for, picking an issue – issue to solution.

“Anatomy” is strange and throws me off. If you remove that, it would be a place to come with a specific issue and see the thought process, step by step, with resources.

[Pause] Hmm. Support perspective. Help with problem solving.

Range of specific areas of interest, find more information about services

Selfexplanatory, like webmd. Search and come up with

Something having to do with common problems/issues. Unsure.

Not a term I expect to see. Issue resolution. If I contact you this is where I’d check in on the issue.

Services

I’m assuming consulting, professional advice, a one-onone engagement.

[Pause] Don’t know. Feel like everything is already covered. Stuff ITLEX does… other things I didn’t know? Onsite training? Mystery tab.

More for marketing – what does this site do? For new users. Might have flash.

Operational. What services CEB provides to its members.

Services offered by CEB

Other services or a summary of what the membership offers. Maybe ones that weren’t included elsewhere.

Services you offer my company or myself

The Corporate Executive Board

Tab Labels/Navigation

User Experience 2008

Slide 2: Tab Version 2

Now we’re looking at a slightly different version of the navigation. Where would you go to… User 1

User 2

User 3

User 4

User 5

User 6

User 7

Member #1

Internal #1

Member #2

External #1

Internal #2

External #2

Member #3

Benchmark your company’s DR strategy?

Topics

[Pause] I guess I would go to Topics and hope there’s one under the Topic I’m interested in.

[Pause] My Favorites? I don’t see anything here. Topics or Training and Development.

[Pause] Training and Development

Topics

Unsure. Training & Dev or Topics

Maybe under Topics.

Get a research document outlining other companies’ PM processes?

Topics

Training and Development

Topics

Training or Topics

A list of services that ITLEX offers?

Events & Webinars

My Membership

Services doesn’t lend itself to any of those.

Topics

My Membership

My Membership or Topics

A stretch. My Membership. I would think it’s not available.

A customizable template for creating an SLA?

Topics

Topics. I expect an icon to visualize different types.

Not detailed enough.

Training and Development

Topics

Your account settings?

My Membership

My Profile and Settings

My Membership

Events and Webinars would be offered there

Workshops, tutorials, interaction online, in categories by skill sets

What would you find in Training & Development?

The Corporate Executive Board

Tab Labels/Navigation

User Experience 2008

Slide 3: Tab Versions 1, 2, 3 Here are 3 different options for navigating our website. We just looked at the first two. The third is what currently exists on the website. Which of these 3 menus would be most helpful to you? Why? User 1 Member #1

User 2 Internal #1

User 3

Middle one probably. When I’m searching for things, My Membership makes sense… everything has to do with search. Everything else is under Topic. When you provide too many tabs, don’t know where to go, don’t want to check what’s under every tab. nd

2 doesn’t align with the questions we asked – but those questions are not what we’d want coming into the site. Prefer #1 with modifications. Anatomy is weird. Services… wouldn’t know. Should be under “Manage My Membership” For Peer Groups, prefers Network with Peers but maybe PG is more appropriate if it’s just Peer Groups there and nothing else. #3 is okay but it needs Topics – I think in Topics. Not necessary to split up Research and Tools but need clarity about what’s a tool and what’s research under the topic. Benchmarking does belong as a separate tab, more like Peer Groups, because your goal is different and you’re in the mood for something else, come to the site for it. Professional Development is standard and reads better than the others. “Benchmarking” is better wording than “Benchmark your company” because you might be benchmarking individuals.

Member #2

I don’t like the middle one. It’s too vague. The top one is cleaner. I would want mouseovers to understand what’s in the tabs.

User 4

I like the last one. It spells out more what each tab is doing. I understand the tabs a lot more.

External #1

User 5 Internal #2

User 6 External #2

User 7 Member #3

I like the middle one. I don’t like to see too many tabs. Sub-tabs are better. Six is okay, if it exceeds that, it’s clutter. I like the middle one because I can read through the options the most quickly. But it also seems as if it st nd might be missing something. But on the other hand, the 1 and 2 I would be more likely to quickly scan through and possibly miss one of the options. I like the middle one because there are fewer options. When I looked at it, I thought “Oh wow.” I would be able to find anything I need in these areas. Anatomy naming doesn’t work. Professional Development is better than Training and Development.

The Corporate Executive Board

Tab Labels/Navigation

User Experience 2008

Slide 4: Services Tab v1

First impression: What can you do on this page? What catches your eye? User 1

Not what I was expecting. These are your products. This is what I like. Services is misleading – “Research Finder” or “Fast Finder” at a glance to see what’s available.

User 2

Don’t know where to look first. It’s hard to read. Wouldn’t think anything is clickable on this page.

User 3

Easy to read. Broken up, easy on the eye. I like it.

User 4

No priority. Everything is equal.

User 5

First impression – pretty good. Unique. I’d want to see a rollover with more information.

User 7

I didn’t realize that you offered so many services. Having it all in one area is nice. If it’s all in one place, I’m more aware of what’s available.

Is this what you expected? Why or why not? User 1

No. [Was expecting one-on-one consulting services as mentioned previously]

User 2

No, was expecting a list.

User 3

Yes.

User 4

No, I was expecting more of a list. I expect to be able to scroll over and see examples of each.

User 5

I didn’t expect benchmarking. I expected Tools & Templates, Webinars.

User 6

The content is more or less what I was expecting, but maybe not in the way it was arranged.

What does Premium Service mean? User 1

Costs extra per research document

User 2

Didn’t pay for this – can’t click

User 3

[v2] Not free. Need to subscribe to log in.

User 4

[v2] Would have to pay more to access. If I clicked Access Now I would have to purchase and use a credit card.

User 5

Not included. Would have to pay more. After clicking “Access Now” I’d have to fill out information to request access.

User 6

An addition you purchase and add to your account. Like an upgrade.

The Corporate Executive Board

Tab Labels/Navigation

User 7

User Experience 2008 Not available to general membership. I would try to click on “Access Now” but wouldn’t be surprised if I had to update my membership

Is there anything on this page that is unclear? Do you know what you would find in each of the ‘boxes’? User 2

Are these all online services or are they something else not on the website? Are these the whole of the membership? Confused.

User 3

Case Studies as a service. I would think it would be under Research.

User 4

Expects to click on the blue links but unsure.

User 5

Case Studies should be under Topics. Moderated Peer Groups should be under Peer Groups, not Services.

User 6

It seems pretty clear.

User 7

No, because the descriptions help.

The Corporate Executive Board

Tab Labels/Navigation

User Experience 2008

Slide 5: Services Tab v2

Be Here is another version of the same page. What stands out to you? User 1

Bundled together. That works. Just renamed.. Webinars, is that events? [Confused about events vs. staff development and support groupings]

User 2

This is much easier for me. Grouped under headings.

User 3

It’s grouped. The groups stand out. This addresses my last point. I like the Peer-To-Peer.

User 4

Much more organized. I got lost in the last one. This is easier to follow and read.

User 5

The previous version was better. This is too much information. The font size is too small. [Font size is the same]

User 6

This is more of what I was expecting. It seems organized in a logical way that I can skim through.

User 7

This is more of a best practices, a standard. I’ve seen this on sites. The other one is a “target” view, which adds value, but this is more standard.

What do you expect to see on the screen when you click Case Studies? User 1

White papers, best practices Might be a personal thing but I don’t like pop-ups. This doesn’t follow the standard. Would be better to have collapsible groups that drill down and are standard. I don’t have time to drill down. First need is search.

User 2

Page that lists all case studies organized by topics

User 3

Break out by subject

User 4

I expect a page with short links to other short summaries of 10-15 case studies that I can click on.

User 5

I expect PDF documents and a search option. Search by organization name, year, issue, type.

User 6

Downloads that are organized in some way. If there are more than a dozen or so, it would be nice if I could search through them.

User 7

List of topics, I hope.

Let’s go ahead and click it [go to slide 6] What can you do here? The Corporate Executive Board

Tab Labels/Navigation

User Experience 2008

User 1

This is good – I can search.

User 2

I like it. It wasn’t what I thought, but easier than what I thought. A surprise but a nice surprise. Happy to see by topic. Prefer this screen before going to a new page.

User 3

I like the ability to search instead of choose a topic. Some are under many topics.

User 4

This makes sense. Searchable is good.

User 5

That’s one of them [Topics]. I would also want to see it by organization or by year.

User 6

This is good. I like having the option to search if I’m researching something specific.

User 7

I expected this but didn’t expect it to be a pop-up. No preference of pop-up or new page.

Switching between slides 4 and 5 Between these two, which do you find more helpful? Why? nd

User 1

I like the 2 one. First one is good if you’re doing a presentation in Powerpoint. 2 is cleaner because it’s grouped together, easier on the eye.

User 2

Definitely 2. Layout is clean, 4 buckets.

User 3

The 2

User 4

User 5

1 one. But for members who are older, 40-45+, they might not like it. I think it’s visually more appealing.

User 6

The second. It’s easier to read.

User 7

I don’t have a strong preference. But Premium Services didn’t stand out on the first one and I noticed it on the second. Could just be the placement of the item.

one. It’s crisper. I was surprised because I liked the 1 one.

General Feedback User 1 Member #1

User 3

Performance is slow when searching for information Content is old “2 years is a lifetime” – but overall service is good You can group everything in 3 ways: o Topics o Industries (ex: Healthcare vs. high tech vs. govt) o Markets (ex: Hardware, Software, etc.) Has seen websites that offer all 3 ways of getting to same data set to meet the needs of everyone. Good example is Gartner.

I used a dashboard once that was too rigid and I wanted to create a light version.

Member #2

User 7 Member #3

A lot of information on your site but it’s hard to navigate. I spend a long time searching and the struggle limits my use of the site. I know there’s good information but search results are convoluted and not precise. They aren’t narrowed down enough. Needs to be more targeted. Too much comes back and I’m overwhelmed – not sure if I’m using the wrong key words. I have to go into the specific documents.

The Corporate Executive Board

Topic Center Templates

User Experience 2008

Topic Center Templates October 2008

Project Summary Purpose With many different layouts being used for Topic Centers throughout Middle Market websites, a standardized template has emerged. We tested variations of this template for general usability and to get a better sense of user priorities.

Process Participants: 5 internal participants (CEB employees in HR and IT) Each participant chose an HRLC Topic Center to explore: Workforce Planning or Budget & Spend. Participants were given tasks to complete using 3 different versions of that Topic Center (pictured below). After completing the tasks, they were asked to identify preferences – which layout made it easiest to find information? Version 1: Vertical Navigation Tabs along the left side of the page

Version 2: Long Page All content on one page with no tabs

Version 3: Horizontal Navigation Tabs along the top of the page

Summary of Findings & Recommendations • • • • • •

Users stressed the importance of being able to see the “big picture” in one glance, above the fold. Image preference varied by person. One person felt the page looked more professional without images but 23 people liked having images. They didn’t find that the images necessarily added value or helped them find information, but liked the way that they broke up the layout/information into clear sections. Preference for simple but populated first page that gives many links and indications of what can be found in each section. Page layout should be well balanced (not a lot of white space on one side). Those who didn’t prefer the “long page” version did like the fact that everything could be accessed on one page, so in some cases it might be worthwhile to include a version like this. Resources were often not noticed or noticed last because of the placement on the page (on the right). Some confusion over the resources being repeated on both sides of the page. Duplicate the Toolbox at the bottom of the page in addition to the top (2 people expected to find it there)

Outcome New standardized template has been used for many topic centers and will continue to be used in the future. As a result, our pages are getting a much more consistent look and feel throughout the sites. These types of pages can also be built more quickly than before because new designs don’t have to be created each time. Note that in the end, the success of Topic Centers depends on the content, not the design. Poor content will produce an unsuccessful Topic Center no matter how usable the layout. The Corporate Executive Board

Topic Center Templates

User Experience 2008

Topic Center Usability Testing th

October 29 , 2008

Methodology We tested with 5 internal participants â&#x20AC;&#x201C; 3 in HR and 2 in IT. (Note: Normally an iterative process is preferred, but time constraints allowed for only one round of testing.) Each participant chose an HRLC Topic Center to explore: Workforce Planning or Budget & Spend. Participants were given tasks to complete using 3 different versions of that Topic Center (pictured below). After completing the tasks, they were asked to identify preferences â&#x20AC;&#x201C; which layout made it easiest to find information? Version 1: Vertical Navigation Tabs along the left side of the page

Version 2: Long Page All content on one page with no tabs

Version 3: Horizontal Navigation Tabs along the top of the page

The Corporate Executive Board

Topic Center Templates

User Experience 2008

Findings & Recommendations Overall Preference Vertical navigation (preferred by 3) - 1 user felt that vertical navigation represented a process better than horizontal navigation, which would represent “sections” - 2 users preferred this layout because it showed more of the content at the top of the page. They didn’t have to scroll to see everything, and their eyes were drawn to the content first, followed by the tabs. Horizontal navigation (preferred by 1) - Navigation and listing of steps horizontally corresponded with each other - Found it to be visually pleasing Long page (preferred by 1) - Liked seeing everything on one page, but didn’t want to be overwhelmed Navigation Recommendations: - Although vertical navigation was preferred in our testing, many new Topic Centers are being created with 2 st nd layers of navigation (1 level on the left vertically, and 2 level on the top horizontally). In these cases, we nd don’t recommend having 2 sets of vertical navigation, so horizontal navigation should be used for the 2 level.  With this in mind, to be consistent, continue using the horizontal navigation tabs on all Topic Centers whenever possible. - When using horizontal navigation, the tabs should be made wider and shorter to maximize the content area of the page. Most users preferred the vertical navigation because it allowed them to see more of the content on the page, so this would solve that problem. - Previous and Next buttons should be made more prominent. During the testing, no one used them or seemed to notice them. General Findings -

Universally, the “What do you want to do? Step 1, Step 2, Step 3” section was identified as the most important part of the first page. Users stressed the importance of being able to see the “big picture” in one glance.

Image preference varied by person. One person felt the page looked more professional without images but 2-3 people liked having images. They didn’t find that the images necessarily added value or helped them find information, but liked the way that they broke up the layout/information into clear sections. o

Recommendation: Make sure a section showing all parts of the topic center is “above the fold” if possible and given appropriate weight on the page. Bulleted lists, such as those in the above screenshot, should be links as well.

Recommendation: Continue to use images as they were used in these Topic Centers.

Preference for simple but populated first page that gives many links and indications of what can be found in each section. Page layout should be well balanced (not a lot of white space on one side).

The Corporate Executive Board

100

Topic Center Templates

User Experience 2008

Those who didn’t prefer the “long page” version did like the fact that everything could be accessed on one page, so in some cases it might be worthwhile to include a version like this.

Resources were often not noticed or noticed last because of the placement on the page (on the right). Some confusion over the resources being repeated on both sides of the page. However, we were using location versions of the files that didn’t have the mouseover highlighting functionality to show how the links were related.

In the Budget & Spend Resource Center, users always noticed the prominent button titled “Take a Short Survey” but had a negative reaction to it. They felt that it didn’t correspond with their reason for coming to the Topic Center. o

Recommendation: If including links or features where we’re asking members to do something, tell them what’s it in for them (if they take the survey, do they get to see responses?).

Recommendation: When appropriate, use buttons to attract the user’s attention since they are easy to notice on the page. However, don’t replace standard links with buttons. Buttons should only be to indicate actions: “Use buttons to do things, use links to jump to other web pages.”

Toolbox - Typically, the toolbox was located where they expected it to be - Print - everyone understood - Mail - everyone understood - Favorites -- most assumed this meant browser bookmarks - Save It -- caused confusion. Saving to hard drive? One page or all tools? Saving to website? Recommendation: For any non-standard actions, provide alt tag mouseover explaining what it does. - 2 people expected to find the toolbox at the bottom of the page Recommendation: Duplicate the toolbox at the bottom of the page in addition to the top

The Corporate Executive Board

101

Topic Center Templates

User Experience 2008

Scripts Used for Testing Workforce Planning Center First Impressions What is your first impression of this page? What catches your eye? Where are you? What can you do here? What would you expect to find? Is anything confusing or unclear? Scenario 1 You want to identify talent gaps within your company’s workforce. Find something that would help you with this challenge. Scenario 2 At the end of this week, you’re giving a presentation to colleagues that explains why workforce planning is important and should be pursued as a high priority. Find something to help you with this task. Scenario 3 You’ve identified talent gaps within your company’s workforce. How can you fill those gaps? Find something here that would help you. Scenario 4 You’ve found this page to be really helpful and want to send it to a colleague. How would you do that? Scenario 5 What can you do in the Toolbox? What do you expect to happen when you select each of these options? Conclusion Having used these 3 versions of the same Topic Center, which was the most intuitive when you were looking for information? Why?

HR Budget and Spend Resource Center First Impressions What is your first impression of this page? What catches your eye? Where are you? What can you do here? What would you expect to find? Is anything confusing or unclear? Scenario 1 Your HR department does not have a formal budget. Your task is to plan how one can be established. What would you do first? How would you find a template to work from? Scenario 2 Your HR department has a formal budget. You want to find benchmarking data to compare your budget allocation decisions. Where are the spending patterns of your peers? Scenario 3 Find a template to write a business case for implementing or upgrading HR information systems. Find a template to write a business case for allowing employees to telecommute (or “telework”). Scenario 4 You’ve found this page to be really helpful and want to send it to a colleague. How would you do that? Scenario 5 What can you do in the Toolbox? What do you expect to happen when you select each of these options? Conclusion Having used these 3 versions of the same Topic Center, which was the most intuitive when you were looking for information? Why? The Corporate Executive Board

102

New User Login Process

User Experience 2008

New User Login Process November 2008

Project Summary Purpose 30-40% of members have never completed registration or logged into the site for the first time. Not all who visit the registration form will actually complete it. Our goal is to revise the process a new member goes through, including all emails and registration forms.

Process Research & Competitive Analysis To begin the project, we looked at usage of our current sign-up process. Through web analytics, we identified drop-off points and baseline statistics. We then completed and documented the sign-up process of a number of sites, including LinkedIn, Facebook, The McKinsey Quarterly, and Mint.com. We also researched best practices for form design, emails, and sign-up processes: • Designing for Sign-Up Virtual Seminar with Joshua Porter • Usability of Transactional Email and Confirmation Messages Nielsen Norman Group Report • Web Form Design: Filling in the Blanks Book by Luke Wroblewski • Designing the Social Web Book by Joshua Porter Usability Testing Based on our research and competitive analysis, we designed a new sign-up process and facilitated usability testing with 6 participants: 3 members, 2 internal CEB employees, and 1 external user. Seniormosts were given the scenario: “You have just purchased a membership to the Finance Leadership Exchange.” Non-Seniormosts were given the scenario: “You are not yet a member of the HR Leadership Council and have never heard of it.”

One of the emails that was tested

The Corporate Executive Board

They were then led through the following processes: 1. Receive Invitation Email 2. Register on website 3. Receive Welcome Email 4. Join a Peer Group 5. Add a Team Member Emails were sent to the users’ inboxes and webpages were prototypes viewed through GoToMeeting (or, for CEB employees, locally on the X drive).

New registration form that was tested

103

New User Login Process

User Experience 2008

Summary of Findings & Recommendations • • • • • • • • •

Use of Seniormost name in subject line of emails is extremely effective. Registration form must be simple and clean to meet users’ expectations. Indicate why we’re collecting information (topics, areas of experience) and what we’ll use it for – this will increase the likelihood of the member providing that information. To be engaging, content must be specific and current, rather than broad. All found the revised registration process to be either easier or equivalent to the registration processes of other websites. Positive reaction to auto-registering for Peer Groups/Webinars, but give clear “opt-out” path and/or indicators of next steps. Many not aware that (or how) you can register team members. Specific numbers and names are engaging: Number of users in a Peer Group, names of colleagues who have already registered. Always give a clear path to change email preferences

Outcome The redesigned process was launched in the January 2009 release. After 1-2 months have passed, we will be able to do ‘before and after’ comparisons to measure the impact. We’re hopeful that the following results will be achieved: • % of registered users who never visit the website will decrease • % of users who exit during the registration process will decrease • % of users who provide personal topics of interest will increase • Number of people who “invite a colleague” will increase

The Corporate Executive Board

104

New User Login Process

User Experience 2008

Usability Testing Findings November 25, 2008

Background & Methodology There were 6 participants: -

3 used FLEx prototypes/content and 3 used HRLC prototypes/content. 2 were seniormosts, 4 were team members 3 were active members, 2 were internal CEB employees, and 1 was an external user with little familiarity with CEB

Throughout this document, participants are referenced by number: [1] Internal CEB employee. Level: Seniormost. Program: FLEx [2] Active member of HRLC. Level: Team Member. [3] Active member of HRLC. Level: Team Member. [4] External user. Level: Seniormost. Program: FLEx. [5] Internal CEB employee. Level: Team Member. Program: FLEx. [6] Active member of HRLC. Level: Team Member.

Seniormosts were given the scenario: “You have just purchased a membership to the Finance Leadership Exchange.” Non-Seniormosts were given the scenario: “You are not yet a member of the HR Leadership Council and have never heard of it.” They were then led through the following processes: 6. 7. 8. 9. 10.

Receive Invitation Email Register on website Receive Welcome Email Join a Peer Group Add a Team Member

Emails were sent to the users’ inboxes and webpages were prototypes viewed through GoToMeeting (or, for CEB employees, locally on the X drive). For emails and prototypes, see Appendix A. For scripts, see Appendix B.

Remembering Their Onboarding Process Members were asked to reflect on their first experiences with the website. They typically did not attend or did not remember their welcome calls.

What do you remember about the first time you logged in to the site? -

I remember thinking “I wonder how often I’ll actually come here” which means it must have seemed cumbersome. [2] I felt I just visited the site. I remember signing on and registering for a webinar to walk me through the site. I didn’t end up attending it though. [3] Easy-can’t visualize but didn’t have any problems [6]

What else would have been helpful at the beginning of your membership? -

Sending an email a few days after the onboarding call to ask if there are any questions. You might already do this.[2] I didn’t get a sense of what all was there-that’s somewhat on me-I should explore. Would be nice to send a specific email with what’s all there or have it front and center. [3] User friendly-packed full of great info [6]

The Corporate Executive Board

105

New User Login Process

User Experience 2008

Invitation Email At the beginning of each session, a customized Invitation Email was sent to each user’s inbox:

Finding / Suggestion

Supporting Quotes

Use of Seniormost name in subject line is extremely effective.

» Comes from boss, cool-ok, what is this? [6] » A note from [Seniormost] gets my attention. Invites you and HR Leadership are attractive. Leadership-positive [2] » Subject-pretty good-definitely makes me pay attention [5]

Content in the body of the email (4 paragraphs) is more important than the right sidebar, but right sidebar does not detract from the email.

» I like that you have 4 areas there. Having 4 is good because a few don’t interest me. But if you had more, it would be too much. [2] » I ignored what was on the right, tells me what I thought it would [4] » Networking opportunity-I work for a small company (worked at big company last 17 years) and need that [6] » Tools and templates-grabs attention-CPE-grabs my attention [6]

Despite having the Seniormost’s name in the subject line, many will still file this away for later. If they don’t complete the registration process, we should send a 2nd reminder email 1 week later.

» If I was busy, I would close it and mark it as unopened – that’s to remind me to go back because it looks interesting. [2] » I would go to link but depends on my day-might at least open it because it comes from [seniormost]. See if it’s something I need to do today [6] » If it’s a busy day, I’d skip over and go back. [3] » Skip to the next email in my inbox and go back later. If it arrives in the morning or at lunch, I might actually go through it.[1]

For companies with many team members (25+), some may not recognize Seniormost’s name. Include the company name closer to the top of the email.

» I know [Seniormost], but some might not know who she is. [3]

The Corporate Executive Board

106

New User Login Process

User Experience 2008

Registration Form After following the link in the Invitation Email, users were taken to the Registration Form:

Non-Seniormost Registration Form. See Appendix A for Seniormost version.

Finding / Suggestion Format must be simple and clean, to meet users expectations. -

Blue background was confusing, so we removed it Keep Submit button above the fold / make sure that topics dip below the fold (no false bottom)

Supporting Quotes » What do I do next [after entering email, title, and password]? Hit enter? [2] » Looks familiar, I think I’ve done this…title, password. I would scroll down to see if there is anything else [3] » Looks like a regular form-basic [5]

Content in the right column should be specific and current in order to be engaging.

» It’s clear enough to not turn me off, but nothing gets me especially excited. I would like to see excerpts from 3-4 things I would find. Topics. 1-2 sentence summaries. Pictures of webinars. More specific content. [2] » Clean, easy to understand, but not thrilled about the right column. The bullets look hard, too wordy. The idea is okay… should be geared towards current projects or headlines. Financial crisis. See what peers are talking about. [1]

Users may not realize they are limited to 3 topics. Give live feedback messaging and allow prioritization of 3 topics + choosing others. – We need to explore this further.

» [selects 6 topics] [5] » I don’t want to limit to 3 topics. Now I have to read the entire list and go back before choosing them. [2]

If we have Title (seniormosts), we should prepopulate it here.

» Is my title not stored?[1]

Indicate what we’re using the Topics for

» I’m not going to select any topics because I’ll get email. [1]

#/list of colleagues is more interesting than

» 27 of my your colleagues is interesting [3]

The Corporate Executive Board

107

New User Login Process

User Experience 2008

“What You’ll Find” – move to top. Programs with more overlap in Topics (ex: FLEx) should re-visit these topics because users will want to select more than 3.

» Yes, I want to tell you about me as I assume you push information. The topics close to me overlap too much accounting/accounting close (accounting close is part of accounting). Internal controlsmaybe call it SOX-seems to close to internal audit. Management Accounting part of FP&A [4]

Users may accidentally select the wrong answer for Direct Report question. On the admin side, we should specify where this information came from (was it entered by member services, the user, or some other way?)

» Direct report? Yes..wait, no, it’s my boss’s boss. No. [2]

Direct report question is effective.

Not one expressed confusion over this question or asked why we needed the information.

Tour link After submitting the registration form, users were taken to the homepage with an overlay:

Finding / Suggestion

Supporting Quotes

Users are more inclined to jump straight to the site because it’s faster. o Pilot this with 2 programs to see how much usage it’s getting. o Make link available elsewhere in the site so that they can return to it later (though unlikely)

» Depends on my mood. If I’m busy, I would just go to the site. If I’m focused on this and think it will be of use, I would click “learn more” [3] » [clicks no thanks] No thanks, continue to site-I don’t read directions [4] » Going to look at the site-maybe features later [5]

Instead of one “Learn about our features” link, experiment with multiple, more specific links: “Network with Peers” “Attend a Webinar” etc. – this will make it seem less timeintensive and more specific to the user.

» Skip learn about features-always in a hurry, click no thanks [6] » I would go directly to the site. If I clicked on the link, I would expect it to go to 1 of 2 things: a tutorial, or a subject-based guide to the website [2]

The Corporate Executive Board

108

New User Login Process

User Experience 2008

Registration Process as a Whole Compared to the registration process on other websites, was this easier, harder, or about the same? » » » » »

Definitely easier. Only had to enter my password once and didn’t have to identify a word in an image. [1] Easier. Only a few things. Selection 3 topics was easy [3] About the same – most of the registration I do is for webinars online [4] I liked the topics checkboxes - Simple [5] Easier- Didn’t have to give a ton of information, clear [6]

Welcome Email Users were asked to check their inbox for the Welcome Email after they completed the “Create My Account” form and viewed the website’s homepage for the first time:

Finding / Suggestion

Supporting Quotes

Subject should not be “Welcome to…” Instead use “Getting started with [Program Name]”

» [From the subject line] I don’t like these kinds of emails. I don’t need a computer to welcome me. Do I need this? Why do I need this? I can’t tell if it’s important. But realistically, I’ve been trained from past experience that these Welcome emails do contain important information. [2]

If auto-registering for events and Peer Groups: o Give one-click ability to unregister o Give indicator of next step (Add to outlook calendar?) o In admin view, we should be able to

» This is cool that you registered for me but I hope you told my assistant or it’s not happening. [1] » Say registered you for-it’s nice-not obligated to go, can change [5] » Well, I’m busy that day. You’re making work for me because I have to go unregister. I don’t know at this point what a Peer

The Corporate Executive Board

109

New User Login Process

see who has been auto-registered for webinars/events because they have a much smaller chance of actually showing up Show more details about what Peer Groups are

User Experience 2008

Group is. “Huh, what does that mean?” [2] » Not sure I’d want to be registered automatically, not sure what peer groups are. Strikes me funny-wouldn’t do anything. [6] » Interesting..I’ve already been entered. I like it. Nice touch. How do I get it on my calendar? I assume you’ll send out a reminder and I would go and put it on my calendar but others might forget and miss out [3] »

Links to Orientation webinars are good, but only include for non-Seniormosts. For seniormosts, these emails should be cc’ed to assistant (if applicable).

» » » »

Some will not read this email. For autoregistration (especially to Peer Groups), it’s crucial to give opt-out/change preference links every time the user receives another email regarding it.

» Oh, you’ve already been registerest for-I didn’t catch that. Yes, good topic. If I was open at that time, interested. I wouldn’t feel obligated to unregister [4]

Link to visit the website should be listed last, not first, because the user was just there.

» Would skip straight to “New to the HR Leadership Council?” instead of the link to access the website. [2]

Add information about how you can register team members

» See Add a Team Member

Keep recommended content in right column – readers find value in it and see how their selections are already being used.

» Good, good-based on your topics [5] » The right column is good. I would have expected it in the previous email. 1] » Customize based on what I picked-those look good [4] » I like the right site-might click on later [6]

The Corporate Executive Board

I like that these are live and not pre-recorded so I can ask ?s. [2] Orientation-probably would attend [5] Might do an orientation online [6] Attend orientation-again, if I was focused, I would register [3]

110

New User Login Process

User Experience 2008

Peer Group Registration Form From the homepage, users were asked to register for a Peer Group, which displayed the following form:

Finding / Suggestion

Supporting Quotes

Re-word the request for resume (“Paste your resume below…”)

» Why would they want my resume? [6] » It’s weird that you ask for a resume and the chances of someone submitting one are very low. You assume that my resume is current. [1]

To encourage more input of information, indicate what we’re using it for (especially Areas of Experience) by showing preview of Find Peers functionality.

» [summary of experience] no-skip over. What’s the purpose? [3] » If I knew what it was used for, I would dput more information [6] » Wouldn’t choose interested because I wouldn’t want to be bogged down by things [6]

Add a link to show definitions of skilled, expert, and interested.

» Interested-skilled, do it, expert-my expertise? [5] » Skilled-what am I saying? [3]

The Corporate Executive Board

111

New User Login Process

User Experience 2008

Peer Group Registration Email After completing the Peer Group Registration Form, users were directed back to their inboxes, where they had received the following email:

Finding / Suggestion

Supporting Quotes

Give clear path to change preferences, but also make it clear what the different options are (Not just opt out, but switch to weekly, etc.)

» Would chose weekly over none [5] » First thing I notice is a lot of emails-not happy-daily, annoyed. Would want to see list of all options for emails [5] » Update email preferences.. didn’t see email notifications. I would instead put to opt out of any (instead of update preferences) [4]

# of peers is important to include – it has a positive impact.

» 1,500 peers-like that [3] » 1,500 peers-could go know [5] » The first thing I notice is how many people --large-- shows how valuable this is [4]

Include links to main actions that can be taken – ask a question, search for answers, etc.

» Where do I ask a question? I’d like to see more about what you can do here [6]

Make sure to include ‘Change Email Preferences’ link in every single peer group email

» Emails-preference-I get it. What if someone doesn’t read this? [3]

The Corporate Executive Board

112

New User Login Process

User Experience 2008

Add a Team Member From the homepage, users were asked to find a way to get a login to the website for a colleague.

Finding / Suggestion

Supporting Quotes

If user accidentally re-enters domain, don’t give an error. Automatically cut off at “@”

» Easy… Oops, it already had a domain. [6]

Both the ability to add and the link on homepage are not obvious enough: - Link should be more specific or re-worded - Add the link elsewhere; on the bottoms of emails and in the welcome email: “Want to share this with your colleague? Register them – it’s quick and easy.” - Wording on page should re-enforce that it’s free

» Not sure if “Add a team member” is clear enough. Add colleague… share the knowledge with your team. Wouldn’t have thought I had the option, but the placement on the page is good. [5] » Didn’t know I could add team members. How would I do that?... Under my account? [6] » Suggestion-to add team members in the welcome email-“invite a colleague”[6]

General Website Feedback Finding / Suggestion

Supporting Quotes

Revisit the quality of search results

» Search function is not useful. Even when I know the exact title, it shows up second in the list. On topics with a lot of responses- go through a lot of unrelated items.

Industry-specific Peer Groups

» I would expect industries to choose from. Anyone in my position would have oversight in all of these topic areas. [4]

Content segmentation by industry

Purge old content or update with “last Reviewed” date

» Tried finding something on diversity but everything was outdated. I was out of the country for a few years and trying to get current on topics but they were all old. » In peer groups, if there are 0 replies, Jovita responds which is good but usually it’s with outdated material (from 2005). I know it’s hard to get people to respond.

The Corporate Executive Board

113

New User Login Process

User Experience 2008

Appendix A: Prototypes Used Invitation Email: Seniormost

The Corporate Executive Board

114

New User Login Process

User Experience 2008

Invitation Email: Non-Seniormost

The Corporate Executive Board

115

New User Login Process

User Experience 2008

Registration Form: Seniormost

The Corporate Executive Board

116

New User Login Process

User Experience 2008

Registration Form: Non-Seniormost

Tour Pop-Up

The Corporate Executive Board

117

New User Login Process

User Experience 2008

Welcome Email: Seniormost

The Corporate Executive Board

118

New User Login Process

User Experience 2008

Welcome Email: Non-Seniormost

The Corporate Executive Board

119

New User Login Process

User Experience 2008

Peer Group Registration Form

The Corporate Executive Board

120

New User Login Process

User Experience 2008

Peer Group Confirmation Email

The Corporate Executive Board

121

New User Login Process

User Experience 2008

Appendix B: Scripts Seniormost Script (Non-Members) Introduction The emails and webpages that we’ll be visiting are prototypes, so some links may not work, or some of the content might not quite match up. As we view the emails and webpages, I’m going to ask you to think aloud and share anything that comes to mind. We want to know if anything seems unusual or confusing throughout the process. Here’s the Scenario: You have just purchased a membership to the Finance Leadership Exchange for your company. 1. Invitation Email Open your inbox and you’ll find an email from Beth. a. What do you think this email is? Now, let’s open the email. When you see it, think out loud. b. What are the first things you notice? c. What would be your next step after getting this email? d. What do you find helpful? Is any information not needed? e. How would you send Kara a list of team members? f. What do you expect to happen when you click the link? 2.

Task: Access the FLEX website Beginning with this email, you want to access the FLEX website. As you go through this process, think aloud. Continue until you’ve arrived on the home page. Registration Form a. What are your first impressions? What are you noticing/reading first? b. Is anything unclear? c. What do you think of these topics? Tour/”Learn About Our Features” pop-up a. Where would you expect this link to take you? b. Would you select it or continue to the site? Do you usually view these on other websites? c. Compared to other websites, would you consider this registration process easier, harder, or average?

3. Welcome Email Let’s return to your inbox to look at a second email. When you see it, think out loud. a. What are the first things you notice? b. What parts of it did you read? c. What would be your next step after getting this email? What do you usually do with emails like this? d. Now, go back and read all of it. What do you find helpful? Is any information not needed? 4.

Task: Join a Peer Group We have a Peer Network on the website which allows you to communicate with peers by asking and answering questions. Using the My Peer Groups box, join a peer group. If user does not fill out the form, prompt them: • What information, if any, would you fill out on this page? • What would you put in Summary of Experience? • How would you expect the information in this form to be used?

Peer Group Registration Email Let’s return to your inbox to see a third email. a. What are the first things you notice? b. What parts of it did you read? c. What would be your next step after getting this email? What do you usually do with emails like this? d. Now, go back and read all of it. What do you find helpful?

Task: Add a team member (IF TIME) a. We’re on the homepage. Let’s say that someone in your group does not have a login to the site, and you’d like to give them access. How would you do that?

The Corporate Executive Board

122

New User Login Process b. 7.

User Experience 2008

What do you expect to happen when you follow the link?

Task: Register for a Meeting (IF TIME) Find and register for a meeting, also known as a CFO Forum. Please fill out all fields in the form.

Non-Seniormost Script (Members) Introduction We’re making improvements to the process that you would go through when you first become a member of the HR Leadership Council. As you may remember, we collect a lot of information from you the first time you log in, and we want to make that process faster and easier. Before we dive in, I have a few background questions for you: When your company first joined the HR Leadership Council, did you attend a welcome call? o What do you remember about it? Do you remember the first time you logged in to the site? What was your experience? When your company first became a member, is there anything else that could have been done during the first month to help you better use our services? Now I’m going to show you some emails and webpages. These are prototypes, so some links may not work, or some of the content might not quite match up. As we view the emails and webpages, I’m going to ask you to think aloud and share anything that comes to mind. We want to know if anything seems unusual or confusing throughout the process. Here’s the Scenario: We’re going back in time to when your company first joined the HR Leadership Council. You haven’t heard about it yet and have just received an email in your inbox. 8.

Invitation Email

If you visit your inbox, you should see an email from Elizabeth Swanson. a. From the subject line, what do you think this email is? Now, let’s open the email. When you see it, think out loud. b. What are the first things you notice? c. What would be your next step after getting this email? d. What do you find helpful? Is any information not needed? e. What do you expect to happen when you click the link? 9.

Task: Access the HRLC website Beginning with this email, you want to access the HRLC website. As you go through this process, think aloud. Continue until you’ve arrived on the home page. Registration Form a. What are your first impressions? What are you noticing/reading first? b. Is anything unclear? c. What do you think of these topics? Tour/”Learn About Our Features” pop-up d. Where would you expect this link to take you? e. Would you select it or continue to the site? Do you usually view these on other websites? Looking back at your expectations from the link in the email (see 1H), how did this process compare? Compared to other websites, would you consider this registration process easier, harder, or average?

10. Welcome Email Now I’d like you to return to your inbox to look for a second email. When you see it, think out loud. a. b. c.

What are the first things you notice? What parts of it did you read? What would be your next step after getting this email? What do you usually do with emails like this?

The Corporate Executive Board

123

New User Login Process d.

User Experience 2008

Now, go back and read all of it. What do you find helpful?

11. Task: Join a Peer Group I know you’re already familiar with Peer Groups, but for this session let’s pretend that you’ve just logged in to the website for the first time. Using the My Peer Groups box, join a Peer Group. If user does not fill out the form, prompt them: • What information, if any, would you fill out on this page? • What would you put in Summary of Experience? • How would you expect the information in this form to be used? 12. Peer Group Registration Email Let’s return to your inbox to see a third email. a. b. c. d.

What are the first things you notice? What parts of it did you read? What would be your next step after getting this email? What do you usually do with emails like this? Now, go back and read all of it. What do you find helpful?

13. Task: Add a team member (IF TIME) a. We’re on the homepage. Let’s say that someone in your group does not have a login to the site, and you’d like to give them access. How would you do that? b. What do you expect to happen when you follow the link?

The Corporate Executive Board

124

Next StepsNew User Login Process

User Experience 2008

Next Steps User Research Usability testing has been a great first step, but nearly all enhancements on our websites still originate internally. They are ideas typically proposed by research teams, sales teams, member services, program leads – not members. Our members must have unmet needs that we don’t know about. These might be subtle but significant challenges that they face in their day-to-day jobs that wouldn’t even come to their minds when we ask them. By doing user research, we can start to uncover these unmet needs and truly have member-driven innovation. This type of research would involve visiting members (or prospective members) in their own work environment to do interviews and field studies. By observing the environment in which they use our websites and learning about their day-to-day challenges, we will be able to gather more information about their needs than can ever be captured through surveys or focus groups.

Measurement As we improve our design process, what impact are we having? At the beginning of 2008, we were lacking in accurate ways to measure success. By implementing Google Analytics for the Middle Market websites and giving business users access to web usage data (pulled from our databases through Tableau), we have started to gain insight into how members are using our websites. We can now measure page views, time spent on pages, paths to achieving goals (whether a goal is defined as a download, registration for a webinar, or anything else we can imagine), and more. By using these tools for before and after measurements, we can begin to see how our enhancements are affecting members and see behavior/usage patterns over time. We can expand on this by not only making data available to everyone involved in the process of web content delivery, but by providing training and guidance on how to use it and make decisions from it. In addition to using web analytics and databases as sources of information, we can explore tools designed to record user behavior from page to page (such as ClickTale or TeaLeaf).

Governance To gain the most benefit from our work in this area, we must be able to share our results and set standards and best practices for the web firmwide. For every new thing that is tried, we should be able to identify successes and failures and learn from both. We can’t do this without more communication (across practices, programs, platforms) and forums to establish style guides, templates, and standards. This is a huge area for growth in 2009 to make sure that our user experience work has a lasting impact.

The Corporate Executive Board

125