Page 1

October 2019 |

U.S. Coast Guard’s

Gladys Brignoni SPECIAL REPORT:

Learning Technology

Workplace Stress Versus Distress - The Path After the MBA - Overcoming Tech Paralysis A Strategic Approach to Digital Learning - PwC’s Digital Revolution


Learner, Know Thyself


oaches learn just as much from their players as their players learn from them. When the person you’re coaching is your own kid it’s even more so. I was my son’s soccer coach last year. Pick any player from his team of 7- and 8-year-olds and it was guaranteed they had more playing experience than their head coach. We had a good season despite that. They learned to play as a team, developed fundamental skills and had fun. Well, most of the time they did. During one game, a loose ball squirted out of the pack right at my son’s feet as he stood a couple of yards from the other team’s goal. He had a clean shot on goal for his first score of the season. He hesitated and in that moment a teammate swooped in for the goal. He was mad at the kid. So before the next game I took my son aside and counseled him to act fast. If the ball comes anywhere close give it a swift kick and go. Don’t think, just follow your instinct, I told him. Without missing a beat, he said, “But Dad, my instinct is to think.”

A good coach can build up valuable meta-skills like communicating persuasively, listening without judgment, thinking critically, reflecting and being self-aware. This isn’t just soft stuff without hard results. Studies show the ability to recognize and understand your own emotions and your resulting behaviors leads to success. A Korn Ferry study found companies whose employees exhibited higher levels of self-awareness enjoyed a higher rate of return on capital. Further, employees at low-performing companies were nearly 80 percent more likely to have low overall self-awareness than those at higher-performing firms. Knowing thyself pays dividends, literally. Brené Brown, author of five bestselling books including her latest, “Dare to Lead,” argues leaders in particular need to develop self-awareness. For good reason, many leaders have a bias to action when they encounter a challenge, she told the crowd during a keynote at the Society for Human Resource Management conference last June. But rather than go straight into problem-solving mode, leaders need to learn to step back and think, Brown said. The aim is to spend the time and energy needed to accurately identify problems. The challenge isn’t to know what to do — it’s to embrace ambiguity and be comfortable not having the answer. For learning and development, that often means I paused a second, said a few words of encourage- slowing down to accelerate results. Valuable meta-skills ment and the game went on. But I kept thinking back like self-awareness aren’t mastered in a single course. to that exchange on the sidelines. It made me realize They require time, practice and reflection. just how much more the person being coached knows For good reason, a lot of attention is focused on about a situation than the coach. And how much reskilling and upskilling workers. The half-life of coaching isn’t about a skill being taught but about the technical skills continues to shrink. But in the push capability being built. to quickly retrain and upgrade the skills of workers, Self-awareness is a powerful thing. It’s often not a it’s important not to lose sight of the tools that can lack of knowledge or the absence of a specific, quantifi- lead to personal and professional success, no matter able skill holding a person back. Rather, it’s the lack of the environment. knowledge of self. Alongside the ability to adapt, learn Success sometimes means recognizing it’s time to new things and embrace change, being self-aware is one change course, not just learn a new skill. That’s the conof the key meta-skills of the modern talent economy. clusion my son came to. He figured out soccer isn’t Knowing our tendencies and emotions, our pat- where his passion lies. Baseball is more his speed. CLO terns of thinking and acting — and how to avoid the worst and bring out the best in ourselves and each other — is a source of significant power. In learning and leadership development it’s a place where you can make game-changing plays. My son knew enough about himself to recognize his Mike Prokopeak own tendencies. He just didn’t quite know what to do Editor in Chief with them yet. That’s where coaching can help.

Lack of skills isn’t what holds people back. It’s what they don’t know about themselves.

4 Chief Learning Officer • October 2019 •


You’re Building the Future from the Ground Up

Gain Skillsets Grow Mindsets Develop Leaders

WE’RE PROUD TO BE A SPOTLIGHT SPONSOR AT CLO SYMPOSIUM, FALL 2019. HOPE TO SEE YOU THERE! ©2019 DeVry Educational Development Corp. All rights reserved. 8/19


What’s your plan for all that potential?


OCTOBER 2019 | VOLUME 18, ISSUE 8 PRESIDENT Kevin A. Simpson





Lauren Wilbur


Steven Diemand


Micaela Martinez


Melanie Lee



ASSOCIATE EDITORS Andie Burjek Elizabeth Loutfi EDITORIAL ART DIRECTOR Theresa Stoodley VIDEO AND MULTIMEDIA PRODUCER Andrew Kennedy Lewis EDITORIAL ASSOCIATES Kerry Snider Francesca Mathewes





David DeFilippo

Josh Bersin Sarah Fister Gale


Matthew Murray


Jack J. Phillips





Nicolai Chen Nielsen Patti P. Phillips Rosina L. Racioppi Caroline Stokes Louise Kyhl Triolo Danny Weill


CHIEF LEARNING OFFICER EDITORIAL ADVISORY BOARD Lisa Doyle, Vice President, Global Learning, Booz Allen Hamilton David DeFilippo, Principal, DeFilippo Leadership Inc. Tamar Elkeles, Chief Talent Executive, Atlantic Bridge Capital Gerry Hudson-Martin, Director, Corporate Learning Strategies, Business Architects Kimo Kippen, President, Aloha Learning Advisors



Free, Live, Online. Chief Learning Officer is pleased to offer live and on-demand webinars!


Driving Strategy: How to Avoid the Top Three Mistakes


The Leadership Challenge


Are Your Leaders Prepared For The Tsunami Of Change In The Digital Age?


CLO Survey: Defining & Mapping Effective High-Stakes Learning

AUG 27

“Grow Your Own” – Tackling Medical Assistant and Healthcare Talent Shortages Check out more previously aired webinars at Available live on the air date and on demand for one year after unless otherwise specified. Check them out today and keep the education going!

Rob Lauber, Vice President, Chief Learning Officer, McDonald’s Corp. Maj. Gen. Erwin F. Lessel, (Ret.) U.S. Air Force, Director, Deloitte Consulting Justin Lombardo, (Ret.) Chief Learning Officer, Baptist Health Adri Maisonet-Morales, Vice President, Enterprise Learning and Development, Blue Cross Blue Shield of North Carolina Alan Malinchak, CEO, Éclat Transitions LLC


Lee Maxey, CEO, MindMax Bob Mosher, Senior Partner and Chief Learning Evangelist, APPLY Synergies Rebecca Ray, Executive Vice President, The Conference Board Allison Rossett, (Ret.) Professor of Educational Technology, San Diego State University Brenda Sugrue, Global Chief Learning Officer, EY Diana Thomas, CEO and Founder, Winning Results David Vance, Executive Director, Center for Talent Reporting Judy Whitcomb, Senior Vice President of Human Resources, Learning and Organizational Development, Vi Kevin D. Wilde, Executive Leadership Fellow, Carlson School of Management, University of Minnesota James P. Woolsey, President, Defense Aquisition University Chief Learning Officer (ISSN 1935-8148) is published monthly, except bi-monthly in January/February and July/August by Human Capital Media, 150 N. Michigan Ave., Suite 550, Chicago IL 60601.  Periodicals postage paid at Chicago, IL and additional mailing offices. POSTMASTER: Send address changes to Chief Learning Officer, P.O. Box 8712 Lowell, MA 01853. Subscriptions are free to qualified professionals within the US and Canada. Digital free subscriptions are available worldwide. Nonqualified paid subscriptions are available at the subscription price of $199 for 10 issues.  All countries outside the US and Canada must be prepaid in US funds with an additional $33 postage surcharge.  Single price copy is $29.99. Chief Learning Officer,, and are the trademarks of Human Capital Media. Copyright © 2019, Human Capital Media. ALL RIGHTS RESERVED. Reproduction of material published in Chief Learning Officer is forbidden without permission. Printed by: Quad/Graphics, Sussex, WI

The use of this seal is not an endorsement by the HRCertification Institute™ of the quality of the program. It means that this program has met the HR Certification Institute’s criteria to be pre-approved for recertification credit.


Human Capital Media is recognized by SHRM to offer Professional Development Credits (PDCs) for the SHRM-CPSM or SHRM-SCPSM. For more information about certification or recertification, please visit



2019 10 Your Career Megan Torrance of TorranceLearning shares her career journey; Airbus’ Louise Kyhl Triolo talks about shifting organizational culture and mindset; and professionals share what they’re reading.

38 Profile Guardian of the Coast

Sarah Fister Gale CLO Gladys Brignoni brings private-sector insights to the United States’ fifth military branch.

60 Case Study PwC’s Digital Revolution

Sarah Fister Gale PwC’s digital upskilling efforts have turned everyday employees into ardent advocates for change.

62 Business Intelligence Technology Choices Need to Be Strategic

Ashley St. John One of the biggest challenges L&D leaders face with technology adoption is how to establish and align a technology strategy with business goals.


8 Chief Learning Officer • October 2019 •

October 2019




A Strategic

Digital Approach to






24 42


Don’t Let Workplace Stress Become Distress


Danny Weill How CLOs can help improve employee well-being in the workplace.


Rosina L. Racioppi Do It Yourself, But Don’t Do It Alone


The Path After the MBA

Sarah Fister Gale Tuition reimbursement for higher education is a waste of money if employees can’t use their new skills.




Jack J. Phillips & Patti P. Phillips Narrative and Numbers


Learning Technology


Josh Bersin VR Makes Learning Authentic, Memorable

Overcoming Tech Paralysis

David DeFilippo Learning to Adapt


Ashley St. John With the speed of technological innovation, how can you prioritize what is best suited to your learners’ needs?

Caroline Stokes EQ and Reskilling in the Age of AI


A Strategic Approach to Digital Learning Matthew Murray and Nicolai Chen Nielsen Given the rapidly evolving digital learning landscape, a clear digital learning strategy is critical.

4 Editor’s Letter

Learner, Know Thyself

65 Advertisers’ Index


Like us:

Watch us:

Join the group:

Chief Learning Officer • October 2019 • 9


Career Advice From


Megan Torrance, chief energy officer at TorranceLearning, answers our questions about her career and the time she’s spent in the L&D space.

one of those clients asked, “Can you build the training that goes into my new LMS?” And that’s when my work in custom course development really got its start. What attracted you to L&D?

How did you start your career in learning? I snuck up on it! In college, I had the amazing fortune to work in an HR department during the early days of the Americans with Disabilities Act. I was hired as a graphic designer, but soon was writing content for their training implementation statewide. After getting a bachelor’s in communication and an MBA, I joined Andersen Consulting (now Accenture), learned Authorware (an early e-learning authoring tool) and how to write training guides … then was never staffed on a project where that was my role. I was managing projects for large shared service center implementations and working my way through the back office functions that nearly every business shares: payroll, personnel admin, payables, receivables, billing, reporting, etc., and finally ended up helping UnitedHealth Group with their learning management system implementation. It was their first enterprise LMS and their software company’s first LMS, too. Once we got that system up and running, I worked for the software company LearnShare with their new LMS implementations and training for a few years. Finally in 2006, Andersen Consulting, Manager 1993-1998 1998 1993

10 Chief Learning Officer • October 2019 •

L&D has such an amazing window into the rest of the organization. We get to help all the other functions perform better. I like to think that, when done well, L&D helps reduce organizational stress loads because we are often providing the keys to competence that help people do their jobs well and confidently. Along the way, we’re combining learning science, technology, facilitation, visual design, experience design and project management to make it all happen. In my role as a vendor in L&D, I have an incredible opportunity to learn because I’m constantly working with a different client’s material. Over the years, I’ve learned how to make spray-dried vinegar powder, cereal pieces, how to use all sorts of software, how to identify a stroke and what do to when you suspect a concussion (useful life skills!), how to work with advertising teams, all about recreational vehicle parts and accessories, both industrial chemistry and food chemistry, how to talk about healthy food choices with preschoolers, clinical research practices, amusement park safety, in-home senior care — the list goes on. Your favorite learning technology trend in 2019? Oh, there’s so much going on — this is a really fun time to be in this industry. But if I had to choose, I

Megan Torrance Inc., Owner 1998-2007

TorranceLearning, Chief Energy Officer 2006-Present

2006 2007 2019

am loving how we’re really seeing traction on learning analytics. We’re seeing a wider adoption of data-driven decision-making in an industry in which we are often not known for our deep evaluation ethos. More and more organizations are using interoperable data (xAPI) to drive those analytics, meaning that we’ll eventually have the interoperability that Shareable Content Object Reference Models gave the e-learning world across a wider variety of learning and performance activities. This means that L&D professionals are broadening their tool set — either by learning the skills themselves or expanding their purview and their teams to include more technology, more data science and more analytics. And it means that organizations can see the L&D function in a new and more powerful light as we are better able to connect what we do to business outcomes. What lessons helped you get where you are? A couple of years ago, consultant Kevin Suboski told me that “business is a network of relationships among people” — it’s so true! And a career is made up of so many lessons from those people, starting with my first boss’s advice to dry my hair before I come to work. Lately I’ve been thinking of the advice of a few formal and informal mentors whose insights and examples are particularly relevant to me now. Mel Drumm at the Ann Arbor Hands-On Museum combines insatiable curiosity with genuine appreciation for the contributions of others — you can’t help but be excited to work with him. Business finance adviser Deborah Christein told me once that saying “no” is just as important as saying “yes” to things. Rob Houck at LearnShare sets an example for listening and hearing someone out before jumping to solutions. That patience is something I try to channel whenever I’m in a tough spot. And my tough-as-nails mother is the one who told me it’s OK to ask for help — that some periods of your life are for giving and others are for receiving. Your most important piece of career advice? This may sound a bit contrarian: Pace yourself. On the one hand, by all means jump into your career with gusto. Say yes. Take on stretch projects. Learn something new. Work hard and long at something you’re passionate about doing right. But also take time to recharge. There are periods in life when steady-as-she-goes is completely fine. There’s a technique in hiking called a “rest step,” where you slow your cadence and rest your muscles while keeping moving. This is on my mind a lot lately. CLO



ions. e quest r fi id p s our ra answer e c n a r r To Megan

The most important part of learning is: Reflection, synthesis, applying in new ways. This is how you know you’ve “got it” — not a multiple-choice post-test.

The most overrated trend in L&D is: I’m going to get myself in trouble with good friends and colleagues here … microlearning and gamification (yes, both!). These are incredibly powerful techniques that are ready to be worked into our “normal” routines for creating great learning experiences instead of thinking of them as buzzwords or something new.

Learning is essential to an organization because: As an officer at the recent NATO Training Technology Conference told me, without people and training “this is just a $1.5 billion hunk of metal and computers.”

The biggest industry misconception is: That measuring training is sufficient. In order to know if training is working, we need to be measuring results instead. The metrics of our success are outside the L&D space.

Know someone with an incredible career journey? We want to hear from you. Send your nomination to Elizabeth Loutfi at Chief Learning Officer • October 2019 • 11


What Are You Reading? Talent Wins: The New Playbook for Putting People First By Ram Charan, Dominic Barton and Dennis Carey The focus on talent before all else (particularly product or process), as well as the emphasis that a talent-first organization has to be led from the CEO’s office (with appropriate CFO/CHRO support), makes this a mustread for CEOs as well as CLO/CHROs looking to influence in this direction. — Rich Cordrey, learning systems and content manager, AutoNation

How People Learn: Designing Education and Training That Works to Improve Performance By Nick Shackleton-Jones [It’s about] design training and educational programs to improve learning, engagement, skills development and performance. What if we have been wrong about learning? “How People Learn” shows L&D professionals a new way of thinking about learning by exploring what happens when we learn. — Jane Underwood, learning and development manager, Wates Group

Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones By James Clear It is such a great way to think about behavior change. I’m interested in it from my own selfish interests to change my behavior, but it also is a very interesting approach on learning design — how can I design a learning experience for continuous incremental improvement using those principles?  Samir Mehta, blended learning leader and digital learn— ing products manager, Center for Creative Leadership

Chief Learning Officer wants to hear from you: What’s at the top of your reading list? Send submissions to Associate Editor Elizabeth Loutfi at

12 Chief Learning Officer • October 2019 •

When: The Scientific Secrets of Perfect Timing By Daniel H. Pink Listened via audio book and it was great research-based info with a companion time hackers guide. I’ve implemented many of the recommendations and I’m more thoughtful about how I use my time. Great book! — Melissa Phillips, training and communications leader, Georgia-Pacific LLC

Thinking in Bets: Making Smarter Decisions When You Don’t Have All the Facts By Annie Duke An excellent read on decision making. How we should measure the reasons and processes not the “outcome” of individual decisions. — Chuck Holden, training and education specialist and LMS administrator, ReliabilityFirst Corp.

Get Real.

The Future


to theAgile. Nothing powers workforce agility more than soft skills. We call them Power Skills™ because they’re that important. Bellevue University’s Power Skills focus on critical thinking, emotional intelligence, communication…in other words, the skills your people need to have to tackle what comes next. No matter how your company changes, skills change or jobs change, with Bellevue University, you’re in charge of where change takes you. Learn from the country’s most experienced educator of working adult learners.

Real Learning for Real Life • 877-824-5516 A non-profit university, Bellevue University is accredited by The Higher Learning Commission (, a regional accreditation agency recognized by the U.S. Department of Education.


Top of Mind Intrapreneurship and unleashing the child within By Louise Kyhl Triolo Career advice from Louise Kyhl Triolo, head of culture, VP of intrapreneurship, Airbus.

I Louise Kyhl Triolo Airbus

s it really possible to shift culture and reinvent large organizations, especially those established for decades? Honestly, for those of us who have truly pushed the transformational envelope within and throughout large systems — it’s a constant challenge, demanding seamless effort, focus and very creative thinking. As a cultivator of the intrapreneurial spirit, I’m always pushing the conventional boundaries to help leaders see differently and help organizations transform. My latest attempt is this: inviting children into the workplace to help envision the future of business! Over the years, I’ve found that one of the biggest obstacles for change lies with our leaders’ fear of losing control. So, what if we could help our leaders experience “letting go” in a safe, yet unfamiliar environment, by getting both children and executives to work side by side on real business challenges? We decided to try it! A group of adolescents were invited to take part in a top-level, one-day executive workshop. Utilizing the Think Wrong methodology, they worked on envisioning the future of business in a more creative, humanistic, societal and environmental way. Initially, the reaction from the executives was surprise and skepticism. In an instant though, the youth and the executives were brought together by a simple drill in which they were asked, “What is your secret talent?” Within hours, the executives were intrigued by the immediate collaborative, trusting and inclusive ways in which the adolescent mind works.

According to

Louise 14 Chief Learning Officer • October 2019 •

Biggest lesson learned from 2019?

One exercise called for each group to share; each executive group chose one person to share, whereas the adolescents naturally split the share-out between themselves. Another exercise was about creating ideas following a certain structure. The executives followed and the adolescents expanded, adding and drawing outside of the boxes to illustrate their ideas, adding to the workshop structure itself. In the end, the kids experienced the abundance of opportunities to be explored and seized in business, whereas the true aha moment for the executives was seeing the children interact — a disruptive mirror to their own rather traditional, “controlled” and formatted ways of functioning. With contrast, we sometimes see clearer. Although we are all born with curiosity to constantly explore, our children naturally show it through their eagerness to create, invent and play with the unknown, isolated from the world of corporate politics until they grow up. We, however, as adults in big businesses “conform” our creativity through established habits: hierarchies, politics and delivery models, for example. We create cultures of operational excellence for maximum business output that can stifle the spirit of the eternal explorer; the intrapreneur, the “creative child” within us, that dreams up the future. So, if we are really looking to shift culture, mindset and practices to stay ahead of the game, we should nurture this creative child within us and create the right conditions to explore the unknown to make our dreams come true. In other words, we need to create an intrapreneurial culture. A culture where each and every one of us can express our unique selves as explorers, innovators and doers. Why not enlist the help of our children in our transformational experiment — after all, they are the ones who will eventually inherit this world from us. CLO Chief Learning Officer wants to hear from you: What are you thinking about? Send your thoughts to Elizabeth Loutfi at

Never give up! In the face of adversity, hold on to your dreams — fight for them because they are the essence of your true self.


Do It Yourself, But Don’t Do It Alone Embrace the power of reaching out • BY ROSINA L. RACIOPPI

O Rosina L. Racioppi is president and CEO of Women Unlimited Inc.

ver the years, I have found the concept of “doing it yourself, but not doing it alone” exceptionally helpful to me personally, to our Women Unlimited participants and to organizations as a whole. “Doing it alone” is often symptomatic of larger problems. For women, it can signal two career-​stymieing obstacles: perfectionism and risk aversion. For organizations, it can mean being out of touch with best practices, innovative trends and demographic shifts. Too often women believe that to prove themselves, they have to do it all. We often see this trend among talented women as they begin our development programs. They are reluctant to delegate, to seek out others for advice and information, or to develop much-​needed internal and external support networks. As these women begin to recognize that trying to do it all themselves has limited them and held them back, they become more willing to reach out to others, resulting in extraordinary benefits to both them and their organizations. For example, when they realize they are not solo players in their organizations, women begin to create networks that keep them informed about the organizational landscape and their role in it. When they effectively delegate, women find themselves better able to focus on leveraging their talents and skills to positively impact the organization. They move beyond just getting the work done. Plus, they have more time to devote to their personal lives. With the help of internal and external support networks, women begin to change behaviors and attitudes that often get in the way of their immediate and long-term success. Finally, when women understand and embrace the power of reaching out, they become more willing to take needed steps to open up new opportunities for success. They are more likely to seek out career-advancing feedback from their managers, take on projects that make them more visible, and make themselves known to corporate movers and shakers. There are also benefits to organizations when women stop doing it all alone. What I call “internal think” is part and parcel of many organizations. It can rear its head in a variety of ways: departments and divisions not interacting with other entities in the organization; managers failing to reach out to other managers; CEOs and top manage-

16 Chief Learning Officer • October 2019 •

ment staying isolated from other organizations within and outside their industry. There are many reasons why it happens. Competitive concerns, day-to-day pressures of getting the job done, or the belief that they have it right and there is no need for outreach. As we work with our corporate partners, we often serve as a conduit for organizations to come together to discuss major issues and initiatives, especially around gender parity. Over and over, we see that old adage come to life: The whole is greater than the sum of its parts.

Combating the ‘doing it alone’ mindset is a battle worth fighting. Here are just a few examples of what happens when corporations and their entities go beyond their borders: • Companies begin to think in new, often profitable ways. They are able to get outside their own heads and see things differently and more creatively. • Top executives are able to expand their networks and reach out to those in like positions who clearly understand the challenges of being a C-suite leader. • Alliances on similar issues are formed, eliminating duplication and often saving money. • Problem-solving improves through interdivisional and interorganizational outreach and brainstorming. As a result, different viewpoints provide effective and formerly not-considered solutions. In my more than 25 years of working with corporations and their talented women, I have found that combating the “doing it alone” mindset is a battle worth fighting. There is a liberating effect that comes from forging relationships with those who, at times, see the world the same way you do and, at others, broaden your horizons. Both individuals and corporations have a great deal to gain from expanding contacts and collaborations. The payoffs resonate professionally and personally. In many ways, reaching out can be a lifeline to greater success at work and beyond. CLO

Future-proof your workforce Stand out in a competitive talent market by offering employees educational benefit programs that improve retention and engagement while driving business performance. Organizations that leverage education as a strategy see: • reduced turnover costs • enhanced employability skills • improved productivity • new talent pipelines We deliver meaningful programs aligned to the needs of all your employees. Learn how Pearson improves business performance by building a workforce that’s more skilled, competitive, engaged, and prepared for the future of work. Copyright © 2019 Pearson Education, Inc. All rights reserved. POLS17603-17964-CH-0819


VR Makes Learning Authentic, Memorable Virtual learning comes to corporate training • BY JOSH BERSIN


Josh Bersin is an industry analyst and founder of Bersin by Deloitte.

’m always a little skeptical about new tools. Virtual reality is no exception. A while ago, I paid my first visit to STRIVR, a company founded by Stanford football players to teach quarterbacks situational awareness on the field using VR. With great skepticism, I put on the glasses and headset to see what the company had done for corporate training. I was blown away. First, the STRIVR team put me into a simulation of a Walmart store during Black Friday. The scenario was designed to help an employee understand how to serve customers in very difficult situations. I felt like I was actually in a store, working in a pressured situation. As I went through tasks, the system coached me on what and what not to do. In the next simulation, I was at a deli counter with people in a queue waiting to be served. I got nervous trying to figure out who to serve first. When I made a mistake or forgot something, the simulation coached me on where to direct my attention to get service right. The third simulation required me to fire an employee who did not want to leave. I had to listen to the employee argue with me and quickly respond with comments that would not exacerbate the situation. To tell you the truth, I did pretty poorly. I could feel my palms sweat as I tried to remain calm and say the right things. In the last simulation, I was at a JetBlue hangar to inspect a plane. I tried to find all the problems in the engine, but sure enough, I missed many things and had to go back several times to get it right. Each of these situations was real, memorable and emotional. Each created “muscle memory” in my mind that no classroom, e-learning or instructor-led training could ever create. Over the past year, I’ve talked with leaders from JetBlue, Walmart and Verizon, all of which are now using VR as part of their training. I’ve seen examples from Chipotle, United Rentals, Fidelity and Intel. Uses range from how to properly inspect planes to working safely on construction sites to teaching customer service skills. New vendors like Vantage Point and Tailspin offer off-the-shelf content for soft skills. Companies such as Warp Industries and CenarioVR are developing lower-​ cost tools. Consulting firms like GP Strategies and Accenture are building these solutions as custom offerings.  The best use of VR is for learning that requires authentic practice in order to truly master. Often referred

18 Chief Learning Officer • October 2019 •

to as immersive learning, this form of real-world practice allows employees to learn by doing, without putting them into dangerous, difficult or potentially costly situations. Typically, these kinds of training don’t scale well, but with VR they do. Every store employee can live through a simulation of a crazy Black Friday sale. Every airplane mechanic can learn how to reliably inspect an airplane engine. Every manager can learn how to diffuse a situation involving an extremely disgruntled employee.

The best use of VR is for learning that requires authentic practice. While implementing VR-based programs can be expensive — some estimate around $75,000 as the starting cost — these investments can be quickly recouped when programs run over time and benefit large numbers of employees. But how do you get started with VR? How do you identify where such an investment will best pay off? My research shows that the highest performing companies allocate up to 40 percent of their training spending for programs that drive competitive advantage for your company or may prevent a loss that dramatically impacts business performance. These would include business-critical skills, competencies and processes. Programs such as IT training, desktop skills, general management skills, project management, and customer service and sales techniques can be purchased off the shelf. Based on the research I’ve seen from VR companies, most VR-based training programs will be 30 percent to 40 percent more effective, measured by retention and actual job performance, than classroom or e-learning. Consider: What are the problems in your organization where such an ROI will give the greatest benefit? Employees today want more and better learning offerings. What better way to get your employees to engage with your company than to give them a state-ofthe-art learning experience — one they will enjoy and remember, and one that will truly give them new skills? VR will only get better over time, so now is the time to consider investing. CLO

Where learning happens D2L is a learning platform for organizations who value learner success as a driver of business success. We believe a great learning experience is critical to driving engagement and retention. D2L’s learning platform helps prepare and engage your workforce with a personalized learning experience, leadership development, peer-to-peer knowledge sharing, and more.

Find out more at

© 2019 D2L Corporation. All D2L marks are trademarks of D2L Corporation. Please visit for a list of D2L marks. The D2L family of companies includes D2L Corporation, D2L Ltd, D2L Australia Pty Ltd, D2L Europe Ltd, D2L Asia Pte Ltd, and D2L Brasil Soluções de Tecnologia para Educação Ltda.


Narrative and Numbers Measure success with stories and data • BY JACK J. PHILLIPS AND PATTI P. PHILLIPS


Jack J. Phillips is the chairman and Patti P. Phillips is president and CEO of the ROI Institute.

ecently, a person involved with soft skills programs described an issue she was facing. She had communicated the value of various soft skills programs to her executives using anecdotes and comments. She was proud of the stories about these programs, but one nagging question remained: Do you have actual data? She also received feedback that her stories needed structure. She needed a thought out process with stories and data. Our ROI Methodology provides a framework for categorizing, collecting, analyzing and reporting six types of data (reaction, learning, application, impact, ROI and intangibles). This framework presents a profile of success with qualitative and quantitative data often collected at different time frames from different sources. Numbers, by themselves, can be boring. When collecting data, always provide a place for respondents to provide comments, which will provide the basis for the narrative. A narrative in a story format makes the results interesting, engaging and memorable. The key is to have both numbers and narrative. Reaction data will have measures to show that participants perceived the program to be relevant, important to their success, and something they would use and recommend to others. In a place for comments, one participant in a program on building trust noted, “This is the most significant professional development program I have attended in my 22 years of work. It was timely and just what I needed.” That brief but powerful story leaves a lasting impression.

Numbers, by themselves, can be boring. Learning data provide measures of learning, whether it is self-assessment, role-playing, test scores or simulations. While important, executives may not be very interested in learning data; however, a good story about learning could be helpful. For example, a participant in a sales training program commented, “I came to this program expecting it to be difficult and confusing. Instead, I’ve learned a tremendous amount and now have the confidence to face my first real customer.” This story brings the participant’s experience to life. Application data often show the extent, frequency and success of use, as well as the barriers and enablers. Comments collected at this level will often be free 20 Chief Learning Officer • October 2019 •

flowing, presenting a great opportunity for stories. In a leadership for high-performance teams program, a participant said, “This was much easier to use than I thought. The reaction of my team was much better than expected and, for me, it is working. The support, tools and job aids provided made it much easier. This will be my new approach going forward.” This story presents a vivid image of success. Impact data are tangible (output, quality, costs and time) and intangible (teamwork, engagement and collaboration). Impact data capture executive and sponsor attention. Displaying those measures requires data and more data. A story can personalize the success. In a program on improving work habits of the team, one participant wrote, “After using this material, I noticed unplanned absenteeism began to reduce quickly. This validated what I learned in the program. I am amazed at the results. Not only did I meet my goal of reducing unplanned absenteeism, but I exceeded it.” The words personalize the impact. A story at the ROI level is also possible, even if the participant hasn’t seen the ROI calculation. One nurse manager, responding to the success of a safe workplace program, said, “I was amazed at how we improved our safety performance. When we evaluated one year with this program, it exceeded our wildest dreams. We avoided some very costly patient accidents. No doubt, this is a positive ROI for the hospital. It’s also an eye-opener of how we can lower costs in the hospital.” This story responds to ROI, even without the actual ROI calculation. This is powerful. Intangibles are measures not converted to money because of time constraints and credibility concerns. Great stories often surround intangibles. One participant in a leadership program said, “This program has enhanced my team’s ability to collaborate, communicate and work together. It’s made an amazing difference in our work and the efforts of my team.” This story connects important intangibles to the program. It is also possible to follow one participant’s story completely through the evaluation levels. Although this may take more time, it shows one person’s experience throughout the process. In short, to be effective, present both numbers and narrative within a framework, through which the audience can logically see how the stories emphasize the data and illustrate the overall value chain. CLO

Better people. Better teams. Better business. BetterUp is the evidence-based, leadership development platform built for lasting behavioral change and business growth.


Learning to Adapt What we can learn from man’s best friend • BY DAVID DeFILIPPO

“C David DeFilippo is principal of DeFilippo Leadership Inc., executive learning officer for Northeastern’s D’Amore-McKim School of Business and an executive coach at Harvard Business School.

hange is the only constant” is often cited as the mindset shift leaders need to adopt with today’s rapid business cycles, burgeoning technological advancements and geometric power of social networks. Heraclitus’ quote is rooted in his theory that life is in flux and therefore constantly changing as a normal course. Contemporarily, for organizations this reality means adapting or risking competitive relevance, and for individuals the forces of change mean learning and adapting or being left behind in the workforce. Given the enormity of these pressures on organizations, educational institutions and even government agencies to keep up, I was recently drawing inspiration from our family dogs, present and past. We are dog lovers — always have been — and I suspect we will always share our home with at least one or two canines by our side. Over the years, I have marveled at our dogs’ ability to adapt, internalize and live with significant change. These adaptive traits are borne out of the history of canines, who diverged from their wolf origins genetically and behaviorally 15,000 to 20,000 years ago and began to be domesticated in Europe and Asia. This domestication process had practical roots in that dogs’ sense of smell helped with hunting for food, their thick coats provided warmth for nomadic hunter-gatherers who resided in camps (hence the expression “a three-dog night”) and they provided safety for humans among other animals. This history and close relationship have made dogs “man’s best friend,” given that dogs are the one species that have most successfully evolved with humans. However, this close connection also provides insights into some differences between canines and the humans they cohabitate with, especially when comparing their respective responses to change. Let’s look at this phenomenon across three categories of change: Structural: A few years ago we rescued a dog whose owner had passed away and integrated her into our home and our pack of two other males. This particular dog was a retired champion show dog with an alpha personality; however, it was evident that she was still confused and probably even grieving the loss of her owner. She assimilated into

22 Chief Learning Officer • October 2019 •

our home over the course of a month by first bonding with my wife and then having a few skirmishes with one of our two males who was equally dominant, all while ignoring me until she had established her ruling place in the pack. This has been the established order ever since.

We can learn a lot from dogs by embracing their adaptive traits and attitude. Physical: Years ago, we had a dog who, at a young age, went blind very quickly, and we witnessed the incredible physical adaptability of the species. With eating and going outside being two established and important aspects of daily dog life, we worried that the new version of these routines would require intervention and be a difficult adjustment. So we marveled at how, with no formal training or direction from us, this dog figured out how to walk from room to room by hugging the walls to find his water bowl, navigate to the back door, and enthusiastically remind us that it was time for his dinner by showing up next to his bowl exuberantly tapping his paws. Process: Our three current family dogs’ internal clocks run like a Swiss watch with respect to waking, eating their bi-daily meals and going to sleep, so each year we are amazed when daylight saving time arrives in the fall and spring and they adjust within a day or two. I, on the other hand, take about a week to get back into my daily routine and feeling like myself again. With these three types of change, the constant pattern among our dogs has been their speedy assimilation, creative adaptability and ability to move forward without dwelling on the past. Given this resiliency, it makes me think we can learn a lot from dogs by embracing their adaptive traits and attitude in order to work through change, both as individuals and as part of teams in organizations. CLO

Maximize the potential of your workforce Become a Capella employer partner and receive tailored learning solutions using our real-world education and training programs aimed at addressing your workforce needs. Differentiate Your Benefits With our special pricing models, we can work with you to provide your employees a degree for low to no cost.

See the Results We assess learning in a way that connects students to their career goals, and prioritizes mastery of specific competencies that can be applied directly on-the-job.

Advance Your Leaders Competency-based, job-relevant training programs to build your employee’s foundational leadership skills.

Keep Good People Attract top talent and keep them engaged with the help of benefits that focus on skill-building, leadership and career development.

Talk to a rep about Capella University today. 866.360.8047

Accreditations: Capella University is accredited by the Higher Learning Commission. Higher Learning Commission:, 800.621.7440 Capella University: Capella Tower, 225 South Sixth Street, Ninth Floor, Minneapolis, MN 55402 1.888.CAPELLA (227.3552) Š 2019 Capella University 19-1274

24 Chief Learning Officer • October 2019 •

Don’t Let Workplace Stress Become

S S E R DIST How CLOs can help improve employee well-being in the workplace. BY DANNY WEILL


mployee well-being plays an important role in business profitability and growth, with higher levels of mental and physical health linked to increased productivity and, ultimately, better company performance. Yet, it’s a factor that’s all too often overlooked, and people are feeling the impacts of stress more than ever. With study after study proving that stress is manifesting in alarming ways, from increased rates of high blood pressure and heart attacks to depression and anxiety, employers and HR leaders should be concerned, since often the cause of this stress is work. According to The American Institute of Stress, work is unequivocally the single biggest source of stress for adults. The Centers for Disease Control and Prevention report that 40

Chief Learning Officer • October 2019 • 25

percent of employees consider their job very or extremely stressful. Nearly two-thirds of respondents in a 2018 Korn Ferry survey said their stress levels at work were higher than they were five years prior, with 76 percent saying work stress has had a negative impact on their personal relationships, 66 percent saying they’ve lost sleep due to work stress and 16 percent who said they were forced to quit a job due to stress.

Good Stress vs. Bad Stress A certain amount of stress is good for us. It provides energy to do the things we need to do and to perform well. It motivates, stimulates and makes us more focused on the task at hand. Our bodies need a certain amount of adrenaline and cortisol in our system to function at our best and reach our highest potential. When experiencing “good” stress we feel energized, excited and in the zone. For example, investment bankers, emergency medical service providers, lawyers and other people who work in intense professions typically understand that stress is inherent in their jobs — and many even thrive on it. However, stress can quickly become distress when there is too much of it, too often or when it lasts too long. So how can CLOs and HR leaders help transform their workplace wellness to better support employees’ personal lives in a way that drives their professional engagement?

mental function. They can have trouble remembering things or be unable to make decisions. Their minds can go completely blank or constantly be racing, making it harder for them to focus on tasks. Emotional reactions to stress can be harder to spot in the workplace. Many employees may be able to keep their emotions under wraps at work, only to have it affect their life at home. Feeling tense, nervous, anxious, irritable and impatient are some common reactions to stress. Additionally, according to Posen, more than half of all patients who present with stress also present symptoms of depression. Employees may rely on unhealthy behaviors in an attempt to deal with the feelings of stress, from compulsive eating, smoking and drinking, to swearing, crying and yelling. Other behavioral symptoms may manifest as knee jiggling, fidgeting and pacing back and forth as the body tries to exert the pent up “fight or flight” energy caused by stress hormones. Conditions of the job or specific role can also help you determine whether an employee’s health and well-being has been or will be affected by stress. Analyzing roles within the company can help to determine whether there is potential for undue stress on employees, allowing you to take steps to avoid it before it causes an issue. Types of work conditions that have been found to cause stress in employees can vary, but there are some common offenders. Incorrectly designed work roles are one, when employees are not clear on job requirements and expectations or are handling too many responsibilities. This can also include roles where an employee’s skills are being underutilized, they are unable to take breaks as needed or they are dealing with a heavy workload. Poor environmental conditions can also contribute to stress. Constant distractions, lack of ergonomic work stations, lack of physical interaction by working in an isolated area, or dangerous conditions that create health and safety concerns are a few examples. Additionally, situations that create job insecurity and lack of room for advancement or promotion can cause stress over time for an employee, leading them to be unsure of their position or value within the company, and can contribute to overall low morale. It’s important to note that not only can stress have an immediate effect on an employee’s well-being, but it can also be detrimental to their health in the long term. The Encyclopaedia of Occupational Safety and

Stress can quickly become distress when there is too much of it, too often or when it lasts too long.

Identify the Signs What you need to look out for in the workplace is abnormal, destructive levels of stress among your team members. Recognizing the symptoms of stress in employees and understanding their impact is critical to effectively dealing with the contributing issues and improving employee well-being. Physician and expert on stress management David Posen says stress can show up in various ways for different people, with symptoms appearing across four categories: physical, mental, emotional and behavioral. Physical symptoms of stress are more easily identified and attributed to stress than other symptoms. Internally, employees can experience headaches, back pain, and tightness in joints and muscles. Symptoms that are externally visible can include clenched hands, frequent trips to the bathroom and, most commonly, fatigue. Employees who are dealing with stress can cause further stress on themselves by working with reduced 26 Chief Learning Officer • October 2019 •

We couldn’t create a 25th hour in the day.

Leadership 480SM is a strategic framework to help you invest your time wisely to hire, promote, and develop exceptional leaders at every level. Find out how:

Health reports that job stress can lead to an increase in cardiovascular disease and risk of heart attack, mental illnesses such as depression, anxiety and burnout, workplace injury, and even suicide. In turn, these can also contribute to factors that affect the company as a whole, such as increased absenteeism, lateness and employee’s intentions for quitting.

How You Can Help Your Staff All this points to the fact that millions of Americans are on the verge of burning out — so much so that the World Health Organization recently listed burnout as an occupational phenomenon caused by chronic workplace stress — and employers need to step in. You can help your staff in a number of ways. Ensure easy access to resources. While employees may not always go to their manager when they have a problem, managers can ensure that an employee has the resources they need in order to begin addressing any issues they may have. These resources can help educate the employee on how to identify the symptoms of stress and the root cause, allowing them to begin to develop a solution that they can recommend implementing with their manager. Notice changes. According to expert Ash Bender, psychiatrist and deputy clinical director of the Mood and Anxiety Program at the Centre for Addiction and Mental Health, being able to observe changes in an employee’s behavior, function and performance in the workplace is one of the most important skills you can cultivate. Periodically reviewing an employee’s performance allows managers to observe pattern changes, address the employee’s issues and recommend individualized solutions. Develop a culture of support. If managers aren’t addressing their stress in healthy ways, neither will their staff. Managers must be educated and trained to minimize workplace stress not only for themselves, but to provide appropriate solutions for struggling employees. Digital tools can be used to create a culture of caring and provide much-needed education to help employees and managers get on the right track to reduce sources of stress. Organizational change. It’s important to identify the stressful aspects of work (such as excessive workload or conflicting expectations) and work with others to design strategies that reduce or eliminate the identified stressors. The advantage of this approach is that it deals directly with the root causes of stress at work in order to prevent issues before they arise.

Prioritize mental health. It comes as no surprise that when organizations champion proper mental wellness, employees are healthier, take fewer sick days and are more productive. This can be achieved by sharing information and access to services on topics like mindfulness, stress management and self-care to support psychological well-being. Relax your rules around paid time off. It can be stressful to know you only get 10 days of paid leave per year. Rather than “waste” those precious days, many employees choose to come to work even when they’re unwell. This presenteeism is detrimental to both individuals and the organization. Giving employees leeway to take mental health or sick days when they need to ensures your employees stay healthy and can come to work feeling their best. Design an uplifting physical work environment. Did you know the spaces around you can affect your mental well-being? Things like natural lighting, calming artwork and comfortable furniture go a long way toward putting your employees’ minds at ease.  Consider alternative therapies. Perks like onsite massages and puppy therapy are growing in popularity. Not only are they effective ways of breaking up the pace of the workday, but they can also re-energize employees and put them in a positive mood. Offer access to financial and legal advice. The cost of living is rising faster than wages are, and employees are feeling it. According to a 2016 Gallup poll, 64 percent of respondents were concerned they may not be able to afford retirement and 41 percent were worried about covering their basic monthly bills. To make matters worse, a survey conducted by ARAG, a legal protection services firm, determined three-quarters of Americans had at least one legal matter to attend to in 2016. Giving your staff access to resources on personal finance and legal advice can help reduce their stress. Stress is an inevitable element in the workplace and your employees’ lives. In order to implement strategies and tactics to keep employees motivated, it’s important to be able to identify the symptoms that prevent them from being productive and engaged. Then, using the tactics described here will help keep stress in check so you can improve and maintain your employees’ well-being — and keep them more focused and productive. CLO

Job insecurity and lack of room for advancement or promotion can cause stress over time.

28 Chief Learning Officer • October 2019 •

Danny Weill is vice president of partnerships at LifeSpeak, a digital employee health platform.


Helping Leaders Embrace Change through Coaching Change can be overwhelming and difficult, especially if it’s constant. Seventy-seven percent of HR practitioners and leaders report that their organization is in a state of constant change. It’s especially so for those tasked with helping to implement change across an organization. It’s been reported widely that organizations’ change management initiatives experience a high rate of failure. In fact, 85% of organizations can cite at least one failed change management initiative within the last two years! Both the human and the financial costs associated with a failed change initiative can harm an organization, so it’s important that organizational success is not left to chance. This change management dilemma prompted the International Coach Federation (ICF) to partner with the Human Capital Institute (HCI) to conduct research on how organizations navigate change. The top cited reason for why change is unsuccessful is resistance from employees. Further, less than a quarter of our survey respondents indicated confidence in their employees’ change capabilities. What can leaders do to address this resistance?

What Steps Can Leaders Take Our research identified communication, planning and leadership as the most critical factors impacting either the success or failure of a change management initiative. An employee’s professional level in the organization can dramatically impact their personal experience with the change. Executives who are responsible for announcing the change initiative to the organization often fail to gain a deeper understanding about what it will be like for the individual who is responsible for a given task before, during and after the change initiative. Our research showed that most leaders of organizations that underwent a recent transformation chose to invest in more traditional learning activities to help employees move the change forward.

Of the learning activities sponsored by leaders, more than half of all organizations surveyed indicated that they offered very formal activities (e.g., classroom training, web-based training, meetings with senior leaders) to help usher in their change initiatives. Unfortunately, these same activities were rated as some of the least helpful, with classroom training and e-learning appearing in the bottom three activities. It’s perhaps not a surprise that a traditional approach to learning when faced with change yields underwhelming results.

What Successful Change Leaders Do Using specific talent and business outcomes as criteria, our research identified a sub-set of respondents’ organizations as high-performing organizations (HPOs). HPOs are more likely to report that change management initiatives meet or exceed their expectations for success and have greater confidence overall in employees’ change capabilities. Why? These HPOs were much more likely to leverage very collaborative learning activities for change management, such as one-on-one coaching and team coaching with a professional coach practitioner.

Unlike traditional development activities, which are generally broad, passive experiences, coaching puts the individual, team or work group in control of the development process.

The learning activities favored more often by HPOs also were considered most helpful to achieve the goals of their change management initiatives. Specifically, the following learning activities with a coaching focus were far more likely to be rated “very” or “extremely” helpful than


more traditional, formal learning modalities: • Work group coaching with professional coach practitioner • One-on-one coaching with professional coach practitioner • Access to manager/leader using coaching skills • Team coaching with professional coach practitioner Unlike traditional development activities, which are generally broad, passive experiences, coaching puts the individual, team or work group in control of the development process. Therefore, development becomes active and specific to individual needs. Coaching can also help employees explore any resistance to the change that they may be feeling.

Proactively Using Coaching through Change For the organizations where coaching is already present, we found they are much more likely to use coaching throughout the change initiative rather than using it as a final effort to save a failing project. In these organizations, Human Resources/Talent Management/Learning and Development professionals who use coaching skills and managers/leaders who use coaching

skills are most likely to be involved in change management initiatives. Furthermore, the following applications are the most frequently cited for using coaching for change management: • Addressing leadership style, strengths and blind spots • Overcoming resistance • Building resilience • Building change readiness • Finding processes and tools Those who lead organizations with strong coaching cultures have already realized the impact that coaching activities can have when navigating the sea of change. For many, activities like team coaching have moved beyond a “nice to have” and become an organizational necessity that allows their employees to engage in a more collaborative and creative process for handling the challenges that inevitably come with launching a major change initiative. To be successful, change leaders need to focus the effort, provide consistency and organization for the stakeholders, and address resistance, fear and unclear expectations and information gaps. They can begin by effectively communicating the rationale behind the change and empowering employees and teams with tailored support that will enable them to thrive in the face of change.

The International Coach Federation (ICF) is the leading global organization dedicated to advancing the coaching profession by setting high standards, providing independent certification and building a worldwide network of trained coaching professionals. To learn more, visit


Why is there so little traction on our most critical skills? By: Michelle Eppler, Ed.D., Associate Vice President, Human Capital Lab, Dean, College of Continuing & Professional Education, Bellevue University

Do a quick google search on “soft skills” and you’ll get millions of articles about the importance of these skills. By now it’s safe to say that nearly every Learning and Development leader understands that soft skills are the key to workforce competitiveness and agility. Agility is important because, with the pace of change in job requirements, those who know how to learn (i.e., listen, communicate, collaborate, think critically, etc.) will be the first to adapt to new ideas, skills, and jobs. Nothing is as effective at creating workforce agility as soft skills. In fact, they’re so important, we call them Power Skills™. In a recent study of nearly 600 learning and development leaders we produced with Human Capital Media and Chief Learning Officer, it was clear that Power Skills are critical. Sixty-three percent of respondents said they have a substantial or critical gap in these skills – three times the number of technical or functional skills gaps. And for many organizations, the gap is widening.

At our organization, it’s almost taboo to even talk about soft skills, let alone measure them.” — Survey Respondent “Making the Business Case for Soft Skills” August – September 2018

This was the sentiment of many of our survey respondents. While most organizations know Power Skills are important, comparatively few are taking steps to measure, train, and assess competencies…let alone build human capital strategies around gaps in these competencies.

In fact, the data show that most organizations lack reliable metrics to evaluate Power Skills mastery. Most reported they measure soft skills through self-evaluation, whether an item on a satisfaction rating or a section on an engagement survey. But reflect on this: Self-reporting would not be a satisfactory tool to use when assessing technical skills competencies. It’s hard to imagine any learning professional suggesting that they were going to hire a programmer because she reported she was “excellent” at JavaScript or Python. Other responses included anecdotal evidence, including manager ratings, customer complaints, or other subjective evidence. See Figure 1. FIGURE 1: METRICS USED TO TRACK SOFT SKILLS 39%

Items on employee satisfaction/ engagement surveys


Anecdotal evidence


Manager ratings


Customer complaints




360-degree team assessments

The issue with competency assessments of Power Skills is that there has not been a reliable tool for assessing them. Until now. Bellevue University’s Human Capital Lab™ has been researching which soft skills are most important to creating workforce agility for more than eight years. The Lab worked with the University to integrate the introduction, practice,


and mastery of these skills into academic curricula, and also stand-alone boot camps on each of the seven Power Skills. Last year, responding to the needs addressed by our study, we created the Power Skills PRO™. In designing the Power Skills PRO we worked with proven testing assessment methodologies to understand how learners think about core concepts related to communication, collaboration, self-management, decision-making, and the other Power Skills. The PRO presents learners with real-world situations and gauges their response to them. Still in its pilot phase, it is designed to test skills. Where to Start

While it’s true that most respondents to our survey indicated a substantial or critical gap in soft skills, it is also true that most recognize they don’t adequately assess competency. Starting with a reliable, objective assessment of competencies is a good place to start. It creates a benchmark you can use against which to evaluate training programs in these critical skills. The Power Skills PRO provides an immediate assessment of competency in each of the seven Power Skills for each individual taking it. Further, we have begun to work with teams to help them understand how they complement each other in various skills. This is an interesting analysis. For example, we have been able to uncover potential causes for workplace issues and/or lack of productivity.

was really excellent at all of the Power Skills except communication. His team did not appear to benefit from his skills in decision-making, values clarification, self-management, or collaboration. The reason became obvious with the team review of Power Skills abilities. If your company or your team elects to use the Power Skills PRO to create a benchmark of your team’s proficiencies with these important skills, experts on the Power Skills from the Human Capital Lab work with you to interpret results and talk about next steps. Another benefit of using the Power Skills PRO to assess Power Skills competencies is that you may identify team members who are very strong at many of these skills. While you may have recognized this potential intuitively, this is another tool for identifying skills. Or you may be surprised to learn that you have teams in your organization who are extremely talented in the Power Skills. If you’d like to be part of the pilot of this groundbreaking tool, contact the Human Capital Lab. Dr. Michelle Eppler is a highly respected education professional who combines her expertise in effective learning with the leadingedge research conducted by Bellevue University’s Human Capital Lab.

In one example, we saw that the team manager

Bellevue University’s Human Capital Lab was founded to research and report credible links between learning interventions and enterprise impact. It was an innovative concept at the time, and since has added to its work through the production of new tools, case studies and research projects that demonstrate the impact that learning makes on employee lives, employee opportunity, and corporate outcomes like revenue, profit, and customer satisfaction. Today, the Lab is known as the credible, inspirational resource about how to plan, engage, and enhance human capital. People are our most important source of productivity. The Lab is devoted to helping enterprises strategically deploy them for optimal employee engagement, productivity, and fulfillment.


Creating Growth With Marketing Analytics (In 4 Steps) By: Darden Professors Rajkumar Venkatesan and Paul W. Farris The question of how to leverage big data has inspired conferences, university courses and degrees — and CMOs in sore need of guidance on what all those numbers are saying. What each organization needs is its own analytics system that is focused on customer behavior. It should be actionable, future-facing and support the organization’s broader strategy. To make better business decisions, managers should be able to answer marketing questions by: • Determining which data are relevant. • Selecting the appropriate technique for analysis. • Performing analysis to gain insights on the relationship between marketing activities and customer behavior. • Using predictive models based on experiments or historic information to simulate hypothetical situations, in order to identify the ideal combination of marketing activities. • Linking insights and the optimal marketing mix to wise marketing decisions.

The Endgame of Marketing Analytics Resource allocation is the endgame of analytics for any company. Using marketing analytics properly, any firm should be able to determine the optimal level of spending it should make on each of its marketing channels to maximize success.

Step 1 Determine the objective function - what is the metric the company wants to set as its goal for optimization? This may be one of any number of methods of assessing business success, including conversion rates to sales, incremental margins and profits, customer lifetime value (CLV), near-term sales lift, new buyers, repeat sales, market share, retention rates, cross-sell rates, future growth potential, balance sheet equity and business valuation.

Step 2 Connect the marketing inputs of a firm to the objective of resource allocation. Business managers’ intuition is of paramount importance in this step, as it allows the marketer to correctly decompose a metric. For example, if a company is examining gross profits, what are the attributes of the business that contribute to those profits, and are the relationships between the various components accounting identities or empirical? An accounting identity can be computed without any unknowns. For example, net profit is gross profit minus marketing costs. If both gross profit and marketing costs are known, net profit can be computed easily. The relationship between marketing costs and unit sales is more complex and driven by numerous unknowns. You cannot directly sum the investments in marketing to obtain sales. The relationship is empirical because the manager must analyze historical data to develop a formula that transforms the marketing inputs into sales. This formula ideally will provide a “weight” that translates a product’s price into sales. These weights provide a best guess based on historical data, wherein several factors in addition to price also affect sales. Empirical implies a best guess or prediction; identities are certain.

Step 3 Estimate the best weights for the empirical relationships identified in the second step. A common method for identifying these weights is to build an econometric (regression) model. Which marketing inputs of interest (for example, price, advertising, sales calls) should be considered as having an effect on the dependent variable? With this regression model, the marketing manager can predict the outcome metrics for different marketing input levels. This is the mathematical


model that describes the relationship between the independent variables (for example, price, advertising, sales calls) and the dependent variable (for example, market share, profits, CLV).

Step 4 In the last step, a firm can reverse the process to identify the optimal value of the marketing inputs to maximize the objective function. This gives a detailed picture of what the company’s precise marketing spend should be on each channel it uses to market its product. The goal of marketing analytics is to determine the effectiveness of a company’s various marketing strategies. For each strategy, the company is looking to assess its return on investment (ROI). Marketing ROI is equal to profits related to marketing measures divided by the value of the marketing investment. Determining ROI is simple arithmetic; however, estimating and defining the effects of ROI is difficult. Maxi-

mizing long-term profits is often not simply a matter of shifting funds from low ROI to high ROI activities, because there may be strategic considerations not fully captured in the ROI measures themselves. Examples are brand building and new customer acquisition versus the need for short-term sales, balancing push and pull efforts to support distribution channels, and targeting market segments that are of strategic importance. To improve marketing success, companies must consistently make good decisions about which customers to target, the resources to be allocated to the selected customers, and nurturing the selected customers to increase future profitability. By using statistical analysis techniques, firms can use past customer behaviors to predict how customers will react to different marketing channels, and managers can then optimize spending on each channel. Darden’s Innovative Management in the Digital Age program will help your high-potential leaders leverage data analytics to improve decision-making, uncover unmet market needs, and guide innovation.

Darden Executive Education is a top-ranked, global provider of professional development. Delivered by the University of Virginia Darden School Foundation and taught by the Darden School of Business’ highly acclaimed faculty, Darden Executive Education prepares and inspires leaders to succeed in a global marketplace. Offering more than 30 open programs and partnering with leading organizations worldwide to develop custom business solutions, we provide personalized, transformational learning experiences at our locations in Charlottesville and Rosslyn, Virginia, online and around the world. Learn more at


The value of business education collaboration New research uncovers the connection between business education and lifelong learning. By Tim Harnett The United States economy shows no sign of slowing. In July 2019, the unemployment rate was 3.7%, little changed from previous months.1 In this tight labor market, sourcing workers will become tougher. A recent survey indicated recruiters already face challenges. Three-quarters of HR professionals with recruiting difficulties identified a shortage of candidate skills as a roadblock to filling job roles.2 Mitigating this skills gap will require a multifaceted approach from both businesses and educators to develop workers who are ready to tackle the jobs of the future.

The vast majority of L&D practitioners see an opportunity for leaders and educators to collaborate directly on a wide range of mission-critical workforce development topics.

Nurturing a successful talent pipeline will take more than posting to multiple job boards or working with staffing firms. Organizations will need to develop partnerships with business schools to fill their talent needs. Thankfully, today’s business schools are more than just recruiting centers. Educators help businesses create solid talent pipelines and networks. Business schools are also adjusting their programs accordingly and working with businesses to more thoroughly develop the incoming workforce. As the world’s largest business education alliance, AACSB recently partnered with the Human Capital Media Research and Advisory Group for insights into what these business

education partnerships should look like. The purpose of the survey was to discover how organizations used partnerships with business schools and how these partnerships applied to the future of work. What topics are most important to business leaders?

One way business education partnerships can be successful is by having conversations important to business leaders. Most L&D practitioners see an opportunity for leaders and educators to collaborate on a wide range of mission-critical workforce development topics, such as experiential learning, the future of work, lifelong learning, and skills/curriculum (Figure 1). Figure 1: Critical conversations business leaders and business school educators should be having 74%




61% 48% 23%

Experiential learning

Future of work

Skills and curriculum

Learning technologies

Diversity and inclusion


Lifelong learning

Corporate learning and development professionals also value business educators for their applied research acumen. More than half of survey respondents expressed a desire to see new business education research on topics such as leadership (51%), workforce education and

1 The Employment Situation. U.S. Bureau of Labor Statistics, June 2019. 2 The Global Skills Shortage: Bridging the talent gap with education, training, and sourcing. SHRM 2019. 3 AACSB Business Practice Survey. Human Capital Media Research and Advisory Group. 2016.


training (53%), and talent development (55%). Now more than ever, organizations support their internal L&D efforts with external resources. In 2016, just half of all organizations used some external resources to support their internal learning function. By 2019, that number was 79%.3 What groups should businesses connect to?

When it comes to the business education ecosystem, many survey respondents reported that they are already connected to a number of groups, such as their peers (72%), area professional groups (71%), and credentialing organizations (54%). Educators should encourage these connections and work with business leaders to create more. Facilitating local and regional networks composed of organizations and business educators may hold promise. A higher percentage of L&D professionals are currently connected to business schools in their immediate area as compared to schools glob-

ally. However, nearly a third expressed a desire to be even more connected to geographically proximate business schools. There’s also an opportunity for business educators to collaborate with organizations on industry-related projects. Eighty-nine percent of survey respondents said that the topic they most want to see at an event on business education is how to develop employees for the future of work. Forward-thinking corporate learning and development professionals recognize the value that business schools can deliver beyond simply filling recruitment pipelines with newly minted graduates. By understanding today’s business pain points, adjusting programs to address skills gaps, and disseminating their research results, educators can work to meet those challenges and improve the workforce of the future. Figures may not total 100% due to rounding. All data taken from the 2019 Business Education Partnerships survey unless otherwise noted. For additional briefing reports related to preparing future leaders, lifelong learning, and more, visit

As the world’s largest business education alliance, AACSB International (AACSB) connects educators, students, and business to achieve a common goal: to create the next generation of great leaders. Synonymous with the highest standards of excellence since 1916, AACSB provides quality assurance, business education intelligence, and professional development services to more than 1,700 member organizations and over 840 accredited business schools worldwide. AACSB’s mission is to foster engagement, accelerate innovation, and amplify impact in business education. The global organization has offices located in Tampa, Florida, USA; Amsterdam, the Netherlands; and Singapore. For more information, visit


Guardian of the Coast CLO Gladys Brignoni brings private sector insights to the United States’ fifth military branch.



ladys Brignoni is not your typical military officer. She was born in San Juan, Puerto Rico, and moved to a small town in Indiana as a child. She spoke no English, and at the time English as a second language curriculum was uncommon. “I had to work really hard to learn the language on my own,” Brignoni said. That obstacle started her down the path to a career in education. Brignoni received her bachelor’s degree from Purdue University, and later her master’s and doctorate from Indiana University. When she landed a job as a professor at Old Dominion University in Norfolk, Virginia, she was thrilled. She taught students of all levels and ages, many of whom were the children and spouses of military personnel. “I loved seeing the lightbulb go on when they learned something,” she said. It was her first brush with the military and foretold a future to come.

Beyond the Ivory Tower After several years of teaching, Brignoni felt like she wanted to make a bigger impact. So she joined the Peace Corps, working as a language and cultural training specialist primarily in Slovakia. “It was a tough job but I loved it,” she said. After her Peace Corps term limit ended, she stayed in government work, holding learning leadership roles at the Joint Forces Staff College, the Center for Naval Intelligence, the Naval Facilities Engineering Command Atlantic, and within the Navy Warfare Development Command. “I never thought I’d work in the military, but I met the qualifications,” she said. This combination of private sector and academic experience along with her experience in government service drew the attention of the Coast Guard, which was looking for its first full-time chief learning officer and deputy commander of force readiness command in late 2011. 38 Chief Learning Officer • October 2019 •

At the time, the Coast Guard was looking for ways to improve and centralize its workforce development programs to ensure “readiness” among all staff, said Rear Adm. Brian Penoyer, force readiness commander for the Coast Guard and Brignoni’s current boss. That didn’t just include readiness for duty. “Military organizations need to develop better citizens,” Penoyer said. Most personnel will return to the private sector after their tour is completed, and the Coast Guard wanted to be sure they had the skills to succeed. That’s where Brignoni’s unique combination of skills and experiences comes into play. “She had become phenomenally important to me,” Penoyer said. “She provides the context and expertise in learning theory and instructional design that many of us lack.”

“She provides the context and expertise in learning theory and instructional design that many of us lack.” — Rear Adm. Brian Penoyer, force readiness commander, U.S. Coast Guard

Officers and Citizens Unite This is particularly important for the Coast Guard, which falls in an unusual position of being a formal military service that operates under the Department of Homeland Security during peace time. It has approximately 85,000 staff, including 40,000 military personnel on active duty, 7,000 reservists, 8,000 civilian personnel and 30,000 auxiliarists who support Coast Guard efforts when necessary. It also supports a diverse and complex set of mission types, which range from environmental protection and ice-breaking efforts, to vessel safety checks, to oil rig inspections. “Most people think we just do search and rescue, but it’s so much more,” Brignoni said.


Chief Learning Officer • October 2019 • 39


All of these workers require training to do their jobs effectively, but it hasn’t always been equally distributed. “We’ve always done well with enlisted and officer resources,” she said. “But we haven’t spent as much on civilians.” Ironically, the nonmilitary staff tend to stay with the Coast Guard longer, building their careers over time. That makes them valuable additions to the team, but also increases the risk of attrition. “These civilians can leave to work for any other agency,” she noted. “If we don’t provide opportunities — Gladys Brignoni, CLO and deputy for them to grow, we will commander, U.S. Coast Guard lose them.” Selecting Brignoni was the first step in creating a learning-centric culture that could address all of these needs.

“Most people think we just do search and rescue, but it’s so much more.”

A Devastating Loss

still being developed, will provide a better framework to support these staff and give them the training aids they need to do their jobs to ensure the safety of everyone involved. “We want to ensure something like El Faro never happens again,” she said.

Big Goals, Small Budget While this program has received strong support and funding, the Coast Guard’s training budget in general is always tight, and new programs must be cost-​effective, far-reaching and relevant for all kinds of staff. That’s forced Brignoni’s team to get creative with their resources. When they began digging into civilian training, for example, one of the biggest problems was that staffers weren’t aware of their training and advancement options. Annual employee survey data showed that while civilians loved working for the Coast Guard, they felt there was no career mobility and that no one cared about their career development. “It was absolutely true,” said Steve Keck, director of the civilian career management team and force readiness commander. In response, Brignoni and Keck created the Civil Career Management Office and built a website to support it. The website is a dedicated space where any Coast Guard employee can identify professional and leadership development opportunities, map their career paths to move up in the organization, and look for joint detail rotations to expand their experience. “It used to take 55 websites to find all this information, but now it’s all in one place,” Keck said. “It’s a one-stop shop for all civil career performance development.” Brignoni’s team also found that many of the relevant online training elements available from other military organizations were inaccessible due to firewall issues. “It prevented us from taking advantage of simple solutions,” she said. So she found a way around it by launching a YouTube Coast Guard training channel.

Prior to Brignoni’s hiring, the Coast Guard’s commandant had established several early action items to support career development across the organization, which included better training for civilians and ship inspectors. Brignoni was tasked with delivering these action items. One of the critical tasks included ramping up training for ship inspectors who make sure shipping vessels meet safety and regulatory standards. This goal became a top priority in 2015, when the SS El Faro, a U.S. cargo ship crewed by U.S. merchant mariners, sunk at sea during Hurricane Joaquin, losing everyone aboard. “It was tragic,” Brignoni said. The vessel had passed its annual Coast Guard inspection, despite former crew members who said it had structural problems, as reported by CNN in October 2015. “Within the Coast Guard it made us realize we need to do a better job of ensuring our workforce has the training they need to meet industry requirements,” Brignoni said. That drove her team to revamp the training program for ship inspectors and environmental protection agents. Following the events, the Coast Guard approved a two-year project to review all tasks necessary to effectively assess vessel safety and compliance, and to work with industry to develop more robust training programs. Brignoni’s team found that these auditors are expected to understand thousands of policies and to be on top of constantly evolving shipboard and port Gladys Brignoni, CLO and deputy commander for the U.S. Coast Guard, technologies. “It can be overwhelming,” takes innovative approaches to creating simple, useful solutions that she said. The training program, which is anyone can use to learn new skills. 40 Chief Learning Officer • October 2019 •


Brignoni is helping bridge the gap between military training and the real world.

Through the channel, she challenged all Coast Guard enlisted, civilian and volunteer members to submit their own “how-to” videos to support current and future employees. The videos have to meet the broad criteria of being helpful and relevant to the job, and they can’t give away any military secrets. “We’ve been overwhelmed by the number of submissions,” she said. The channel is an excellent example of Brignoni’s innovative approach to creating simple but useful solutions that anyone can use to learn new skills or close gaps. “It’s really popular, and it was really cheap to build,” she said.

xAPI is Changing Everything Brignoni’s biggest impact has come through a partnership with the Department of Defense to participate in the Advanced Distributed Learning Initiative, a government program that conducts research and development on distributed learning and shares outcomes with other agencies. The ADL’s goals include identifying relevant industry standards, leveraging tools and content to improve the effectiveness of learning programs, and developing distribution networks to share content across agencies. ADL stakeholders collaborate closely with industry and academia, making it a perfect fit for Brignoni, who aspires to blend best practices from the military, private sector and academia to meet the needs of her learners. A key outcome of this partnership has been the Coast Guard’s adoption of Experience API (xAPI), which is a specification that organizations can use to collect data about a wide range of learning experiences that individuals have completed. The open-source platform lets users define their learning experiences in simple language (i.e., “I completed this activity” or “I took this course”), which are captured in the platform. These “statement objects” are compiled and

linked to similar objects to demonstrate an individual’s skills and to find patterns across roles and job categories to indicate success and advancement. The data is captured in a consistent format, regardless of where and how the learning took place, making it easy to organize and analyze. “The single biggest thing Gladys has done for the Coast Guard is to show us what’s possible with xAPI,” Penoyer said. “It’s allowing us to take into account what people are learning on their own.” Through the xAPI technology, Brignoni’s team is developing ways for Coast Guard staff to record all of their learning experiences — on the job and outside of their Coast Guard work. “We believe so much learning occurs outside the classroom,” she said. Documenting that experience will give the Coast Guard insights into the true abilities of its workforce, and to focus interventions in areas where there are gaps. This is particularly valuable for the thousands of volunteers and reservists who bring vast knowledge to the table, Penoyer said. “They have lives outside the Coast Guard where they develop all kinds of skills. If I don’t know what they know, I can’t take advantage of it.” They hope to use the platform to reshape job descriptions, and to give Coast Guard staff clear career paths so they can move up the ranks. Brignoni is also exploring ways to translate those experiences into formal certifications and other real-world proof of skills. She’s currently working with Keck on a Coast Guard University concept in which the Coast Guard will partner with higher learning organizations and universities to provide micro-certificates that match military training and experience to academic degrees.

All Hands on Deck Brignoni noted that she doesn’t spend all of her time in the office tackling technology issues and partnering with academia. When the Coast Guard needs her unique set of skills, she is happy to head into the field. Most recently, during Hurricane Maria, Brignoni returned to Puerto Rico to help with communication challenges and to support an onboarding program to help DHS staff and volunteers better understand the culture and how to successfully work with local communities. “It has been the most rewarding experience working in the Coast Guard thus far,” she said. However, Penoyer and Keck are happiest when she’s in the office. “Gladys is a disruptive thinker in a good way,” Keck said. “She uses her experience from the Navy and higher education to introduce new ideas, and it is bridging the gap between military training and the real world.” CLO Sarah Fister Gale is a writer based in Chicago. Chief Learning Officer • October 2019 • 41

42 Chief Learning Officer • October 2019 •

Tuition reimbursement for higher education is a waste of money if employees can’t use their new skills. BY SAR AH FISTER GALE


f you think your company doesn’t have an MBA sponsorship program, you might want to look again. While many large organizations don’t have a formal policy in place, when local leaders identify high performers interested in continuing education, they often just make it happen. That can be a good thing and a bad thing, said Daniel Szpiro, dean of the school of professional programs at Marist College in Poughkeepsie, New York. “In these cases, the ultimate decision-makers are people who are closest to that employee, and who understand their potential better than someone in a corporate learning and development office.” However, when these sponsorships occur ad hoc, companies have no way to track the return on investment or to set goals for how the company will benefit from providing this expensive piece of education. This is a common problem, even in companies that do have a formal MBA sponsorship program, Szpiro said. Companies treat MBA sponsorship as a benefit rather than an investment, which means they fail to think about what the employee — and the company — will do with that asset once it is attained. “You would never invest $100,000 in a piece of equipment without having a plan for how it will benefit the business,” he said.

Chief Learning Officer • October 2019 • 43

When companies fail to leverage an employee’s new MBA skills through promotion, advancements or leadership opportunities, that investment is wasted and those employees will get frustrated, said Michael Desiderio, executive director for the Executive MBA Council. “If there is no path forward, that’s a problem.”

‘You Don’t Need Those Skills’ This was the case for Alan Thompson, global sales manager for Bird.i in Stirling, Scotland, a startup that curates satellite imagery for business applications. Thompson became interested in higher education later in life. He dropped out of school at 16 and worked full time in the financial industry for years before launching his own wealth management company in 2015. That’s when he realized he needed more business training. His partner at the time had gotten an MBA a decade prior. “He said it was the most valuable thing he had ever done,” Thompson said. So when Thompson discovered his work experience qualified him for an executive MBA program at Edinburgh Napier University, he signed up and covered the cost himself. “It was fantastic,” he said. “It opened my eyes to new ideas and taught me to ask the right questions.” It also changed his career focus from trying to make the most money possible to finding opportunities to use his skills to have an impact.

Companies treat MBA sponsorship as a benefit rather than an investment. Before completing the program, Thompson sold his business and got a job as a manager at Prudential U.K., leading a sales team in a newly created region. Prudential didn’t offer tuition reimbursement, but Thompson continued the program, completing it a year later. That’s when he realized his new skills didn’t align with his current job. “I had gotten all of this new knowledge and value from the MBA, but the business didn’t recognize the value,” he said. His manager wouldn’t give him any new responsibilities and wouldn’t help him find a more senior position in the company. So he quit and took the job at Bird.i. “I would have stayed if they had given me an opportunity to showcase my new abilities, but they wouldn’t,” he said. “I had to move to contribute.” It’s a common problem for companies that don’t have a formal strategy for what to do with employees 44 Chief Learning Officer • October 2019 •

once they complete MBA programs. “People develop these new skills and a new way of looking at their work, but their managers tell them, ‘you don’t need those skills,’ ” Thompson said. Thompson admits his timing might have been off, but even in companies that do offer tuition assistance, such planning is essential to keeping MBAs engaged and employed. “The MBA is not just about getting the credential,” Desiderio said. “You have to map a path forward together.” Otherwise employees will quit — reinforcing the false notion that companies shouldn’t pay for higher education, because employees will just leave once they get it. “Employers are the ones perpetuating the problem,” Szpiro said.

Ambition and Business To generate the most value from MBA investments, companies need to change the way they think about the value of higher education to employees and the company, said Nicole Rogers, talent strategy manager for Accenture in Chicago. Accenture’s Global Scholars program covers the majority of MBA tuition costs and gives employees the chance to work full time on a two-year program away from the office. While it’s a big investment of money and time away, Accenture views these programs as an opportunity for its best people to be advocates for the company within their MBA cohort, and to bring a fresh perspective back to the business. “It introduces diversity of thought and new skills,” Rogers said. “I think that combination is relevant.” Each candidate in the program is sponsored by their supervisor, who works with the learning and development team to create a long-term plan for their return. Employees also have access to a career counselor throughout the education process, first to help them make the decision to pursue the MBA, then to develop a career plan once they achieve it. “These one-on-one conversations ensure that an employee’s interests and new skills will align with areas where Accenture wants to grow,” Rogers said. Finding the balance between individual ambition and business need is important, she added. “You want to think about the long-term value an MBA has for your people and for your organization when assessing whether to invest in this kind of program.”

Degrees for All AT&T has a similar attitude toward higher education. The company supports employees’ pursuit of various degrees, including MBAs. The company has several university partnerships that allow employees to complete MBA programs online while still working full time, reimbursing the full cost of tuition.

Ideal Modality for High-Stakes Learning

CLO learner

82% of CLOs would use collaborative learning but 87% still rely on the classroom.

To make your collaborative transition, contact us today.

“Any employee can get a master’s degree,” said Jason Oliver, vice president of AT&T University Operations in Dallas. “As long as they have the support of their supervisor and a performance development plan, we are in.” These investments are part of AT&T’s ongoing reskilling efforts to prepare employees for the future of work. While many employees pursue technical degrees, Oliver believes MBAs are just as relevant. “You can always acquire new technical skills, but we also need strong leaders who understand global markets and how to lead in this business environment.” To ensure employees are successful in their academic pursuits, supervisors meet with them every semester as part of the tuition reimbursement process to discuss their progress and to have personal development discussions. Employees can also keep their internal employee profile updated on competencies they’ve acquired, courses they’ve completed and credentials they’ve achieved. This ensures hiring managers across the organization are aware of their training when vetting internal candidates. “It’s all part of our talent development ecosystem,” Oliver said. Hamed Pakravan, an AT&T scrum master, is in the midst of completing his online MBA through a program at North Carolina A&T, one of the organization’s university partners. Pakravan’s background is in technology and program management, but he aspires to leadership roles. “My focus is on improving business processes, so when I heard about the MBA program I jumped on it immediately.” While the online program doesn’t offer the face-toface opportunities that an on-campus MBA provides, he’s found it to be surprisingly collaborative. “There are a lot of group meetings and peer work, and the technology makes it effortless,” he said. He discovered two other AT&T employees from other parts of the country in the same program and has made a number of new network connections. He also appreciates the lack of commute, though he admits that even working from home, it is a big time commitment. Pakravan spends two to three hours studying on most weeknights, and another three to five hours on the weekends. “You need to be self-driven to make it work,” he said, noting that even with the full support of his supervisor, “no one is going to tell me to do the work but me.”

A Long-Term Plan Part of what drives Pakravan is knowing the future opportunities the MBA will afford him. AT&T has a leadership development track for high performers that requires them to have an MBA. Employees accepted into the program spend 18 months doing three rotations in different areas of the business to find where 46 Chief Learning Officer • October 2019 •

their skills will be the best fit and to identify opportunities for their next career move. Pakravan plans to apply as soon as he completes his MBA. “A lot of people question the ROI of an MBA,” he said. He agrees that without a career plan linked to the training it’s easy for these new skills to “go into cold storage.” But in a company like AT&T, where leaders actively support ongoing education and show that they value it by creating leadership opportunities for graduates, the training has clear value. “They see the future coming, and they are invested in reskilling their

Finding the balance between individual ambition and business need is important. people to fill these roles.” AT&T isn’t alone. Almost half of companies (49 percent) offer some graduate education assistance, according to the Society for Human Resource Management’s 2018 “Employee Benefits” survey. Similar data from the Executive MBA Council show that 20 percent of online MBA students received full tuition reimbursement in 2018, and another 34 percent got some of their costs covered. “Full reimbursement is definitely trending down, but more than half are getting some support,” said Desiderio. However, it’s not enough just to cover costs. “You have to be willing to sit with these individuals, look at their performance development plans and have an honest conversation about their future in the company.” Desiderio noted that there may be times when an MBA won’t be a good investment for the company if they don’t need or foresee putting that person in a leadership role. But in most cases, it can create new opportunities for succession planning and customizing the skills of the future workforce. “These conversations need to happen earlier in their career planning, and it has to add value for both sides,” Desiderio said. HR leaders also need to realize that ambitious people will pursue further education — whether you want them to or not. “People want training and the chance to control their own development, but they also need your support,” Desiderio said. If companies want to keep their high performers, figuring out how these advanced degrees can benefit the business may be the best way to meet everyone’s needs. CLO Sarah Fister Gale is a writer based in Chicago.


Learning Technology

Tech Paralysis As we head into the Fourth Industrial Revolution, it’s easy to feel like a deer caught in digital headlights. With the speed of technological innovation, how can you prioritize what advances are best suited to your learners’ needs?

48 Chief Learning Officer • October 2019 •



t has become widely accepted that the Fourth Industrial Revolution is upon us. Coined by Klaus Schwab, founder and executive chairman of the World Economic Forum, the concept refers to how technologies such as artificial intelligence, autonomous vehicles and the “internet of things” are merging with the physical lives of humans, “blurring the lines between the physical, digital and biological spheres.” It can be a bit frightening when you consider the rate at which things are changing — like trying to envision the spatial size of the universe. As Schwab wrote in an article for Foreign Affairs in December 2015, “The speed of current breakthroughs has no historical precedent. When compared with previous industrial revolutions, the Fourth is evolving at an exponential rather than a linear pace.” With so much technological transformation happening around us, it can be easy to feel overwhelmed — both as individuals and within organizations. This was a topic of frequent discussion during the 2019 Association for Talent Development conference in Washington, D.C., this past May. Chad Udell, managing partner of new product strategy and development at enterprise mobility service provider Float, was one presenter who focused on this hot topic during the conference. Udell, with his colleague Gary Woodill, led a session titled “How to Handle the Shock of the New: Introducing a Framework for Evaluating Emerging Technologies.” Based on their book, “Shock of the New: The Challenge and Promise of Emerging Technologies,” the ses-

sion grabbed attendees’ attention with Udell’s use of the phrase “tech paralysis.” Everyone is facing some degree of technology overload these days, and the learning and development space is certainly no exception. Following, Udell shares some insights on how to face the growing wave of technology head on and prioritize what’s necessary, what’s a nice-to-have, and what’s likely a passing fad. How can the overwhelming amount of transformation happening right now lead to tech paralysis in a business?

Chad Udell: We’re at a really interesting intersection here of a variety of converging forces. We’ve got this overwhelming juggernaut of growth in terms of technology that started around the beginning of the internet age but has rapidly transformed and grown faster and faster with mobility and ubiquity of digital devices. With these types of overwhelming growth, it can be very easy to become completely overwhelmed in the sense that you just don’t even know quite where to start. It’s the whole cliche of, “How do you eat an elephant? One bite at a time.” People don’t even know which bite to begin with, because a lot of it just looks so darn tasty! That’s the other thing that’s also very problematic — that there are many, many choices to be made, and many of those choices are good. There certainly are some bad ends and some bad pieces that you want to avoid, but because there are so many options, it’s

Chief Learning Officer • October 2019 • 49


Learning Technology

difficult to whittle them down. When you couple that with the fact that businesses do not have unlimited funds or unlimited resources such as people to research and experiment, determining where you can place those bets is a paralyzing activity in and of itself. How do you define the “shock of the new”?

Udell: What we’re discussing when we talk about the shock of the new is this constant state of reeling from punch after punch of being hit in the jaw with new gadgets, new tools, new processes and new technologies that we’re expected to essentially become fluent with. So, we have digital transformation, and we have mobile transformation, and now we’re expected to be fluent with these technologies in every aspect of our business. And because most of the time people are concerned very much with the day to day, it’s really difficult for them to elevate above that to understand how to work with these items strategically, how to think about things strategically and how to be able to overcome that shock.

business, Google Cloud, Microsoft’s Azure Cloud, etc., have really taken off. Well over 50 to 60 percent or more, depending on the cloud service that we’re talking about, of Fortune 500 businesses are depending on these cloud services to do their work. This is obviously in the broader sense in business, but we’re also seeing a transformation in the learning space. There are many cloud services and tools out there, everything from your learning management system to your authoring tools to your ELCMS to a variety of very specific point solutions for things like assessments, mentoring, coaching, scheduling, workshops, audience response tools and so on. The decentralization of these tools and the decentralization of that data is leading to this kind of new disruption, or “shock of the new,” which is almost like cognitive services as a platform. So, outsourcing decision-making, outsourcing performance support, outsourcing a variety of different key business processes that typically would have been part of the training department’s domain — generating performance-support tools, knowledge bases, software simulations and training systems — all of those types of things are now starting to move outside of the four walls of the enterprise and into the cloud as well, and that is very disruptive and is only going to increase.

Business leaders need to acknowledge as part of that decisionmaking process that it’s not a one-size-fitsall type of decision.

What are some of the most “shocking” technologies that have emerged in the past five or 10 years, or that are just beginning to emerge?

Udell: One of the things that I think has been most shocking to business is the rampant and widespread adoption of mobile before many businesses were ready to fully adopt it. My business has been in practice since 2010 building mobile apps for enterprise, and I would say that we were a bit early in the sense that we were ready to deploy mobile applications for the Fortune 500 way back in 2009, 2010, when we were just getting started. But many businesses did not have strategies in place to deal with those types of things — governance, policies, procedures, acceptable use patterns … ways that you would employ these devices for success inside your business. That was very, very disruptive. Following on from that very closely: As the devices became disintermediated from the business and became personal devices in people’s pockets, the data sources themselves were becoming disintermediated, and this was kind of the advent, the rapid takeoff of the cloud. People were using cloud services on their phones and on their tablets before the business was able to fully support them. And now we’ve seen in 2015, 2016, 2017 and beyond, Amazon’s web services 50 Chief Learning Officer • October 2019 •

How do you suggest learning leaders decide what is important and what is a fad?

Udell: In “Shock of the New” we lay out a framework [called the BUILDS framework] that you can use for decision-making processes around understanding the business impact of the software; implications in terms of user experience and how to create a great experience for employees; how to determine what the impact of the system is going to be upon your workers and your organization; what types of learning models it supports and how that fits in with the culture of your organization; what dependencies need to be put in place or need to be in place prior to making decisions around that technology; and what types of signals are in the broader industry in order to make your decision less of a bet and more of an educated guess. I think business leaders need to acknowledge as part of that decision-making process that it’s not a one-sizefits-all type of decision. Every business within every vertical industry is going to have its own type of lens they would need to use this framework for. While some

Attempt the end and never stand to doubt; nothing's so hard but search will find it out.” —Robert Herrick (1591-1674) Transform expert opinions into actionable insights The HCM Research and Advisory Group delivers thought leadership tailored to your business. Survey our highly engaged group of practitioners and create customized whitepapers and infographics based on our research. Work with us to gather and discuss strategic insight for the leaders of tomorrow and beyond. Learn more at


Learning Technology

industries, say for example microprocessors, semiconductors and things like that, may be extremely hightech and highly dependent upon information systems in order to do their work, there are just as many businesses that are decidedly low-tech but still have some need for technologies. I’m thinking about things like agriculture, logistics, and so on, which superficially seem like very low-tech industries but in reality are heavily dependent on technology for success. So therefore, the way that they use those technologies is going to be radically different. Even inside of a business you’re going to have audiences that have differing levels of technological savviness and willingness to adapt and accept those technologies. While you may have sales professionals who are heavily incented to close deals, and therefore they’re going to take just about any way that they can to get a leg up in order to close that deal or hit that quota, you may have some other professionals who, honestly, are just there because they work the 9 to 5, or it’s a weekend job. Or maybe this is even a secondary job where they don’t really want to be put into a position where they have to put up with a lot of baloney in order to get through things. So making that technology as easy as possible, as accessible as possible, and really selling that whole “what’s in it for me,” is vital to achieve success, which will help you prevent buying into fads or pushing fads that ultimately erode your credibility as a learning leader. It’s important that as we look to determine if a technology is something that’s important or a fad, we are looking at it from an audience-specific point of view.

core of the problem, provide the information when they’re doing the task to operationalize that training, and potentially even remove or at least reduce the need for training in the first place. That being said, we don’t really advocate or act as a proponent in the book for any one technology. It’s rather a comprehensive look at all technologies. We talk about some technologies like internet of things, blockchain, 3-D printing, AR and so on, but this framework should be able to be used with any number of those technologies plus some more forward-​thinking ones that might be a number of years out — the redesigned internet; what will happen when IPv6 [Internet Protocol version 6] is pervasive everywhere; internet of things; “internet of everything”; blockchain and beyond, into the realm of even quantum computing, quantum internet and robotics. What we are advocating for is not necessarily thinking that any one technology is particularly important, but rather [having] a broad awareness of technologies in general and how to put plans in place to achieve success with these advanced technologies. It’s really taking a technology-​​agnostic approach to evaluating technologies. Hopefully this will serve as a kickstarter to get involved in the broader technology discussions that are going on inside of your organizations. Even if you haven’t already begun explorations of how blockchain or internet of things or AR are going to reshape your workers from a learning and development perspective, make no bones about it, there are definitely professionals inside your company who are doing some amount of research, development, prototyping and even possibly pilot testing around some of these more advanced technologies. So, informing yourself on them, creating a task force around this concept of, let’s get a workforceof-the-future mindset, let’s talk about and review some of these advanced technologies, this is going to allow you to reenter those larger discussions that are taking place in your hallways, and not be relegated to, “Oh, learning and development, you’ll get that technology after operations has it and after sales has it.” You should be able to use this as a jumping off point to be part of that vanguard and part of that discussion to help lead these decisions at your organization. Technology is definitely going to be a core component of employees being able to do their jobs going forward. CLO

As we determine if a technology is important or a fad, we should look at it from an audience-specific point of view.

Broadly speaking, do you feel that certain technologies are particularly important for learning functions to take advantage of?

Udell: There are certainly technologies that myself and my partners at Float have our eyes on. We come from a mobile background, and our heritage is in delivery of content for mobile devices and providing information in a just-in-time, just-enough, and just-for-me type format, so we really are keeping our eyes not just on microlearning and mobile learning but different formats and forms of that medium. And we feel that augmented reality is a very, very promising technology in that space because it can definitely close that gap in performance by providing just-in-time information in the flow of the activity and not just supplement or augment training but in some ways actually get to the 52 Chief Learning Officer • October 2019 •

Ashley St. John is Chief Learning Officer’s managing editor.

The future of learning is here

Diagnostics and Self Assessments

Behavioral and Technical Development

Professional Learning Academies

Sales Leadership Training

Discover your people’s potential, unlock their hidden talent, and drive your organization’s performance with Kaplan Leadership and Professional Development. We help you align your learning strategy with your business goals. We scale across teams and borders.

Together with you, we make learning work.




Learning Technology

A Strategic

Digital Approach to

learning The digital learning landscape has rapidly evolved and is fundamentally different today than just five years ago. Given these developments, it’s critical to have a clear digital learning production strategy.

54 Chief Learning Officer • October 2019 •



o you have moments when you realize you don’t have a clear grip on your digital learning ecosystem? For example, when you speak with a potential vendor, only to find out they’ve already run an event with a business group in another region? Or when a business partner points at a popular app on their phone asking, “Why can’t we do this?” and you don’t have a good response? Or when you see a line item for licenses in your budget and you have to ask your production manager what it’s for? It can be challenging for learning leaders to actively manage and tell a convincing story about the digital tools and platforms we have in our portfolio. We tend to focus on one or two larger systems (learning management systems or learning experience platforms), often at the expense of our complete range of tools and platforms. Consequently, we don’t have an integrated road map or comprehensive point of view on how to create world-class learning experiences, enable our teams to rapidly build and iterate content and courses, and rigorously measure outcomes. And that

means we don’t have the right ecosystem in place to enable and track the behavior and mindset shifts needed to achieve our employees’ performance goals and execute on our business strategy. Typically, it’s a story of haphazard adoption, periodic bursts of innovation and lack of a clear digital learning strategy. Common reasons could be that: • Tools are introduced for a single program and then organically spread to other programs without clear guidance on the potential impact to behaviors and performance. • Purchasing decisions are based on leftover budget or are scattered across different groups. • The existing skill set of designers and developers encourages inertia and determines which production and delivery services are performed internally or externally. • Selection processes are driven by a “wow” factor — what is seen

Chief Learning Officer • October 2019 • 55


Learning Technology

at an expo or because a competitor L&D function is doing something with a certain platform. • Tool and platform management is decentralized and fragmented, with no clear accountability or internal champions. • Learning functions become dependent on legacy homegrown apps and platforms, which are inherited and difficult to retire. One financial services client we partnered with had an energetic innovation team that would successfully pilot new tools and platforms — but struggled to achieve broader adoption because it was too removed from the broader business needs the learning function was serving. The innovation agenda was not strategically connected to longterm value planning for the learning team. An uncoordinated approach like this is increasingly a problem because the authoring and delivery tools we use impact our learning solutions, our learning organization and processes, and our ability to generate meaningful reporting and analytics. As our approaches and delivery options become increasingly digitized, we should be treating these choices as a priority, and as an enabler to achieve and maintain business impact. A large pharmaceutical client we worked with had more than 20 disparate LMSs globally, leading to confusion among learners and costing millions of dollars in maintenance a year. Using outdated or suboptimal digital tools not only negatively affects the effectiveness of learning — and by extension the business value at stake — but also can impact your organization’s ability to successfully recruit and retain top talent. Part of the issue is that we might not have a good way of describing our tools and platforms

Number of different technologies in use




13 10






Source: Towards Maturity CIC, 2017.

56 Chief Learning Officer • October 2019 •




and explaining our selection and use decisions. We gravitate toward highly technical differentiations or buzzwords like gamification or microlearning. Our categorizations can be blurry. The challenge is exacerbated by the growing volume and complexity of our tools and delivery platforms. As Figure 1 indicates, the number of technologies learning functions use has been rising steadily over the past few years. Additionally, the increasingly dynamic digital learning landscape has rapidly evolved and is fundamentally different today than just five years ago. Figure 2 (on page 58) outlines some of the key shifts we have noticed in digital learning ecosystems. Given these developments, it’s critical to have a digital learning production strategy and perform a periodic ecosystem analysis. The lack of a clear strategy can contribute to the perception from the business that L&D is not capable of meeting the organization’s reskilling, transformation and development needs.

Establishing a Digital Learning Strategy We find that a great digital learning strategy has a number of key characteristics. First, it is proactive and specific, driven by your organization’s current and future requirements. Second, it is focused on learner needs and fit for purpose to serve the continuous omnilearning expectations of your organization’s people. Third, it should be founded on the generation of meaningful learning data and the analysis of impact metrics. And fourth, it is a key component of your team’s internal brand and innovation agenda. A clear digital learning strategy has numerous benefits. It establishes a point of view for the business on the criteria and rationale for the adoption and retirement of new technologies. Advancing your organization’s digital transformation, it aligns your priorities and decisions to your broader business objectives. The strategy ensures consistency and avoids redundancy across learning teams. And it achieves efficiencies from skills building, templates and production processes. As a starting point for developing a perspective on your digital learning ecosystem, ask yourself a few fundamental questions: Is your ecosystem the right size? Do you have too few tools or too many? Is it balanced? Do the tools you have fit your learning needs? How do you make decisions on adding and removing? What’s your process for managing the platform life cycles? Finally, how do you support it? What skills are required? What budget? What integration questions?

Go Beyond Learning Paths

Move Up the Learning Curve Learning paths are only as good as their

learning experiences. Real

learning experiences result in real impact – high performing leadership, diverse views, digital savvy - and require

collaboration, practice, and mentorship.

Contact a NovoEd Learning Specialist today and accelerate your organization’s learning curve. 415.379.0145

Real Learning for the Skills That Matter Most



• Legacy offline tools, updated to the new version every couple of years.

• Cloud-based tools and HTML/responsive requirements for multiple devices.

• Specialist developer skills, using linear development processes that tied organizations to specific tools.

• Shorter lifespan expectancy and easier to switch.

• An industry dominated by dedicated learning vendors.

• Emerging tools and niche providers.

• Evolving tools (new features, repositioned in new ways) and blurring lines between categories.

• Tools and platforms focused on PC delivery.

• Use of non-specialized tools — not just for learning; used for presentations, ideation, marketing.

• An emphasis on e-learning courses and expectation that tools published into SCORM packages.

• Consumer grade, ease of use — lower barriers to entry, DIY and designer-developers.

• Long-term contracts and software management.

• Shift to microlearning, digital assets/knowledge, xAPI, etc.

If you’re struggling to answer these questions, it could mean you don’t have alignment or a clearly communicated strategy. We propose a practical approach to get started, focusing on your desired future state (not your current state) to determine what authoring tools and delivery platforms should be in your ecosystem. This is a two-step process: (1) Map your learning needs on a digital learning grid. (2) Determine the “importance index” for each tool and platform using data-driven criteria to assess their relative significance for your learning needs. FIGURE 3: DIGITAL LEARNING GRID High importance

Medium importance

Low importance

Self-paced simulations AR/VR User-generated content Habit apps

Video role-play Recurring games Virtual coaching

Team-based simulations Connected classroom Team-generated content

Self-studies Adaptive learning Interactive video Bots

MOOCs/SPOCs LMS/LXP Learner journeys

Virtual classroom Event-support app

Microlearning Videos/animations Interactive guides/e-books Performance support Reinforcement app

Wikis Podcast series Daily quizzes

Webcasts Communities

Knowledge/informational Source: Matthew Murray and Nicolai Chen Nielsen.

58 Chief Learning Officer • October 2019 •


Solo/on demand


Step 1: Map Your Learning Needs A digital learning grid (see Figure 3) maps tools, modalities and platforms with a focus on the learning experience, not from the perspective of the learning function’s needs or features of the technology. (Ignore the importance rankings for now — we will get to that in a bit.) The vertical axis ranges from presentational approaches at the bottom to application and immersion at the top. Although categorizing learning technologies according to learning levels can be blurry, this axis roughly corresponds to Bloom’s Taxonomy. Tools and platforms toward the bottom are generally better suited to knowledge transfer; those at the top are generally better suited to experiential learning and synthesis. Learning by doing is achieved primarily through the tools and platforms at the top; learning in the flow of work is achieved principally through approaches at the bottom. The horizontal axis ranges from solo and asynchronous learning on the left to social and synchronous learning on the right. The tools on the left are generally well suited to personalization, anytime/ anywhere access, and just-in-time/just-enough learning. Platforms toward the right benefit from live, group-based sharing and interaction. In the middle you have intermittent learning or learning from others over time. The grid is not an evaluative framework — top right is no better nor worse than bottom left. The objective here is to identify which learning needs should be covered by your digital learning production and delivery technologies. There will be exceptions and the mapping of the tools and platforms is STRATEGIC APPROACH continued on page 64

BUILD A SMARTER WORKFORCE Purdue University Global’s tailored education solutions and degree programs can help your organization better manage talent and stay ahead of the competition. Our personal approach to helping you meet core business objectives goes far beyond training your workforce. Our expertise in talent development spans industries and functions including cybersecurity, health care, retail, operations, customer service, and more. Through a strategic partnership, we can customize a plan to help you attract, retain, and promote talent while managing the return on educational investment. Purdue Global—a world-class university that delivers world-class education solutions.

16040 02/19

Learn more at


PwC’s Digital Revolution BY SARAH FISTER GALE


wC may be best known as a tax and auditing services firm, but its consultants are expected to be knowledgeable about all financial and business trends. These days, most of those trends involve some level of digital transformation. “We have a lot of conversations about what it means to be digital,” said Joe Atkinson, PwC’s chief digital officer. While everyone has a different answer to that question, the constant thread is that digital knowledge for PwC employees can’t be theoretical. PwC clients rely on their consultants to help them navigate the digital landscape and to implement solutions that will make their businesses more efficient. “So all of our upskilling efforts have to be focused on business outcomes, client outcomes and people outcomes,” Atkinson said. This realization led to PwC’s current digital upskilling journey, which began two years ago and continues to evolve. “Digital upskilling has to be a focus for us as a knowledge organization,” said Sarah McEneaney, digital talent leader for PwC in Chicago. A key element of the program is that it isn’t just for consultants. Every one of PwC’s 50,000 U.S.-based employees have access to digital training tools, courses and content, and they are all encouraged to use the training to support their own growth and the business goals, McEneaney said. “We expect everyone to opt-in.”

What’s Your Digital Fitness Score? PwC’s upskilling journey began with the development of its Digital Fitness App, a virtual training tool that employees use to learn about the latest digital trends, including artificial intelligence, machine learning, automation, augmented reality, blockchain and dozens of other topics. With each new topic, the app gives users an assessment to gauge their existing knowledge, which Atkinson likened to a digital fitness credit score. “It’s a rapid way to figure out what you know, and what you need to learn,” he said. The app uses that score to suggest content that will close gaps and help users build confidence with the topic. All of the content is in short, easy-to-digest formats, including blogs, podcasts and short interactive lessons, so employees can learn as they go. “We wanted it to be available whenever they have time,” Atkinson said. 60 Chief Learning Officer • October 2019 •

SNAPSHOT PwC’s digital upskilling efforts have transformed the learning culture, driven process improvements and turned everyday employees into ardent advocates for change.

The content won’t make them experts, he noted. But it does provide them with a baseline understanding of the topic and the business implications related to it. “It allows them to have effective conversations about how these technologies can be part of business solutions,” Atkinson said. The app has been extremely popular. It launched in 2017, and today, 45,000 employees access it every week. An updated version is expected to launch in 2019, which PwC will share with clients as well as employees.

Badges Drive Talent Mobility The second phase of the upskilling journey encourages employees to expand their knowledge through microdegrees and certifications, which require more time and effort than the app training. Some of the certification programs are built in-house around specific PwC service offerings, while others are accessed through third-party organizations. When an employee completes any of these programs, they receive a badge they can share on their internal and external profiles. The badges encourage friendly competition and break down barriers to talent mobility, McEneaney explained. It’s a popular way for employees to showcase their new skills, and it gives hiring managers a way to identify and vet internal candidates, ensuring the best employees are selected for new projects. Atkinson’s goal for 2019 was for every employee to achieve the “digital acumen badge,” a PwC microdegree that requires users to go on four “client quests,” where they have to use leading-edge technology to solve fictitious business problems. As of July 2019, more than 80 percent of U.S. PwC employees had completed the badge certification. “I’m particularly proud of that because it’s not required,” he said. “We just encourage them to do it and hope they are inspired.”

2,000 Agents of Change Employees who want to take their learning further also have the chance to opt-in to one of PwC’s 20 digital academies, where they can get in-depth, face-toface training on topics like data cleansing and desktop automation. To date, 30,000 employees have completed one or more of the two-day academy courses and are now using those skills on client projects. Finally, for employees who are even more inspired to learn and share their digital knowledge, in 2018 PwC launched its digital accelerator program, where employees apply to become digital learning advocates for the organization. “Their job is to change the way work gets done,” Atkinson said. Over the course of two years, 2,000 employees will be selected to complete an immersive three-day off-site digital training program, then to take that new knowledge back to their teams. Once they return, digital accelerators continue to dedicate 10 percent of their time to continuing digital education, and they are expected to promote digital solutions and support digital learning on their teams. “It is a huge investment to take 2,000 people out of the business for hundreds of hours of training,” McEneaney said. Once employees become digital accelerators, part of their performance review is based on how well they share the knowledge and encourage adoption of new technologies. McEneaney said that 6,000 employees applied to the program and 1,200 have completed the initial training.

Learning to Lead Patricia Miller knew she wanted to be a digital accelerator as soon as the program was announced in January 2018. “I applied later that day,” she said. The application required her to explain why the program was a good fit for her, so she made an animated video depicting a real-life client project where her team figured out how to pull data out of a procurement system that ultimately delivered cost savings for the client. “I figured if I could do that with the minimal tools I had, imagine what I could do with some training,” she said. The PwC team agreed, and she was one of the first 1,200 people to enter the program. “It was amazing to be cocooned with so many like-minded individuals who were so excited to be a part of this organization,” she said. Once the training is over, the accelerators stay connected through social networking and company events, and they rely on each other for advice and encouragement. “If I hit a roadblock, I know these people can help me,” Miller said.

To expand the digital accelerators’ influence, Atkinson’s team created a digital lab, where accelerators (and anyone else) can share automation tools they built to solve a specific business problem, which others can download for their own projects. Employees can also rate the apps and offer feedback. “It lets us see the digital assets that people are using, what’s working and what doesn’t scale,” Atkinson said. PwC pays incentives to developers of automation tools that generate the most business value as a way to encourage innovation.

“Digital upskilling has to be a focus for us as a knowledge organization.” — Sarah McEneaney, digital talent leader, PwC

Top-Down Support All of these training tools and opportunities to share innovation have created a culture of digital transformation at PwC. The Digital Fitness app is boosting the baseline knowledge of the entire workforce, while the academies, badges and digital accelerators are embedding new experts in the organization and incentivizing employees to innovate their workflows. Internal metrics show these efforts paying off. Atkinson reports that some of the best automation tools have been downloaded thousands of times, including a bot that moves data from spreadsheets to reports, saving users 10 to 15 minutes per use. “It’s a small automation, but if 10,000 people save 15 minutes per task, that adds up,” he said. Internal survey results also show a 20 percent uptick in ratings for “opportunity to innovate on the job,” and employment data show digital accelerators are 50 percent less likely to leave the company. Atkinson attributes the success of the upskilling effort to the company’s top-down support for training, combined with giving employees the freedom to leverage the training as they see fit. He encourages other companies not to shy away from making significant investments in digital training, and to trust their people to step up to the challenge. “Digital upskilling is at the core of how our organization will continue to thrive in the face of unprecedented and accelerated disruption,” Atkinson said. “If you don’t invest in your people, you will get left behind.” CLO Sarah Fister Gale is a writer based in Chicago. Chief Learning Officer • October 2019 • 61


One of the biggest challenges L&D leaders face with technology adoption is how to establish and align a technology strategy with business goals. BY ASHLEY ST. JOHN


hief Learning Officer is uniquely situated to see all the new and interesting things organizations are doing with technology in their learning and development functions each year, thanks in large part to our LearningElite and Learning In Practice awards programs. Take, for example, KPMG, one of the LearningElite 2019 top five. In 2018, the company launched its Experience Disruptors and Trends Series, technology-enabled simulations that challenge teams of participants to perform in the role of their clients and navigate a shifting business landscape. AT&T, also in the 2019 LearningElite top five, has prioritized the virtualization of all of its content, a tremendous undertaking. They are more than halfway to that goal, and 100 percent of retail training is completely virtual. At the CLO Symposium on Oct. 14 in Chicago, the winners of the 2019 Learning In Practice Technology Award will be announced. The companies range in size from 1,700 to 1.5 million employees and sport a number of ambitious initiatives: One is utilizing gamification to better engage learners across 48,000-plus stores in 145 countries; another embedded a customized, fully mobile-enabled project management and learning tool into its LMS. In the learning space, organizations large and small are experimenting with technologies and exploring how to best serve their internal and external audiences. However, the aforementioned organizations have been recognized for their programs for a reason: They represent the best of the best in learning and development. What does the general industry snapshot look like for 2019? According to data from the Chief Learning Officer Business Intelligence Board’s “2019 Learning State of the Industry” report, almost 80 percent of companies are making learning technology at least a medium-level investment priority over the next 12 to 18 months, with

62 Chief Learning Officer • October 2019 •

33 percent ranking it as high priority and 16 percent ranking it as essential (Figure 1). L&D strategy and content delivery also ranked high in terms of investment priority. However, when asked what the anticipated greatest L&D challenges were for the next year, 44 percent cited budget (Figure 2). The Chief Learning Officer Business Intelligence Board is a group of 1,500 professionals in the learning and development industry who have agreed to be surveyed by the Human Capital Media Research and Advisory Group, the research and advisory arm of Chief Learning Officer magazine. This survey was conducted from January to March 2019. So, where will those learning tech investments be made? In large part, they will be made in e-learning delivery; analytics, workforce performance metrics, evaluation and dashboards; and competency management (Figure 3). And while 34 percent of organizations are currently not developing or purchasing anything for mobile platforms, the 66 percent who do are concentrating largely on courseware, apps, information resources and on-the-job support (Figure 4). It seems that the most pressing challenge L&D leaders face is how to establish and align a technology strategy with business goals, as well as align technology operations, practices and systems across different business units (Figure 2). After all, as authors Matthew Murray and Nicolai Chen Nielsen discuss in their article “A Strategic Approach to Digital Learning” on page 54, it’s critical to have a clear digital learning production strategy. Without a road map, whatever the investment is, it will be difficult to keep learners satisfied and measure outcomes. CLO Ashley St. John is Chief Learning Officer’s managing editor.

Figures’ sources: Chief Learning Officer Business Intelligence Board’s “2019 Learning State of the Industry,” N=537. All percentages rounded.

Technology Choices Need to Be Strategic


High priority

Medium priority

Low priority

Not a priority


40% 33%



25% 23% 16% 7%






L&D strategy


Learning technology

Content delivery






39% 33% 28%

Align strategy with business goals

Align operations, practices and systems


Attract, engage and Integrate data from retain employees multiple systems

Boost productivity, Mobile technology reduce costs, upgrade service

Respond to changing technologies


40% 34% 29%


27% 23%

E-learning delivery

Analytics/ metrics/evaluations/ dashboards

Competency Learning management management/capability systems development (LMS/L&DMS)

Instructor-led learning delivery

Mobile learning delivery

Social learning tools/platforms


30% 27%


34% 18% 13%

We do not develop or purchase anything for mobile platforms



Information resources/references

On-the-job support



Chief Learning Officer • October 2019 • 63


Learning Technology

STRATEGIC APPROACH continued from page 58 not mutually exclusive or collectively exhaustive. The positioning of LMSs, LXPs and other systems in the center box is to represent that you ideally need an integrating portal/single point of entry to connect the various elements of the learning journey experiences.

Step 2: Determine the Importance Index for Each Tool and Platform Once you have the grid in place, you can determine the relative importance of each tool and platform, based on your organization’s learning priorities. Undertake an assessment of what’s important to your organization’s current and future (next 24 months) learning needs, driven by the skills, behaviors and mindset shifts necessary to achieve your business results. The assessment can range from simple to complex. The more data points and perspectives you incorporate, the stronger the analysis. Remember, this is not an assessment of your current level of use. Instead, this is an objective evalua-

Adapting Your Learning Portfolio 1. Prioritize your research and development agenda — complete a tool and platform canvas for all tools and platforms and use this to guide your actions. R&D should be more than just a perk or a reward for top performers. 2. Leverage analytics and data to determine effective tools that generate ROI. 3. Establish strong working relations with IT. 4. Reconsider budgeting and cost management (procurement, faster budget cycles). 5. Establish proactive vendor relations and management. 6. Promote skills-building and training — identify early adopters/ influencers who can teach others. 7. Identify internal champions and formulate a change communication plan. 8. Build in time to regenerate your content if sunsetting — this needs to be tied to a more modular, product-oriented production model. 9. Learn from product management — life cycle, cost-benefit, portfolio management, core and secondary products. 10. Cultivate the support of sponsors and stakeholders — use the outputs of your analysis to generate buy-in across the business.

— Matthew Murray and Nicolai Chen Nielsen

64 Chief Learning Officer • October 2019 •

tion of which tools and platforms should be used to best support your learning requirements. A simple approach is to assign high/medium/low levels of importance to each tool and platform category (e.g., gold/silver/bronze indicators). A deeper analysis could involve a criteria-based approach to determining relative importance (through a thorough analysis of learning needs across your programs and best fit for purpose). We recommend you factor in a number of considerations: volume and scalability, significance (you might have a niche, high-profile program with specific digital needs), your organization’s learning culture, evaluation data from existing programs, your vision for your future learning approach (e.g., you wish to move to more experiential formats), and other relevant data points. Exercise discipline when assigning an importance index. Larger, more diverse learning environments will likely have more high/medium allocations than smaller, more homogeneous situations. But if you don’t have differentiation across the categories, then you aren’t being rigorous. For instance, some businesses have little need for augmented reality since their workforce stares at a screen all day (holding a phone up in front of a monitor is of limited value when you can integrate informational layers directly on the monitor screen). Some organizations are physically collocated in one place and don’t require virtual classroom deliveries. An example output is provided in Figure 3. It’s important to note: The process of completing this exercise can be very revealing. You can build alignment by running it as a team activity. The exercise can generate powerful dialogue and raise divergences you may not have realized existed. Since it might surface latent disconnects, be on the lookout for defensiveness and encourage participants to keep the conversation data-driven. Suggest that they regard it as an opportunity to clarify and update the team’s value proposition. For instance, it might require your virtual classroom team to more clearly articulate the value of their service from a learning needs perspective. It can be a positive way to break down legacy mindsets (“We’ve always done it this way!”) and keep the focus on where the organization should be helping to drive business success. Involvement in the process ensures buy-in across the team around the ecosystem strategy. Once you have your importance indexes identified, you can then assess the degree of alignment between your desired state and your current state of reality. Closing the gap between the two will become the basis of your action plan. This plan could require you to change the conversa-

tions you have with SMEs and business partners and actively managing your portfolio of tools and platforms. You may need to address redundancy or an overabundance of tools in a lower-​​impact area. You might want to rebalance your ecosystem to better address high-priority areas. See the sidebar, left, for practical tips to adapt your learning portfolio.

The Bottom Line Building digital success is one of the top critical factors in establishing a high-performing learning culture, according to Towards Maturity’s 2019 annual research report, “The Transformation Journey.” A European financial services organization we worked with was able to achieve a 17 percent increase in learner satisfaction and greater than 20 percent reduction in costs by transforming its curriculum to a largely digital, media-enabled learning approach. Undertaking a strategic assessment and putting in place new approaches will enable you to shift the narrative


A digital learning strategy establishes the criteria and rationale for the adoption of new technologies.

Clifford Capone Vice President, Group Publisher 312-967-3538 Daniella Weinberg Regional Sales Manager 917-627-1125 CT, MA, MD, ME, NH, NJ, NY, PA, RI, VT, Quebec, New Brunswick, Newfoundland, Europe

around your digital learning production and delivery. Doing so can not only reduce L&D expenditures and improve efficiency, but have tangible impact on the learning experience, degree of learning and business impact. CLO

Kevin M. Fields Director, Business Development 312-967-3565 Melanie Lee Business Administration Manager

Matthew Murray is a learning architect at McKinsey & Cos.’ McKinsey Academy. Nicolai Chen Nielsen is an associate partner at McKinsey Academy.


Advertisers/URLs Page

AACSB Bellevue University BetterUp Capella University CLO fall symposium CLO Webinars D2L Darden DDI DeVryWORKS GP Strategies HCM Research and Advisory Group

International Coach Federation Intrepid Kaplan Leadership Pipeline Institute NIIT NovoEd Pearson PennCLO Purdue University Global Saba Skillsoft Talent Tracker

36-37 13, 32-33 21 23 29 7 19 34-35 27 5 3 51

Advertising: For advertising information, write to Back Issues: For all requests, including bulk issue orders, visit our website at or email Editorial: To submit an article for publication, go to Letters to the editor may be sent to

30-31 45 53 6 15 57 17 2nd Cover 59 47 Back Cover 3rd Cover

Permissions and Article Reprints: No part of Chief Learning Officer can be reproduced without written permission. All permissions to republish or distribute content from Chief Learning Officer can be obtained through PARS International. For single article reprints in quantities of 250 and above and e-prints for Web posting, please contact PARS International at

List Rental: Contact Mike Rovello at 402-836-5639 or

Subscription Services: All orders, inquiries and address changes should be addressed to

Computer Fulfillment PO Box 8712 Lowell, MA 01853

or call customer service at 800-422-2681 or 978-671-0446 or email

Chief Learning Officer • October 2019 • 65


EQ and Reskilling in the Age of AI Tying reskilling to career development • BY CAROLINE STOKES


Caroline Stokes is founder of Forward, an executive headhunting and executive coaching company for global innovation leaders, and author of “Elephants Before Unicorns: Emotionally Intelligent HR Strategies to Save Your Company” (Entrepreneur Press, 2019).

ith decades of artificial intelligence scaling ahead of us, technology is evolving faster than ever before. That means it’s more important than ever for people leaders to continually provide skills-based training so team members may gain comfort with the technology, giving their organizations an edge in the marketplace. There are two key components you need to think about to successfully reskill your workforce. First, carefully consider where your organization needs to go and which technology would best aid your progress. Second, get buy-in from your teams. Before you decide to reskill your team, let’s examine the simple yet crucial first step of considering where you want your organization to go. From there, you can determine whether you actually need to implement a new technology or process. If the answer is yes, you may then develop a plan to reskill whichever teams most need to adapt. Leaders who take this important first step deploy strategic thinking and appreciative inquiry to where their organization needs to go and how to get it there. Appreciative inquiry is a concept that was developed in the 1980s by David L. Cooperrider and Suresh Srivastva. People leaders use appreciative inquiry for organizational improvement. Rather than focus on “problems” that need to be solved, they begin by appreciating the resources already available within the cohort. Then, they examine how these resources may be engaged for positive change. This framework is tremendously helpful for leaders seeking to determine which technologies and human systems would best evolve their organizations. Rather than thinking “Oh no, we’ve got to change everything or we’re going to be left behind by our competitors!”, appreciative inquiry invites leaders to first consider what their companies are doing right. The focus is on moving from good to better. Leaders considering implementing organizational change with the appreciative inquiry model might ask questions along these lines: What is working well? Where do we need to go? Which services are we currently providing that are likely to be automated soon? How can we reskill the workers performing those job functions into other areas that will provide added value to the company? This approach values everyone in your organization and celebrates the successes you and your teams have

66 Chief Learning Officer • October 2019 •

achieved, while looking ahead and imagining how the future can be even better. The second component of reskilling your employees is getting buy-in from your teams. Any time you ask employees to change the way they operate, you open the door for significant pushback. If teams are constantly asked to change the way they do things because of the C-suite’s whims, they will grow very frustrated. Company leadership must agree on the value of the intended change so employees will have more confidence in it.

Appreciative inquiry invites leaders to first consider what their companies are doing right. Rarely do I see companies consider reskilling as part of career development. If team members can see how what they’re being asked to learn relates to the overall trajectory of their careers and the mission of the organization, they’re more likely to perceive value in it. The C-suite must determine how urgent the needed change is. Does the entire sales force need to be reskilled tomorrow? If the need is not extremely urgent, consider having employees opt in; you can incentivize the reskilling process as you wish. Once individuals begin to enjoy success, you can present them to your teams as case studies so that other employees are motivated to gain the new skill. Recognize that many of your employees are likely fearful about the advancement of AI and the rapid rate of technological evolution. This is where the onus falls on you, a leader who cares directly about your employees, to tie the reskilling process to team members’ professional development. Put yourself in your direct reports’ shoes. If someone were to tell you that you need to change the way you do your job, how would you like that information communicated? What assurances would you need? Such questions will guide you well when you find yourself in this delicate position. There is never a reason to abandon your humanity. CLO

Collected, organized and tailored to your needs. WORKFORCE DATA FOR YOUR ORGANIZATION


Leading mission-driven teams requires the ability to lead through influence and motivation, rather than authoritarian fiat. The Skillsoft Leadership Development Program addresses the changing way in which organizations need to develop their leaders by providing a scalable, cost-effective, and engaging leadership development experience designed for the modern, digital learner. Please attend our CLO Symposium sessions below:

Six Truths of Modern Leadership Development: How to Drive Change Today to Prepare Leaders for the Digital Economy

Heide Abelli Senior Vice President, Content Product Management, Skillsoft

October 15th, 2019, 11 A.M., Vevey 1

LevelUp: Leadership Development in the Digital Age Julie Curtin Director, Learning and Performance Innovations, BorgWarner

October 14th, 2019, 3:00 P.M., Vevey 2

Profile for Chief Learning Officer

Chief Learning Officer - October 2019  

Guardian of the Coast: CLO Gladys Brignoni brings private-sector insights to the United States’ fifth military branch.

Chief Learning Officer - October 2019  

Guardian of the Coast: CLO Gladys Brignoni brings private-sector insights to the United States’ fifth military branch.