CSM SUCCESS PROGRAM Project Proposal
Project Sponsor:
Blake Adams
Project Manager:
Michael Sanford
Instructional Designer:
Michael Sanford
Subject Matter Expert:
Blake Adams
Client:
Appointmently, Inc.
Business Line:
Technology / Scheduling Services / CRM
Business Impact Priority:
Reduce Churn
© Michael L. Sanford, II 2023 Please be aware that any product or corporate names mentioned in this document are fictitious and are used solely for identification and explanation purposes without any intent to infringe upon the copyright or trademarks of entities whose business operations may be similar to the imaginary client presented herein. The Program Content in this document is designed to help you improve your talent development skills and is intended for personal use only. Please note that the Program Content may be updated or modified without prior notice. It is important to understand that you cannot copy, share, or use the Program Content to train others. However, once you have completed the course, you are welcome to share your insights with your peers, refer to specific models or concepts that you learned in the course (as long as you do not use them for training purposes), and review the Program Content to reinforce your learning. It is important to note that the Program Content is provided "as is" and does not constitute professional advice or endorsement of any views or opinions expressed therein. You are solely responsible for using the Program Content, and any consequences that may arise.
Project Proposal Not for external use.
Program: CSM Success
Version 2.1 Date: 07.08.2023 -2-
Contents Initial Proposal: CSM Success ............................................................................................................. - 5 1. Needs Assessment ............................................................................................................................ - 6 1.1 Needs Statement: ........................................................................................................................ - 6 Current State: .................................................................................................................................................... - 6 Desired State: .................................................................................................................................................... - 7 -
1.2 Purpose Statement:..................................................................................................................... - 7 1.3 Target Group: ............................................................................................................................... - 7 1.4 General Description of the Instructional Product: ................................................................... - 8 2. Learner and Context Analysis .......................................................................................................... - 9 2.1 Learner Analysis .......................................................................................................................... - 9 2.2 Application Context...................................................................................................................... - 9 2.3 Learning Context ....................................................................................................................... - 10 3. Outcomes Analysis .......................................................................................................................... - 12 3.1 Goal Description ........................................................................................................................ - 12 3.2 Goal Statement .......................................................................................................................... - 12 3.3
Subskills Analysis Flowchart ............................................................................................. - 13 -
4. Performance Objectives and Assessment Items ........................................................................ - 14 4.1 Terminal Objective ..................................................................................................................... - 14 4.2 Enabling Objectives................................................................................................................... - 14 4.3 Objective – Activity – Assessment Alignment ........................................................................ - 15 4.4 Assessment Development ........................................................................................................ - 18 4.5 Assessment Tool ........................................................................................................................ - 21 Call Simulation Scorecard ............................................................................................................................. - 21 -
5. Instructional Strategy....................................................................................................................... - 22 5.1 Course Overview ....................................................................................................................... - 22 5.2 Activity Map................................................................................................................................. - 23 6. Product Prototype ............................................................................................................................ - 25 7. Formative and Summative Evaluation .......................................................................................... - 25 7.1 Expert Reviews (Kirpatrick Level 1: Reaction) ...................................................................... - 25 7.2 One-to-One Evaluation (Kirpatrick Level 1: Reaction) ......................................................... - 25 7.3 Small Group Evaluation (Kirkpatrick Level 1: Reaction)...................................................... - 25 7.4 Field Test (Kirpatrick Level 2: Learning) ................................................................................. - 26 Project Proposal Not for external use.
Program: CSM Success
Version 2.1 Date: 07.08.2023 -3-
7.5 Summative Evaluation (Kirpatrick Levels 3 & 4: Behavior and Results) ........................... - 26 Appendix ................................................................................................................................................. - 27 1.
Kahoot! Quiz Prototype ............................................................................................................. - 27 -
2.
Appointmently Call Simulation Scorecard.................................................................................. - 28 -
3.
Appointmently eLearning Module Prototype / Wireframe ....................................................... - 29 -
Document Revision Log........................................................................................................................... - 30 -
Project Proposal Not for external use.
Program: CSM Success
Version 2.1 Date: 07.08.2023 -4-
Initial Proposal: CSM Success A client with a smaller customer success department is reporting that newer customer success agents are not ramping up properly. These agents have lower-than-normal customer retention rates, and their customers are not taking advantage of their platform in ways that they could. This leads to a net loss in revenues (ARR) and lower customer satisfaction across the organization. Much of this issue is due to the customer success managers not knowing the product they are selling and supporting. To resolve this the enablement team will create a training program consisting of: • • •
One-on-one coaching to teach customer success managers (CSMs) more about the product and customers they serve eLearning within an LMS (learning management system) to teach the CSMs about the product and different customer-types (and how those customers use their product) "Cookbooks" or PDFs with at-a-glance information that CSMs can use and distribute to customers to learn more about how the product could work for them
Target learners include customer success agents that have worked at this company for less than 6 months. These tend to be men and women who are educated and technologically literate in their 20s-40s. The purpose of the project is to empower customer success agents with the information they can use to empower their customers and enable them to use the tools the product provides.
Project Proposal Not for external use.
Program: CSM Success
Version 2.1 Date: 07.08.2023 -5-
1. Needs Assessment 1.1 Needs Statement: The management team of Appointmently has published a request for proposals to address shortcomings identified in their customer success department among their newer Customer Success Managers (CSMs). Bill Morton, the Appointmently Director of People Operations and sponsor of the project, has identified the following key performance indicators as failing to meet expectations: lower than typical customer retention rates, limited utilization of the Appointmently product offerings among retained customers, net loss in revenues, and lower customer satisfaction scores for CSMs. Upon review of customer satisfaction surveys, customer interviews, CSM shadowing, CSM interviews, and CSM Manager interviews, we found other impacts related to these key performance indicators, including lower-than-expected customer satisfaction surveys for different departments within Appointmently. Based upon these observations and reviews of available data, we have identified a key contributing factor to these metrics is related to a lack of knowledge of the products offered by Appointmently among the CSM staff. Consequently, the CSMs are limited in their ability to satisfy customer needs during calls for assistance with the product. Furthermore, the CSMs have limited ability to cross-sell/promote other product features during interactions with customers. As the front-line employees interacting with customers at the moment of greatest frustration or difficulty in realizing the benefits of the product, this lack of knowledge as to how to help the client or how to promote other features of the product to enhance the user experience beyond simply resolving the immediate concern has impacted customer satisfaction metrics across the enterprise due to the central role of the CSM in addressing customer needs. We believe a training intervention will address these needs if it can achieve the following: • Provide CSMs with a comprehensive understanding of the product offering and all functions the product is capable of. • Provide CSMs with coaching/training to better address customer needs and identify cross-selling opportunities within service calls involving direct client interaction.
Current State: CSMs have a limited understanding of the Appointmently product offerings available. CSMs fail to create opportunities to enhance the client experience by promoting product features that would benefit the customers they interact with during service calls.
Project Proposal Not for external use.
Program: CSM Success
Version 2.1 Date: 07.08.2023 -6-
Desired State: CSMs will have robust training on the full spectrum of Appointmently product capabilities during onboarding. They will have access to comprehensive job aids with detailed information on the capabilities and features of the entire suite of product offerings from Appointmently. CSMs will have coaching and ongoing professional development training on navigating challenging conversations with customers and how to make those conversations opportunities to recommend other product features not being utilized by the client.
1.2 Purpose Statement: The purpose of the Customer Success Intervention will be to provide the CSM team at Appointmently with comprehensive onboarding training as to the full spectrum of capabilities of their product offering, as well as job aids to further augment their performance in an ongoing manner after the initial training process is complete. In addition to product knowledge development, the CSMs will receive scenario-based training on cross/up-selling techniques to maximize the benefit of any interactions they may have with customers during service or help desk calls. Finally, managers for the CSM team will also receive specific training related to coaching the CSMs to reinforce the learning intervention and align key performance indicators identified within the needs assessment with the key performance indicators the CSMs are being evaluated upon in their reviews with their managers.
This will provide the Learners with greater clarity about their role expectations, the tools to meet those expectations, and alignment of expectations and the key performance indicators that will be the basis of their reviews with management.
The intervention will, in turn, provide Appointmently a more capable team of CSMs, improved customer retention rates, stronger revenue performance, and higher customer satisfaction ratings across the enterprise due to improved service at the point of contact with clients.
1.3 Target Group: The intended learners in the intervention are the Customer Success Managers. A secondary learner group will be the managers the CSMs report to for the intervention's coaching and KPI alignment portions.
Project Proposal Not for external use.
Program: CSM Success
Version 2.1 Date: 07.08.2023 -7-
Target learners include customer success agents that have worked at this company for less than 6 months. These tend to be men and women who are educated and technologically literate in their 20s-40s.
1.4 General Description of the Instructional Product: There are three main components to the overall intervention we have presented to the client for this proposal:
•
eLearning Modules within Appointmently's existing learning management system built utilizing Adobe Captivate. The digital training modules will include a thorough introduction to the client's product offering, the different types of clients the company serves with their products, and the other use cases for their product. Each module will begin with a pre-module assessment, followed by interactive slides with short videos, infographics, and audio narration of the content. Closed captioning and supplemental audio descriptions of imagery will be available for the hearing and vision impaired. The curriculum is designed for asynchronous delivery, but with weekly due dates corresponding to facilitated virtual office hours sessions where the learners participate in retrieval exercises utilizing Kahoot to add social and gaming elements as an ice breaker activity and then guided discussions about topics from the week's modules. These virtual sessions will take place in Microsoft Teams Meeting rooms. The final eLearning component will be Teams messages from a chatbot loaded with retrieval exercises and nudges to quiz Learners about the material covered in the initial eLearning modules to keep their knowledge current with the latest developments and to refresh their recall of the original material on a semiweekly basis.
•
Job aids in the form of "Cookbooks" or PDFs with at-a-glance information that CSMs can use during calls with customers, with product features, how-to instructions, FAQs, and scripting suggestions for how to present the features to customers. These will also include pdf pamphlets for circulation to customers with corresponding information to the versions provided for the CSMs.
•
Finally, in addition to the interventions described above, we strongly advise the client to contract for the development of a manager training program in support of this intervention for the CSMs to provide ongoing support in line with the objectives and criteria established in the new intervention. The supervising managers for the CSM team should receive specific training related to coaching the CSMs to reinforce the learning intervention. The annual review process, job descriptions, and basic operations procedures of the CSMs will be adjusted to align with the key performance indicators identified in the needs analysis, interviews, and surveys of the stakeholders, CSMs, and their managers. This additional step will align the feedback they receive in annual reviews, manager expectations, and all other aspects of the department to reflect the CSM’s context of how they should be utilizing the information included in the training modules and job aids for consistency and to ensure that there are no conflicting imperatives for the CSMs to consider in their performance or expectations.
Project Proposal Not for external use.
Program: CSM Success
Version 2.1 Date: 07.08.2023 -8-
2. Learner and Context Analysis 2.1 Learner Analysis The following table provides a summary of the key learner characteristics and their implications on the learning design for the target population of customer success agents at a tech firm.
The information takes into account their prior knowledge and experience, motivations, and cognitive and physical abilities. The table provides a concise overview of the characteristics that need to be considered when designing a learning initiative for this group of learners. This information will help inform the development of a tailored and effective learning experience that addresses the needs and goals of the learners.
Learner Characteristics Prior Knowledge/Experience
(Data Sources) Description Customer success agents at a tech firm with roughly 6 months of experience. Some have bachelor's degree; some have graduate degrees. (Hiring Data and Company-facing Org charts)
Motivation
Want to be more successful in empowering customers with the app, meet their metrics, and advance their careers with raises and promotions. (Data from metrics and company surveys) Due to ADA and privacy constraints– unable to collect information from company registry. (NA)
Cognitive/Physical Abilities
Implications on Design Learners have some prior knowledge and experience, but need further training to support customers effectively. This means – product education with a bit more training and experience with the product itself. The design should focus on hands-on learning and practice to help learners build skills and meet their goals.
Will need to consider possible cognitive/phisical abilities in broad strokes when creating training materials.
2.2 Application Context The table summarizes the context characteristics of the learners in a learning design initiative. It includes information about the work environment, physical setup, tools and resources, requirements and constraints, cultural context, organizational context, and technology context.
Project Proposal Not for external use.
Program: CSM Success
Version 2.1 Date: 07.08.2023 -9-
The description column provides the data sources for each context characteristic, and the implications on design column highlights how each context characteristic may impact the design of the learning initiative. This information can be used to ensure that the learning design takes into consideration the context in which the learners will be performing skills and knowledge once they have acquired them.
Context Characteristics Work Environment
(Data Sources) Description Remote work, primarily from home office, with access to shared office space (WeWork) (Data from Company Policy)
Physical Setup
Laptop, keyboard, mouse, monitor provided, typical home office environment (Data from Company Policy)
Tools and Resources
LMS (Lessonly), Knowledge Management (Guru), Salesforce, Gong, Zoom, Slack, Loom (Data from Company Policy) Internet connection, provided tools, potential distractions (Data from Company Policy) Western society, primarily United States based (Data from Company History and Demographic Data) Customer-facing, managed by a team of 20 (Data from Company Policy) MacOS (Data from Company Policy)
Requirements and Constraints Cultural Context
Organizational Context Technology Context
Implications on Design Provide flexibility in design for remote work and potential distractions at home. Ensure tools and resources are accessible for remote workers. Ensure compatibility of design with provided hardware and consider potential physical constraints in the home office setting. Ensure design is compatible with and leverages available tools and resources. Address internet connectivity concerns and potential distractions in design. Consider cultural context and potential cultural biases in design. Consider team dynamics and collaboration in design. Ensure design is compatible with MacOS technology.
2.3 Learning Context The table provides information about the learning context for customer success agents at a tech firm. The table includes three columns: "Learning Context Characteristics", "Description (Data Sources)", and "Implications on Design".
Project Proposal Not for external use.
Program: CSM Success
Version 2.1 Date: 07.08.2023 - 10 -
The rows in the table describe the environment in which the learners will attempt to achieve their skills and knowledge, including their remote work setup, the tools provided to them, the requirements and constraints they face, and the cultural context they are a part of. The data sources for the information in the table are the company's policies and practices, as well as the learners' experience. The implications on design highlight how the characteristics of the learning context will influence the design of the learning program, including considerations for remote work setup, access to tools, and cultural context. Context Characteristics Physical Environment
Tools and Technology
Organizational and Social Context
Technical Environment
Project Proposal Not for external use.
(Data Sources) Description Learners work remotely and in most cases, their work environment is at home in a home office. They are provided with a laptop (Mac), keyboard, mouse, and monitor. They also have access to a shared office space called "WeWork" if needed. (Data from Company Policy) Learners are provided with the necessary tools such as LMS (Lessonly), Knowledge Management tool (Guru), work software (Salesforce and Gong), and communication tools (Zoom, Slack, and Loom). The only requirement is an internet connection and access to these tools. (Data from Company Policy) Learners are part of a larger team of 20 customer-facing employees, managed by a team. The cultural context is typically Western society United States-based culture. (Data from Company History and Demographic Data) Learners use MacOS and are empowered to use any other tools they need to help customers. (Data from Company Policy)
Program: CSM Success
Implications on Design The design needs to consider the potential distractions and lack of structure that might come with working from home. Providing flexible options such as access to a shared office space can help learners overcome these distractions. The design needs to ensure that learners have access to and are proficient in using the tools they need to perform their job and complete their training.
The design needs to consider the impact of the learners' team and the cultural context on their learning experience and performance. This can include incorporating teambased learning activities or addressing cultural differences in communication styles. The design needs to consider the technical requirements and limitations of the learners' devices and operating systems. This can include ensuring compatibility with MacOS and providing technical support as needed. Version 2.1 Date: 07.08.2023 - 11 -
3. Outcomes Analysis 3.1 Goal Description The Learners for this intervention are Customer Success Managers with Appointmently who have been with the company for less than six months and work remotely from their homes or coworking spaces when available. Because the CSMs are not working in an office together, there are challenges to their ability to learn by observing senior members of their team, and this intervention is intended to overcome this lack of informal learning opportunities that would otherwise enable the learners to learn by observing others how to perform their role efficiently and effectively. The target group are college-educated men and women in their 20s to 40s with a background in the technology sector. The learners will have access to Slack (team chat-based communication utility), Salesforce (cloud-based customer service management system), Guru (cloud-based knowledge management system), and the intervention training materials as tools to assist them in meeting expectations after the program. The key performance indicators that the program has been successful will include improved customer retention rates, higher utilization of product offerings, increased revenues, and higher customer satisfaction ratings. Specifically, after completing this program, learners should be able to: • Identify use cases for Appointmently features and instruct clients on how to improve their operations by implementing said features in their workflows efficiently. • Explain the capabilities, functions and best practices of the Appointmently product/platform. • Anticipate and manage customer needs during calls for assistance with the product. • Create cross-selling opportunities within service calls involving direct customer interaction. • Coordinate with clients to share how other product features can enhance the customer experience beyond resolving their immediate concern during service calls. • Identify when to utilize the available resources designed for in the moment of need information such as “cookbooks” and PDFs with at-a-glance information that CSMs can use and distribute to customers to learn more about how the product could work for them.
3.2 Goal Statement By the end of the instructional program “CSM Success,” CSMs at Appointmently will be able to coach clients with any feature of the product in a manner that is efficient, effective in addressing the client’s challenges, and convert a mere user into a fan of the services provided by the company. Drawing on their technical training, the knowledge Project Proposal Not for external use.
Program: CSM Success
Version 2.1 Date: 07.08.2023 - 12 -
gained during the program, and the resources created for in the moment of need, Slack and Salesforce – learners will provide best-in-class service by empowering the clients with the platform specific skills and knowledge they will have obtained through this intervention. This is a cognitive skill domain goal involving knowledge acquisition and application.
3.3
Subskills Analysis Flowchart
Project Proposal Not for external use.
Program: CSM Success
Version 2.1 Date: 07.08.2023 - 13 -
4. Performance Objectives and Assessment Items 4.1 Terminal Objective CSMs will identify the opportunities to facilitate learning for their clients in the range, use, and benefits of the Appointmently platform for improving their personal and institutional efficiency and effectiveness while working with clients on service calls.
4.2 Enabling Objectives Outcomes 1.0 Identify product features, client needs, and use cases where product features can most effectively and efficiently address those needs using job aids and other knowledge artifacts provided during the program. 1.1 Identify product features and client use cases.
1.1.1. Identify product features. 1.1.2 Identify client use cases.
1.1.2.1 Demonstrate comprehension of client use cases for sales teams. 1.1.2.2 Demonstrate comprehension of client use cases for recruiting teams. 1.1.2.3 Demonstrate comprehension of client use cases for marketing teams. 1.1.2.4 Demonstrate comprehension of client use cases for information technology teams.
Project Proposal Not for external use.
Objectives 1.0. During service calls, learners will identify the relevant platform feature related to the inquiry, the use case presented by the client and provide relevant technical support, recommend at least one additional feature of the platform that would further improve the client experience. 1.1. During service calls, CSMs will identify based on the context of the conversation which product features the client is referring to and the relevant technical functions, use cases, and best practices for the Appointmently platform. 1.1.1. During service calls, CSMs will identify the client’s description of the product feature they need assistance with. 1.1.2. During service calls, CSMs will identify the client’s description of their use case for the Appointmently platform which they need assistance with. 1.1.2.1. During service calls, CSMs will be able to recommend product features and how they function for users working in sales roles or leading sales teams. 1.1.2.2. During service calls, CSMs will be able to recommend product features and how they function for users working in recruiting roles or leading such teams. 1.1.2.3. During service calls, CSMs will be able to recommend product features and how they function for users working in marketing roles or leading such teams. 1.1.2.4. During service calls, CSMs will be able to recommend product features and how
Program: CSM Success
Version 2.1 Date: 07.08.2023 - 14 -
Outcomes 1.1.2.5 Demonstrate comprehension of client use cases for customer success teams. 1.1.2.6 Demonstrate comprehension of client use cases for non-profit teams. 1.2 Determine efficient and effective technical solutions, and coach the client through their application to the problem. 1.2.1. Determine efficient and effective technical solutions for client problems. 1.2.1.1. Demonstrate comprehension of troubleshooting solutions for the product/platform. 1.2.2. Communicate the instructions and best practices for client problems.
Objectives they function for users working in information technology roles or leading such teams. 1.1.2.5. During service calls, CSMs will be able to recommend product features and how they function for users working in customer success roles or leading such teams. 1.1.2.6. During service calls, CSMs will be able to recommend product features and how they function for users working in non-profit organizations or leading such teams. 1.2. During service calls, CSMs will identify the most efficient and effective solution to the obstacle reported by the client, and appropriately communicate how to apply that solution to said obstacle. 1.2.1. During service calls, CSMs will identify the most efficient and effective technical solution to an obstacle. 1.2.1.1. During service calls, CSMs will identify the settings, features and related interactions that would result in the fifteen most common causes of support calls. 1.2.2. During service calls, CSMs will concisely communicate the technical instructions to address the client issue and provide appropriate additional resources to support their understanding and best practices for use of the platform.
4.3 Objective – Activity – Assessment Alignment Objectives 1.0. During service calls, learners will identify the relevant platform feature related to the inquiry, the use case presented by the client and provide relevant technical support, recommend at least one additional feature of the platform that would further improve the client experience.
Activity 1.0 Working in groups of three, the learners will take turns reading through a case scenario based on a recorded CSM call. One Learner will simply observe and take notes to provide feedback, one will participate in the CSM role during the simulated call and the third will act as the client placing the call.
1.1. During service calls, CSMs will identify - based
1.1 Working in an eLearning Module, while listening to CSM calls from actual clients, the
Project Proposal Not for external use.
Program: CSM Success
Assessment Call Simulation Scorecard: was the product feature identified, was the use case identified, was the proposed solution appropriate and was one additional product feature recommended. eLearning Module with
Version 2.1 Date: 07.08.2023 - 15 -
Objectives on the context of the conversation - which product features the client is referring to and the relevant technical functions, use cases, and best practices for the Appointmently platform.
Activity learner will indicate in the module which product feature is being described, the description of the technical functions of the feature, the client use case referred to in the call, and what the best practices are for that combination of feature and use case.
1.1.1. During service calls, CSMs will identify the client’s description of the product feature they need assistance with.
1.1.1. During facilitated training sessions, learners will participate in a Kahoot retrieval exercise where various product feature descriptions will be presented as quiz questions, and learners will “buzz” in using the Kahoot app on their mobile device to provide their answer as to which feature they believe the description is referring to. 1.1.2. During facilitated training sessions, learners will participate in a Kahoot retrieval exercise where various client use cases will be presented as quiz questions, and learners will “buzz” in using the Kahoot app on their mobile device to provide their answer as to which use case they believe the description is referring to. 1.1.2.1 After completing an LMS hosted eLearning module that simulates the product features and their uses for sales teams, the learner must successfully identify 15 out of 15 features and use cases for sales teams during a quiz built into said module.
1.1.2. During service calls, CSMs will identify the client’s description of their use case for the Appointmently platform which they need assistance with. 1.1.2.1. During service calls, CSMs will be able to recommend product features and how they function for users working in sales roles or leading sales teams. 1.1.2.2. During service calls, CSMs will be able to recommend product features and how they function for users working in recruiting roles or leading such teams. 1.1.2.3. During service calls, CSMs will be able to recommend product features and how they function for users working in marketing roles or leading such teams. 1.1.2.4. During service calls, CSMs will be able to recommend product features and how they Project Proposal Not for external use.
Assessment recordings of service calls and multiple-choice questions to demonstrate understanding of objectives presented in recordings. Kahoot Quiz / Retrieval Exercise
Kahoot Quiz / Retrieval Exercise
Fill-in-the-blank Quiz/Assessment within eLearning Module, scored and tracked by LMS.
1.1.2.2 After completing an LMS hosted eLearning module that simulates the product features and their uses for recruiting teams, the learner must successfully identify 15 out of 15 features and use cases for recruiting teams during a quiz built into said module.
Fill-in-the-blank Quiz/Assessment within eLearning Module, scored and tracked by LMS.
1.1.2.3 After completing an LMS hosted eLearning module that simulates the product features and their uses for marketing teams, the learner must successfully identify 15 out of 15 features and use cases for marketing teams during a quiz built into said module.
Fill-in-the-blank Quiz/Assessment within eLearning Module, scored and tracked by LMS.
1.1.2.4 After completing an LMS hosted eLearning module that simulates the product features and their uses for information technology teams, the learner must successfully
Fill-in-the-blank Quiz/Assessment within eLearning Module, scored
Program: CSM Success
Version 2.1 Date: 07.08.2023 - 16 -
Objectives function for users working in information technology roles or leading such teams. 1.1.2.5. During service calls, CSMs will be able to recommend product features and how they function for users working in customer success roles or leading such teams. 1.1.2.6 During service calls, CSMs will be able to recommend product features and how they function for users working in non-profit organizations or leading them. 1.2. During service calls, CSMs will identify the most efficient and effective solution to the obstacle reported by the client, and appropriately communicate how to apply that solution to said obstacle.
Activity identify 15 out of 15 features and use cases for information technology teams during a quiz built into said module.
Assessment and tracked by LMS.
1.1.2.5 After completing an LMS hosted eLearning module that simulates the product features and their uses for customer success teams, the learner must successfully identify 15 out of 15 features and use cases for customer success teams during a quiz built into said module. 1.1.2.6 After completing an LMS hosted eLearning module that simulates the product features and their uses for non-profit organizations, the learner must successfully identify 15 out of 15 features and use cases for non-profit organizations during a quiz built into said module. 1.2 Working in groups of three, the learners will take turns reading through a case scenario based on a recorded CSM call. One Learner will simply observe and take notes to provide feedback, one will participate in the CSM role during the simulated call and the third will act as the client placing the call.
Fill-in-the-blank Quiz/Assessment within eLearning Module, scored and tracked by LMS.
1.2.1. During service calls, CSMs will identify the most efficient and effective technical solution to an obstacle.
1.2.1. Working in an eLearning module that simulates the Appointmently platform, the learner will be prompted with different goals and limitations based on client use cases of the platform and work through the most efficient and effective solution to the scenario. After completing each scenario a brief video will provide feedback via a demonstration of the most efficient and effective technical solution to the scenario. 1.2.1.1. Working in an eLearning module that simulates the Appointmently platform, the learner will be prompted with different goals and limitations based on client use cases of the platform and identify the cause of the simulated support call in the scenario. After completing each simulation a brief video will provide feedback via a demonstration of the settings, features and related interactions that would cause the issue from the simulation support call, including how to resolve the issue.
1.2.1.1. During service calls, CSMs will identify the settings, features and related interactions that would result in the fifteen most common causes of support calls.
Project Proposal Not for external use.
Program: CSM Success
Fill-in-the-blank Quiz/Assessment within eLearning Module, scored and tracked by LMS. Call Simulation Scorecard: was the proposed solution the most efficient and effective, and was the solution appropriately communicated to the client. eLearning Module with platform simulation and immediate video feedback of the correct solution to the scenario. eLearning Module with platform simulation and immediate video feedback of the correct solution to the scenario.
Version 2.1 Date: 07.08.2023 - 17 -
Objectives 1.2.2. During service calls, CSMs will concisely communicate the technical instructions to address the client issue and provide appropriate additional resources to support their understanding and best practices for use of the platform.
Activity 1.2.2. Working in groups of three, the learners will take turns reading through a case scenario based on a recorded CSM call. One Learner will simply observe and take notes to provide feedback, one will participate in the CSM role, and the third will act as the client placing the call.
Assessment Call Simulation Scorecard: was the proposed solution the most efficient and effective, was the solution appropriately communicated to the client, and were additional resources provided to share best practices with the client to support ongoing use of the platform.
4.4 Assessment Development Objectives 1.0. During service calls, learners will identify the relevant platform feature related to the inquiry, the use case presented by the client and provide relevant technical support, recommend at least one additional feature of the platform that would further improve the client experience. 1.1. During service calls, CSMs will identify - based on the context of the conversation - which product features the client is referring to and the relevant technical functions, use cases, and best practices for the Appointmently platform. 1.1.1. During service calls, CSMs will identify the client’s description of the Project Proposal Not for external use.
Assessment Item Call Simulation Scorecard with blanks for identifying the product feature, use case or other learning objective presented in the case presented to the learner, the corresponding response from the learner as to the which product feature, use case, etc. was presented and a final blank for providing feedback to the learner in general for that case scenario.
Scoring Peer evaluated and with direct, immediate feedback as to presentation, accuracy, efficiency, and effectiveness.
eLearning Quiz with video and audio files of customer calls wherein the learner will identify product features referred to in the files, and answer additional questions about the related technical functions, use cases, and best practices of relevance to a customer expressing the information included in the example case.
eLearning Module, tracked by LMS and scored by LMS.
Kahoot! Quiz Game for retrieval practice regarding product features.
Kahoot platform tracks and reports to
Program: CSM Success
Version 2.1 Date: 07.08.2023 - 18 -
Objectives product feature they need assistance with.
Assessment Item
1.1.2. During service calls, CSMs will identify the client’s description of their use case for the Appointmently platform which they need assistance with.
Kahoot! Quiz Game for retrieval practice regarding product use cases.
1.1.2.1. During service calls, CSMs will be able to recommend product features and how they function for users working in sales roles or leading sales teams. 1.1.2.2. During service calls, CSMs will be able to recommend product features and how they function for users working in recruiting roles or leading such teams. 1.1.2.3. During service calls, CSMs will be able to recommend product features and how they function for users working in marketing roles or leading such teams. 1.1.2.4. During service calls, CSMs will be able to recommend product features and how they function for users working in information technology roles or leading such teams. 1.1.2.5. During service calls, CSMs will be able to recommend product features and how they function for users working in customer success roles or leading such teams.
Fill in the blank eLearning Module/Quiz requiring identification of 15 relevant product features and their function that are of relevance to clients working in sales roles.
Scoring facilitator the responses of all learners to the questions asked during the quiz. Kahoot platform tracks and reports to facilitator the responses of all learners to the questions asked during the quiz. eLearning Module, tracked by LMS and scored by LMS.
Fill in the blank eLearning Module/Quiz requiring identification of 15 relevant product features and their function that are of relevance to clients working in recruiting roles.
eLearning Module, tracked by LMS and scored by LMS.
Fill in the blank eLearning Module/Quiz requiring identification of 15 relevant product features and their function that are of relevance to clients working in marketing roles.
eLearning Module, tracked by LMS and scored by LMS.
Fill in the blank eLearning Module/Quiz requiring identification of 15 relevant product features and their function that are of relevance to clients working in information technology roles.
eLearning Module, tracked by LMS and scored by LMS.
Fill in the blank eLearning Module/Quiz requiring identification of 15 relevant product features and their function that are of relevance to clients working in customer success roles.
eLearning Module, tracked by LMS and scored by LMS.
Project Proposal Not for external use.
Program: CSM Success
Version 2.1 Date: 07.08.2023 - 19 -
Objectives 1.1.2.6 During service calls, CSMs will be able to recommend product features and how they function for users working in non-profit organizations or leading them. 1.2. During service calls, CSMs will identify the most efficient and effective solution to the obstacle reported by the client, and appropriately communicate how to apply that solution to said obstacle. 1.2.1. During service calls, CSMs will identify the most efficient and effective technical solution to an obstacle. 1.2.1.1. During service calls, CSMs will identify the settings, features and related interactions that would result in the fifteen most common causes of support calls. 1.2.2. During service calls, CSMs will concisely communicate the technical instructions to address the client issue and provide appropriate additional resources to support their understanding and best practices for use of the platform.
Project Proposal Not for external use.
Assessment Item Fill in the blank eLearning Module/Quiz requiring identification of 15 relevant product features and their function that are of relevance to clients working in non-profit organizations.
Scoring eLearning Module, tracked by LMS and scored by LMS.
Call Simulation Scorecard with blanks for identifying the product feature, use case or other learning objective presented in the case presented to the learner, the corresponding response from the learner as to the which product feature, use case, etc. was presented and a final blank for providing feedback to the learner in general for that case scenario.
Peer evaluated and with direct, immediate feedback as to presentation, accuracy, efficiency, and effectiveness.
eLearning Quiz with video and audio files of customer calls wherein the learner will identify product features referred to in the files, and answer additional questions about the related technical functions, and best practices of relevance to a customer describing the information included in the example case. eLearning Quiz with video and audio files of customer calls wherein the learner will identify product settings, features and related functionality referred to in the files, and answer additional questions about related cause that would prompt the customer to place a support call. Call Simulation Scorecard with blanks for identifying the product feature, use case or other learning objective presented in the case presented to the learner, the corresponding response from the learner as to the which product feature, use case, etc. was presented and a final blank for providing feedback to the learner in general for that case scenario.
eLearning Module, tracked by LMS and scored by LMS.
Program: CSM Success
eLearning Module, tracked by LMS and scored by LMS.
Peer evaluated and with direct, immediate feedback as to presentation, accuracy, efficiency, and effectiveness.
Version 2.1 Date: 07.08.2023 - 20 -
4.5 Assessment Tool Call Simulation Scorecard
Project Proposal Not for external use.
Program: CSM Success
Version 2.1 Date: 07.08.2023 - 21 -
5. Instructional Strategy 5.1 Course Overview Module Name Module 1: Program Orientation – Success Metrics
Objectives Assigned 1.0
Time Estimation 45 minutes
Module 2: Client Use Cases
1.0, 1.1, 1.1.2, 1.2.2
Self-Paced Component: 45 minutes (2 units, total 1.5 hours) Facilitated Component: 1.5 hours
Module 3: The Appointmently Solution
1.0, 1.1, 1.1.1, 1.1.2.1, 1.1.2.2, 1.1.2.3, 1.1.2.4, 1.1.2.5, 1.1.2.6, 1.2, 1.2.1, 1.2.1.1, 1.2.2
Self-Paced Component: 45 minutes (2 units, 1.5 hours total) Facilitated Component: 1.5 hours
Module 4: CSM Success
1.0, 1.2, 1.2.2
1.5 hours
Pre-instructional activities: The program orientation module will include an ice-breaker activity (learner-created slides to introduce themselves to the group. The facilitator will then introduce themselves and provide an overview of the program, focusing on connecting the objectives of the intervention with the goals of the CSMs in their roles with Appointmently.
Content presentation: Content will be presented using eLearning modules hosted in the company LMS, slide deck presentations during virtual facilitated sessions, and online quiz retrieval exercises/games using the Kahoot! Platform and process/procedure manuals or “Cook Books” of Appointmently use cases, platform features, and best practices.
Student participation: Learners will participate in online quiz games via Kahoot! and work in small groups during facilitated sessions to complete case studies and simulations wherein they will provide their peers' feedback on their performance during the simulation.
Assessment activities: eLearning module quizzes will measure learner understanding of the related material and provide immediate feedback. Call Simulations will provide peer feedback based on case studies and learner responses to the prompts. The Kahoot quizzes also produce Project Proposal Not for external use.
Program: CSM Success
Version 2.1 Date: 07.08.2023 - 22 -
reports reflecting the learners’ performance during the games and indicate specific topics or individuals struggling with any topics.
Follow-through: All learners can access digital copies of the procedure manuals, technical support documents, use case scenarios, “Cook Books,” and other resources or learning artifacts from the program available on the Appointmently intranet. Additionally, there will be individual coaching sessions to provide additional support, guidance, or advisement to provide feedback and supportive instruction as needed for the learners in the program.
5.2 Activity Map Objective
Event
Strategy/Activity
Resources/ Media
Time Frame
Icebreaker: Learner created selfintroduction slide presentation. Set goals: Learner introduction to program and connection between objectives of the program and the learner’s tasks during the intervention.
Slide presentations.
45 minutes
Module 1: Program Orientation and Success Metrics 1.0
Gain attention, focus on goals
Module 2: Client Use Cases 1.1
Connect to prior knowledge, present and organize content information
eLearning Quiz with video and audio files of customer calls wherein the learner will identify product features referred to in the files, and answer additional questions about the related technical functions, use cases, and best practices of relevance to a customer expressing the information included in the example case.
eLearning module hosted on company LMS, customer call videos and audio files for case studies
45 minute submodules (2)
1.1.2
Connect to prior knowledge
Kahoot! Quiz Game for retrieval practice regarding product use cases.
Kahoot! Quiz
30 minutes
Project Proposal Not for external use.
Program: CSM Success
Version 2.1 Date: 07.08.2023 - 23 -
Objective
Event
Strategy/Activity
Resources/ Media
Time Frame
1.2.2
Synthesize and evaluate, take action and monitor progress, extend and transfer
Case Study / Call Simulation for identifying the product feature, use case or other learning objective presented in the case presented to the learner and providing feedback to the learner in general for that case scenario.
Call Simulation Score Card, Call Simulation Case Studies
1 hour
Module 3: The Appointmently Solution 1.1, 1.1.1, 1.1.2.1, 1.1.2.2, 1.1.2.3, 1.1.2.4, 1.1.2.5, 1.1.2.6, 1.2.1, 1.2.1.1
Gain attention, connect to prior knowledge, present and organize content information
eLearning Quiz with video and audio files of customer calls wherein the learner will identify product features referred to in the files, and answer additional questions about the related technical functions, use cases, and best practices of relevance to a customer expressing the information included in the example case.
eLearning module hosted on company LMS, customer call videos and audio files for case studies
45 minute submodules (2)
1.1.1, 1.1.2.1, 1.1.2.2, 1.1.2.3, 1.1.2.4, 1.1.2.5, 1.1.2.6
Connect to prior knowledge
Kahoot! Quiz Game for retrieval practice regarding product use cases.
Kahoot! Quiz
30 minutes
1.0, 1.2, 1.2.1.1, 1.2.2
Synthesize and evaluate, take action and monitor progress, extend and transfer
Case Study / Call Simulation for identifying the product feature, use case or other learning objective presented in the case presented to the learner and providing feedback to the learner in general for that case scenario.
Call Simulation Score Card, Call Simulation Case Studies
1 hour
Module 4: CSM Success
Project Proposal Not for external use.
Program: CSM Success
Version 2.1 Date: 07.08.2023 - 24 -
Objective
Event
Strategy/Activity
Resources/ Media
Time Frame
1.0, 1.2, 1.2.2
Synthesize and evaluate, take action and monitor progress, extend and transfer
Case Study / Call Simulation for identifying the product feature, use case or other learning objective presented in the case presented to the learner and providing feedback to the learner in general for that case scenario.
Call Simulation Score Card, Call Simulation Case Studies
1 hour
6. Product Prototype Module 3 eLearning module wireframe.
7. Formative and Summative Evaluation 7.1 Expert Reviews (Kirpatrick Level 1: Reaction) Subject Matter Experts (SMEs) from various backgrounds, including current highperforming CSMs, managers, engineers, and marketing team members, will be provided with access to the content in a secure online environment. They will be asked to thoroughly review the content and complete a structured feedback form covering aspects such as accuracy, relevance, and technical aspects of the intervention materials. Regular meetings will be held with SMEs to discuss their feedback and incorporate suggested improvements, ensuring that the program effectively meets learners' needs and expectations.
7.2 One-to-One Evaluation (Kirpatrick Level 1: Reaction) For the one-to-one evaluations, a diverse group of learners will be scheduled for individual sessions in a controlled environment. The program will be presented using paper prototypes or digital modules, depending on the stage of development. A designer will observe each learner as they work through the material, taking notes on their interactions, questions, and general experience. The observer will also encourage the learner to think aloud and share their thoughts on the content and design, enabling the design team to gain valuable insights into potential areas of improvement.
7.3 Small Group Evaluation (Kirkpatrick Level 1: Reaction) A small group of learners, ideally representing different backgrounds and roles, will be invited to participate in a focus group session. Each participant will work through the modules individually, and then the group will discuss their impressions and experiences. The session will be recorded, and a designer will be present to facilitate the conversation, but not actively intervene. This approach will allow the design team to gather additional feedback from peer interactions and identify any potential issues or improvements needed.
Project Proposal Not for external use.
Program: CSM Success
Version 2.1 Date: 07.08.2023 - 25 -
7.4 Field Test (Kirpatrick Level 2: Learning) Based on the feedback from one-to-one, expert, and small group evaluations, the digital modules will be further refined and deployed for a field test (pilot) of the program. A mix of trainers, managers, high-performing CSMs, and low-performing CSMs will participate in the pilot, offering diverse perspectives on the program's effectiveness. Web analytics or "click-through" data will be collected to assess usability and focus group surveys will be administered at the end of each session to gauge participants' learning outcomes. The facilitated portion of each module will be recorded for subsequent analysis by the design team.
7.5 Summative Evaluation (Kirpatrick Levels 3 & 4: Behavior and Results) After the program is fully implemented, a summative evaluation will be conducted to measure its impact on participants' job performance and the organization's overall results. Electronic surveys will be administered immediately after each eLearning module to assess participants' predicted ability to apply their knowledge on the job (learning/reaction) and how much the program has already impacted their job performance (behavior/results). Supervisors and managers of the program participants will receive a checklist of desired behavioral changes to track each learner's adoption of the desired outcomes. They will also provide feedback to the learners based on prompts built into the checklist to reinforce positive outcomes from the intervention objectives. Data collected from these evaluations will be analyzed to identify areas for improvement and inform any necessary updates to the intervention. This process ensures that the program remains effective and relevant in achieving its intended outcomes and aligns with the Kirkpatrick Model's four levels of evaluation. Evaluation Item
Description
Expert Reviews
SMEs review content and provide feedback on accuracy, relevance, and technical aspects. Individual learners work through the material while being observed by a designer. Learners participate in focus group sessions to discuss their impressions and experiences. Pilot participants engage with the refined program, providing feedback on learning outcomes. Surveys assess participants' predicted ability to apply knowledge on the job and program impact. Supervisors track learners' adoption of desired behavioral changes and provide feedback. Data analysis identifies areas for improvement and informs necessary updates to the intervention.
One-to-One Evaluation Small Group Evaluation Field Test Electronic Surveys Manager Checklist Summative Evaluation
Project Proposal Not for external use.
Program: CSM Success
Version 2.1 Date: 07.08.2023 - 26 -
Appendix 1. Kahoot! Quiz Prototype Cust omer Success Quiz 1.1 - Det ails - Kahoot !
3 /6 /2 3, 12 :4 1 AM
Custom er Success Quiz 1.1 Ap p oin t m en t ly Prod u ct Qu iz! 0 p lays • 0 p layers
A p rivat e k ah oot
Quest ions (3) 1 - Qu iz W hich product feat ure is the best opt ion for solving back-… and-fort h em ails w it h a group Apw p hen oin t m en t ly Book about people are m e! available?
20 sec
Ap p oin t m en t ly Book It Now ! Ap p oin t m en t ly Sch ed u ler Th at is n ot a feat u re of an y of ou r p rod u ct s.
2 - Tru e or false There is no feat ure in Appoint m ent ly to help a … custom er check t heir schedule. Tru e
20 sec
False
ht t ps: //c reat e.kahoot .it /det ails/3 f 2f 57b7- 53 3 9 - 4 73 0 - 8 8 3 7- 8 d1af 0 d79 5d2
Project Proposal Not for external use.
Program: CSM Success
Page 1 of 2
Version 2.1 Date: 07.08.2023 - 27 -
2. Appointmently Call Simulation Scorecard
Project Proposal Not for external use.
Program: CSM Success
Version 2.1 Date: 07.08.2023 - 28 -
3. Appointmently eLearning Module Prototype / Wireframe
Project Proposal Not for external use.
Program: CSM Success
Version 2.1 Date: 07.08.2023 - 29 -
Document Revision Log Modified By:
Date and Section Amended
Revision Notes
Michael Sanford
07.08.2023 | All
Proposal Drafted for Client
Project Proposal Not for external use.
Program: CSM Success
Version 2.1 Date: 07.08.2023 - 30 -