Innovative Learning Group

2022 Brandon Hall Silver Award, Excellence in Learning - Best Unique or Innovative Learning and Development ProgramA global pharmaceutical company with 11 industrial sites, five research and development sites, and 16,000 employees partnered with Innovative Learning Group (ILG) to create an innovative approach to its Aseptic Best Practices Training program.SITUATIONAs with many pharmaceutical companies, Quality Control's training centered around reading numerous, complex Standard Operating Procedures (SOPs) and participating in lengthy on-the-job training (OJT). In addition, analysts were trained on all knowledge and skills required to perform a test, even if they had learned these skills when trained on earlier tests. As a result, time to qualify to independently perform a single QC test exceeded five weeks.SOLUTIONThe Company must deliver safe and effective products to patients around the world. This is both a Company core value and a regulatory requirement. Consistently achieving this level of compliance requires that these products be produced by employees using aseptic processes in an aseptic environment. "Aseptic" means that the processes and work areas are free from contamination of harmful microorganisms. There are four levels of work area classifications for aseptic processing: from Class A, which has the most stringent aseptic requirements, through Class D, which has fewer, but specific requirements. Every action of every employee in classified work areas is critical to maintaining aseptic conditions.

Each Company site had its own in-depth "best practices," aligned with regulatory requirements, for how employees must act in the various classified areas. In addition to each site having its own compliance best practices, sites also had their own training requirements supporting the best practices.

Additionally, training and practice on the aseptic skills needed to be scheduled when the work areas were not operational. Unqualified employees are not allowed in certain areas when actual product is being processed. This meant that aseptic best practice training was spread out over more time than otherwise necessary.

The Company built a global Aseptic Best Practice Training Program that would be based on a single set of aseptic best practices across the company and that would allow for more practice. The target audience for the initiative was employees who work in Class A and B areas, with the training focused on skill building.

The initiative began with a research phase. During this phase, an ILG analyst conducted interviews with: Sterility Assurance Center of Excellence stakeholders; operators and mechanics working in the targeted classifications; managers of these employees; and on-the-job trainers.

The goals of these interviews were to:

• dentify the global set of aseptic best practices that were critical to include in the program.
• Identify what "good" looked like for each of these best practices (that is, identify the standards for performance).
• Define the reasons for each best practice in terms of aseptic processing.

ILG then organized the collected best practices into logical groupings, which became the nine Aseptic Best Practice Training Program modules.

Given the importance of training and practice when needed and without taking down the production line, virtual reality (VR) was identified as an appropriate delivery method for eight of the modules. (The best practices in the ninth module were better addressed using a solution blending video and interactive e-learning.) To ensure the VR modules could be effectively implemented within Company's technical infrastructure, the team conducted a technology assessment. As a result, the use of VR was finalized, the Oculus Quest 2 was selected as the VR headset for the training, and a means for tracking completion of the modules was identified.

The next phase of the project was for ILG's analyst to create detailed designs for all nine modules. The design document for each module presented the content outline and instructional strategies for each learning objective. The content outline included the scenario setting, the task, the best practices, and the "whys" for each practice. The instructional strategies explained what would happen in VR.

The VR team then began to develop the VR modules - one at a time. Development began with the creation of storyboards, based on the design document. These storyboards outlined the actual actions the learner would perform in the VR. The ILG analyst wrote the instructional content (the audio and text instructions, the feedback, and the "whys"). Once the storyboards for a module were completed, detailed 3D graphics development and VR programming were performed by ILG's VR team.

The resulting VR experiences provided the Company with a unique way to train learners in aseptic best practices. The virtual space is realistic to the Company, and the scenarios in the modules are realistic situations that employees in Class A and B areas face regularly. The VR allows them to learn, practice, and demonstrate these tasks, without actually being in that physical space - and without needing a live trainer until final on-the-job qualification. The Aseptic Best Practices training, because it was designed to fit the needs and the use, is succeeding.RESULTSKey measurable benefits identified in the business case for the Aseptic VR Training Program are:

• Reduction in the time it takes for an employee to qualify (be approved) to work in a Class A or Class B area.
• Increased operational equipment effectiveness, which measures the percentage of time manufacturing is productive.
• Employee demonstration of a harmonized set of aseptic best practices through harmonized training and the ability to practice when needed.

To date, employees who have experienced the VR have reported the following key benefits:

• All participants reported the VR was easy to use.
• There were no reports of motion sickness.
• A strong majority of participants gave a rating of a 7 or higher on their ability to perform correctly on the job immediately after training. (Rating scale was 1-10, where 10 was highly confident).
• A strong majority of participants indicated they could perform correctly on the floor after only one to three practices in the VR.
• All participants found the VR effective in building skills in the aseptic best practices.

The success of the Aseptic Best Practice Training Program to date, the strength of its business case, and the support of global leadership have led to the Company's commitment to developing the entire program. The Company anticipates that the program will serve as a powerful tool to drive best-practice performance for all employees involved.

SITUATIONThe Kaiser Permanente (KP) Health Plan Workforce Development (HPWD) group has a simple mandate: to provide the most effective, efficient learning solutions possible to its internal customers. In doing so, HPWD plays an important role in attracting and retaining new generations of workers, an outcome crucial to KP's continued well-being as an organization.HPWD provides learning solutions of all types to approximately 7,500 employees in customer sales, underwriting, and account management, health-plan product technology, marketing, operations, and support. Learning needs include everything from onboarding and career development to product knowledge and job-specific skills.

To achieve its goals, HPWD has long realized it must stake ground on the leading edge of learning practice and act as a learning thought-leader within KP. In 2018, augmented and virtual realities (AR and VR) were part of that leading edge and were generating widespread "buzz" in the learning and development world. However, HPWD needed to understand these technologies better before determining how they might be meaningfully applied at KP.SOLUTIONHPWD engaged Innovative Learning Group (ILG) to 1) educate a small group of HPWD personnel in essential AR and VR characteristics, uses, and tools; and 2) help them develop an AR and VR learning strategy tailored to HPWD's needs. The joint HPWD and ILG team met for a two-day working session that addressed the first objective in context of tackling the second one. In addition to guiding the session, ILG provided HPWD with hands-on experience using sample AR and VR applications.

By the end of the second day, the team had defined the components of the strategy. After the meeting ILG documented the strategy in a report, which was reviewed, revised, and finalized with the help of HPWD. The components are described below.

Vision

Creating a vision for the future is an important first step in developing any learning strategy. Here, the HPWD participants described the AR- and VR-delivered learning they hope to have in place in five years' time:"HPWD has successfully rolled out several targeted, sophisticated, performance-focused solutions using AR and VR technologies... Starting from onboarding and continuing throughout the employee's tenure at Kaiser, employees experience VR and AR solutions ranging from virtual reality tours of showcase Kaiser facilities...to contextual AR performance support that pushes critical information (such as product features or regulatory updates) at the moment of need."The vision continues with the benefits that are being realized, such as the following:

• Reduction or elimination of travel costs between KP headquarters and its seven regions while still providing immersive "you are there" tours of important KP facilities
• Letting supervisors practice difficult person-to-person interactions in low-risk virtual environments using avatars
• Supplying field sales personnel with just the right customer-relevant information at just the right time based on their geographic location
• Enhancing, customizing, and reinforcing onboarding content by pushing information and activities to new hires based on their workplace, task, or time on the job
• Giving learners the opportunity to virtually observe and practice skills used in different physical contexts throughout the organization, all within their training classroom

The vision concludes with results: overall, as shown in performance-related evaluation data, AR and VR solutions have produced improved performance and shorter time to proficiency. Moreover, the solutions are meeting the expectations of KP's early-career new hires for the latest technologies; KP is perceived as a future-facing, "cool" place to work.

This vision guided or influenced the thinking throughout the rest of the strategy.

Audiences and Macro Learning Needs

HPWD participants defined the major learning audiences by number, location, work characteristics and challenges, and key knowledge, skills, and tools used. This compilation of information is a necessary first step in determining and prioritizing learning needs.

Next, the team identified possible AR and VR applications that would be useful for each audience. This exercise helped the team tie the abstract concepts of AR and VR to the real-world context at KP.

Learning Delivery Methods

This section is a survey of the different types of AR and VR technologies currently available, categorized by hardware used (phone-, tablet-, or headset-based). The description of each type includes characteristics such as learning purpose, cost, level of real-time performance feedback provided, whether it's hands- or cable-free or requires dedicated floor space, and complexity of using the equipment for learners.

Procurement

The HPWD participants defined the logic they expect to follow in obtaining AR and VR learning solutions. Procurement may include buying a solution from a vendor, building a new solution in-house or with external resources, or some combination of these. HPWD's set of criteria identify the circumstances for using each procurement method. The section also outlines how HPWD will divide tasks when partnering with vendors and lists additional factors (cost, timeline, internal capacity, audience) that will affect procurement decisions.

Technology and Infrastructure

This section describes the tools and facilities needed in three areas:

1. Development of AR and VR solutions. Software tools for developing AR and VR applications are categorized by their capabilities and level of coding expertise needed to use them. HPWD will use this information to determine what tools to use for in-house development of applications.
2. Tracking AR and VR solutions. Explains the advantages of using Experience API (xAPI) to track either bookmarking and completion of training solutions (typically VR) or usage metrics for performance-support solutions (typically AR) and how xAPI may be supported in KP's learning management system.
3. Facilities and other physical infrastructure. Provides requirements for setting up a "lab" location at KP headquarters where AR and VR solutions will be available for employees to try out. Once these types of solutions have become familiar to HPWD audiences, the lab will be used primarily for people to take headset-based VR training.

Assessment and Evaluation

The team defined an evaluation process HPWD could implement for AR and VR solutions. HPWD already has a robust process in place for evaluating conventional learning solutions. The AR and VR version will be an adaptation of it.

Organizational Structure

HPWD participants identified the roles needed to implement the AR and VR strategy. Responsibilities and required skills and knowledge are described for each role.

Implementation Plan

HPWD participants defined the steps needed to implement the strategy. The plan consists of specific tasks to be completed and is the indispensable final piece of the strategy. By providing small, concrete, relatively simple things to do, it provides a path to get started on something that otherwise may seem overwhelming.

RESULTS

Within the first seven months after the AR and VR learning strategy was finalized, HPWD completed the following tasks:

• Make KP learning leaders aware of the strategy and the potential of AR and VR learning solutions.
• Educate all HPWD personnel on the potential of AR and VR learning solutions.
• Purchase KP-branded Google Cardboard-like viewers and select demonstration apps.
• Provide sample AR and VR experiences to staff at the HPWD All Hands Meeting.

Meanwhile, the Executive Director of HPWD has set aside funds and is working with his team to plan the increasing integration of AR and VR solutions over the next four to five years. The goal? To reach the vision laid out in the learning strategy, in which AR and VR are just another couple of tools in the learning toolbox.