Scenario-based training

The Problem

This concept project is for ShareCare, an agency that matches live-in childcare providers from foreign countries (au pairs) with host families in the United States. 

‍

The company was experiencing a high rate of match breaks between au pairs and host families within the first year of employment, causing a high financial loss and creating emotional stress on all parties involved in the matches. 

‍

Agency field staff, who have to mediate the needs of both au pairs and host families, were experiencing burnout from the stress of handling repeated match breaks. As a result, it was difficult for the agency to retain field staff.

‍

‍My Solution

Drawing from my three years of experience as an agency field staff member for an agency similar to this client, I served as the subject matter expert (SME) for this project. 

‍

After defining the problem, I began to analyze it to determine if there was a performance issue that could be solved or improved through training. I determined that there was a point in the process where field staff met with host families before the host family selected an au pair. This meeting was referred to as the host family interview, and it served as an opportunity for field staff to set clear expectations about what host families should discuss and look for in a potential au pair. 

‍

To train field staff, I proposed a scenario-based eLearning solution that would help the field staff navigate the host family interview successfully. The eLearning experience would invite field staff to practice aligning host family expectations with reality. As a result, this would reduce the number of first-year match breaks.

‍

My Process

Because this project involved solving complex and ill-defined problems, I chose the Hasso-Plattner Institute of Design’s design-thinking model as my design methodology. I find it is useful when working with complex problems because it re-frames the problem in a human-centric way to the advantage of the learner or end-user, leading many to refer to it as the human-centered design model. This is a five-stage model: empathize, define problem, ideate, prototype and test.

‍

Empathizing

I begin my projects by learning about the learners’ unique challenges and struggles. This helps me to empathize with my learners which enables me to create e-learning experiences that are relevant to their needs. 

‍

To empathize for this project, I interviewed host families and au pairs to discover what things worked well and what didn’t in their living environments. I interviewed other field staff and documented how they handled problematic situations. I conducted mediations and learned about common “hot-button” issues for many host families and/or au pairs. I used this information to help define the problem and begin the mental process of crafting an effective solution.

‍

‍Defining the Problem

After synthesizing the various data and pain points I gathered in the empathize stage, I defined the following problem statement:

‍

‍Field staff agents need to establish realistic expectations with host families about what to expect from their au pair in order to minimize misunderstandings due to personality or cultural differences when their au pair begins living with them. 

‍

According to research, field agents were using the host family interview to cover a list of agency rules and regulations as outlined in a handbook but were not covering unique case situations. In order to help host families establish realistic expectations unique to their families’ needs, the agents would need to look for things that may be problematic with au pairs if not addressed early. If not addressed appropriately, however, host families could easily be overwhelmed by the interview and choose to opt out of getting an au pair altogether. I proposed a scenario-based training solution to provide a real-world context for the field staff to practice the necessary soft skills needed when speaking with host families.

‍

With the problem defined and a delivery method for training established, I began to identify key issues the field staff needed to discuss with their host families during the interview based on my observation and interview data. To visualize this and make sure that I was keeping the focus on specific actions the staff agents should do, I developed an action map.

‍

‍

This map laid out the project goal in the center (establishing host family expectations) and branched to display key actions that the field staff should do (what points they should expressly discuss during the interview). These actions led to more finite actions that all related back to the project goal. 

‍

‍Ideation

Next, I began the process of developing the story elements for the project. I used my action map as a guide to create my scenarios. I identified five important actions on the map to use as a basis for my scenarios and, from these, brainstormed ideas to further develop each scenario and decision points in the story. After iterating through many ideas, I focused this project on au pair availability, establishing work hours, discipline and rewards measures, pet care, and driving standards. I documented my scenarios and decision points with a text-based storyboard, which included scripts for user scenarios, navigation options, questions for the learner, consequences based on the user answer choice, and help text for optional help screens the learner could select at key decision points throughout the course.

‍

‍

With the story development complete, I turned my attention to the visual design elements and attributes needed for the project. I knew these scenarios would be text-heavy, so I needed to break up the content to reduce cognitive load on the learner. One way I accomplished this was by “hiding” additional information behind a virtual help button. In addition to breaking up the content, this feature also provided guided instruction for the user. However, since accessing the help feature was optional, it still allowed the user to control their learning experience and only access supporting content as needed.

‍

‍

With the user in control of the scenario, I wanted to provide them with visual feedback regarding their answer choices. I did this in multiple ways. I provided feedback screens for each scenario showing the correct and incorrect scenario choices. I used both sound and color to indicate correctness. A quick audio playback response was given when the user chose an answer - an upbeat chime for correct responses or a deeper warning sound for incorrect responses. I used a progress bar to give visual feedback as the user progressed through the scenarios. In addition to showing a numeric increase to the progress bar on the correct response screens, the progress bar was colored green to indicate accuracy. Incorrect response screens showed a red progress bar since red is often associated with warnings.

‍

‍

I developed the visual components of the project using Adobe XD. I iterated through many ideas and versions before settling on the finished design. Working in XD before moving into an authoring tool allowed me to make quick design changes without losing valuable production time.

‍

‍

‍

I made the design decision to use photorealistic imagery versus illustrative styling. This was to help immerse the learner in the environment. I wanted the imagery to paint a picture in the learner’s mind as they read the scenario. When possible I tied the on-screen text to the imagery as well, like naming a host family pet “Spot” and showing an image of a dalmatian.

‍

Following Mayer’s Personalization Principle, I created a mentor character to provide guided instruction throughout the course. I created the character to be engaging and relatable to the learner in an effort to increase learner engagement. 

‍

To continue with immersive design, color played an important role. I ideated through several color themes before settling on a rich blue and green color combination. The blue represented warmth, trustworthiness, and dependability. The green was a complimentary color that also symbolized growth and stability. I chose soft drop shadows and calming white backdrops to house the text areas, but again used repetition to capture a slice of the main scenario image to repeat on the question/response screen. This was done to visually reinforce the story element in the learner’s mind while they were contemplating an answer choice.

‍

‍

With both the story and visual elements developed, I combined the text-based storyboard with the visual designs from XD to create a visual storyboard. Seeing all elements together in this format allowed me to think through the project flow, make decisions regarding interactivity and determine if and when I wanted to include sound effects or narration. The visual storyboard became an important design document as I moved forward to the next phase of the design model, prototyping.

‍

Prototyping

After getting feedback on the visual storyboard, I moved into the authoring environment. For this project, I used Articulate Storyline. With the visual storyboard in hand, creating a prototype was straightforward. I created a prototype to include: an opening video sequence, intro scene, one complete scenario with setup, question choices, result screens, and a conclusion slide. The purpose of the prototype was to test the functionality and interactive flow of the project and gain feedback. 

‍

If this was not a concept piece, this would be a key point in development to solicit feedback on the project functionality from key stakeholders. I knew gaining feedback was critical, so I submitted this to several peers for review. The majority of feedback I received was on the decision screens for correct and incorrect responses. Testers felt there was not enough of a visual distinction between the two screens. To address this, I did a few things. I added a title that says, “Correct” on the correct screen, and changed the color of the rating bar to either green or red. I also added a quick transition with a green or red color block that flashed on the results screen to draw attention to the rating bar and give a visual cue to the user about the accuracy of their response. Testers confirmed that these changes made for a better user experience.

‍

‍

Testing

With the prototype complete, I moved into full development mode. I completed the remaining four scenarios and added the slide content for the virtual assistant. I added triggers, states, and layer slides as appropriate. I used custom Javascript to code the progress bar indicator. This had to be adjusted for every scene and tested for accuracy. I stored the progress level and the learner’s name in variables that were tracked throughout the course.

‍

‍

With a lot more pieces and parts, it was important that I tested regularly to ensure I was continuing to achieve the desired results. This process was iterative and took an extensive amount of time, but I believe there is no substitute for good quality control. Additionally, I believe accessibility is not a factor that should be overlooked, so part of my testing measures included checking that my tab order was correct for keyboard-only users and entering alternate text on images for learners using a screen-reader.

‍

‍

I also included a restart and an exit button on the screen at all times so the user never felt trapped in any one spot in the course. I created validation screens that appeared if the user selected either button. The purpose of this was to make sure the user intentionally selected those buttons and understood that if they did choose to exit or restart the course none of their course progress would be saved.

‍

‍

Results

I am drawn to data-driven results. Because this is a conceptual project, I don’t have data to provide feedback on whether the overall number of match-breaks reduced after training or to survey and see how field staff reacted to the training. However, the active learning approach and real-world nature of this project should translate to transferable skills and improved job performance. 

‍

From seeing the results of various course testers, the story-driven scenario approach used in the project allowed the learner to quickly engage with the content in a way that allowed them to learn by doing. The scenarios allowed learners to practice guiding host families through options they need to consider before selecting an au pair. They were given opportunities to replay the scenarios and select alternate choices to determine various outcomes. This allowed them to gain feedback from both correct and incorrect choices and gain an understanding of why those decisions were important. The choice to involve the learners in the decision-making process made the content much more engaging than simply presenting content to the learner to memorize. 

‍

Takeaways

I enjoyed creating and working from an action map. Because I used an action map based on user research to create my scenarios, I had confidence that my content was aligned with my objectives throughout the project. I hope to transfer this approach to other e-learning projects. 

‍

During project development, I enjoyed delving into Javascript. There are countless applications where the use of Javascript can enhance Storyline projects and really take projects to the next level. I plan to continue looking for ways where I can utilize this skill in e-learning. The next step I would tackle on a project like this one would be to delve into xAPI. As I mentioned, I’m drawn to data-driven results, so being able to see how people interact with my eLearning on a slide-level basis would help me optimize it even further.