Best Practices for Health Care Team Training Evaluation
Weaver, et al. (2011) published an article, entitled “Twelve Best Practices for Team Training Evaluation in Health Care,” in The Joint Commission Journal on Quality and Patient Safety.
The article intends to identify best practices for health care team training evaluation based upon nearly three decades of research and practice. Specifically, the authors state:
“Our goal is not to present a new methodology for evaluation but to distill principles from the science and temper them with the practical considerations faced on the front lines, where evaluation efforts compete with limited human, financial, and time resources” (p. 342).
They go onto clarify that:
“Although the 12 best practices may be perceived as intuitive, they are intended to serve as reminders that the notion of evidence-based practice applies to quality improvement initiatives such as team training and team development as equally as it does to clinical intervention and improvement efforts” (p. 341).
The twelve best practices, arranged according to three phases of training (Planning, Implementation, & Follow-Up), are:
Planning
- Best Practice 1. Before designing training, start backwards: Think about traditional frameworks for evaluation in reverse.
- Best Practice 2. Strive for robust, experimental design in your evaluation: It is worth the headache.
- Best Practice 3. When designing evaluation plans and metrics, ask the experts—your frontline staff.
- Best Practice 4. Do not reinvent the wheel; leverage existing data relevant to training objectives.
- Best Practice 5. When developing measures, consider multiple aspects of performance.
- Best Practice 6. When developing measures, design for variance.
- Best Practice 7. Evaluation is affected by more than just training itself. Consider organizational, team, or other factors that may help (or hinder) the effects of training (and thus evaluation outcomes).
Implementation
- Best Practice 8. Engage socially powerful players early. Physician, nursing, and executive engagement is crucial to evaluation success.
- Best Practice 9. Ensure evaluation continuity: Have a plan for employee turnover at both the participant and evaluation administration team levels.
- Best Practice 10. Environmental signals before, during, and after training must indicate that the trained KSAs and the evaluation itself are valued by the organization.
Follow-up
- Best Practice 11. Get in the game, coach! Feed evaluation results back to frontline providers and facilitate continual improvement through constructive coaching.
- Best Practice 12. Report evaluation results in a meaningful way, both internally and externally.
References
Weaver, S.J., Salas, E., & King, H.B. (2011). Twelve best practices for team training evaluation in health care. The Joint Commission Journal on Quality and Patient Safety, 37, 341-349.
Team Building and Development Resources
Below I have provided a sample of the books, chapters, and articles on the topic, along with some of the thought leaders conducting work in this area.
Books
Bowers, C., Salas, E., & Jentsch, F. (2006). Creating high-tech teams: Practical guidance on work performance and technology. Washington, D.C.: American Psychological Association.
Hackman, J.R. (2002). Leading teams: Setting the stage for great performances. Boston, MA: Harvard Business School Press.
Levi, D. (2013). Group dynamics for teams (3rd ed.). Los Angeles, CA: Sage Publications.
Salas, E., & Fiore, S.M. (2004). Team cognition: Understanding the factors that drive process and performance. Washington, D.C.: American Psychological Association.
Stanton, N.A., Hedge, A., Brookhuis, K., Salas, E., Hendrick, H.W. (2004). Handbook of human factors and ergonomics methods. Boca Raton, FL: CRC Press.
Wageman, R., Nunes, D.A., Burruss, J.A., & Hackman, J.R. (2008). Senior leadership teams: What it takes to make them great. Boston, MA: Harvard Business School Press.
Chapters
Hackman, J.R., & Katz, N. (2010). Group behavior and performance. In S.T. Fiske, D.T., Gilbert, & G. Lindzey (Eds.), Handbook of social psychology (5th ed.) (pp. 1208-1251). New York: Wiley.
Kozlowski, S. W. J., & Bell, B. S. (2003). Work groups and teams in organizations. In W. C. Borman, D. R. Ilgen, & R. Klimoski (Eds.), Handbook of psychology: Industrial and organizational psychology: Vol. 12 (pp. 333-375). London: Wiley.
Salas, E., Rosen, M. A., Burke, C. S., Goodwin, G. F., & Fiore, S. (2006). The making of a dream team: When expert teams do best. In K. A. Ericsson, N. Charness, P. J. Feltovich & R. R. Hoffman (Eds.), The Cambridge Handbook of Expertise and Expert Performance (pp. 439-453). New York: Cambridge University Press.
Salas, E., Stagl, K. C., & Burke, C. S. (2004). 25 years of team effectiveness in organizations: Research themes and emerging needs. In C. L. Cooper & I.T. Robertson (Eds.), International review of industrial organizational psychology: Vol. 19 (pp. 47-92). New York: John Wiley & Sons.
Salas, E., Stagl, K. C., Burke, C. S., & Goodwin, G. F. (2007). Fostering team effectiveness in organizations: Toward an integrative theoretical framework of team performance. In R. A. Dienstbier, J. W. Shuart, W. Spaulding, & J. Poland (Eds.), Modeling complex systems: Motivation, cognition and social processes: Nebraska Symposium on Motivation: Vol 51 (pp. 185–243). Lincoln: University of Nebraska Press.
Stagl, K. C., Salas, E., & Burke, C. S. (2007). Best practices in team leadership: What team leaders do to facilitate team effectiveness. In J.A. Conger & R.E. Riggio (Eds.), The practice of leadership: Developing the next generation of leaders (pp. 172-197). San Francisco, CA: Jossey-Bass.
Articles
Burke, C. S., Stagl, K. C., Klein, C., Goodwin, G. F., Salas, E., & Halpin, S. M. (2006). What type of leadership behaviors are functional in teams? A meta-analysis. Leadership Quarterly, 17(3), 288-307.
Hulsheger, U. R., Anderson, N., & Salgado, J. F. (2009). Team-level predictors of innovation at work: A comprehensive meta-analysis spanning three decades of research. Journal of Applied Psychology, 94(5), 1128-1145.
Ilgen, D. R., Hollenbeck, J. R., Johnson, M., & Jundt, D. (2005). Teams in organizations: From input-process-output models to IMOI models. Annual Review of Psychology, 56, 517–543.
Klein, C., DiazGranados, D., Salas, E., Le, H., Burke, C.S., Lyons, R., & Goodwin, G.F. (2009). Does team building work? Small Group Research, 40, 181-222.
Kozlowski, S. W. J., & Ilgen, D. R. (2006). Enhancing the effectiveness of work groups and teams. Psychological Science in the Public Interest, 7, 77–124.
Mathieu, J., Maynard, M.T., Rapp, T., & Gilson, L. (2008). Team effectiveness 1997-2007: A review of recent advancements and a glimpse into the future. Journal of Management, 34, 410-476.
McGrath, J.E., Arrow, H., & Berdahl, J.L. (2000). The study of groups: Past, present, and future. Personality and Social Psychology Review, 4, 95-105.
Salas, E., Cooke, N.J., & Rosen, M.A. (2008). On teams, teamwork, and team performance: Discoveries and developments. Human Factors, 50, 540-547.
Salas, E., DiazGranados, D., Klein, C., Burke, C.S., Stagl, K.C., Goodwin, G.F., & Halpin, S.M. (2008). Does team training improve team performance? A meta-analysis. Human Factors, 50, 903-933.
Thought Leaders
J. Richard Hackman, Ph.D. [1940-2013]
Essential Reviews of the Teams Literature
In an increasingly complex work environment, organizations are relying more than ever on effective teams to meet their goals.
But, what are the best ways to create and support teams? One method of answering this question is to identify best practices from the scientific literature.
Below are eight reviews that provide an overview to the breadth and complexity of research on teams:
- Ilgen, D. R., Hollenbeck, J. R., Johnson, M., & Jundt, D. (2005). Teams in organizations: From input-process-output models to IMOI models. Annual Review of Psychology, 56, 517–543.
- Kozlowski, S. W. J., & Bell, B. S. (2003). Work groups and teams in organizations. In W. C. Borman, D. R. Ilgen, & R. Klimoski (Eds.), Handbook of psychology: Industrial and organizational psychology: Vol. 12 (pp. 333-375). London: Wiley.
- Hackman, J. R., & Katz, N. (2010). Group behavior and performance. In S. T. Fiske, D. T. Gilbert, & G. Lindzey (Eds.), Handbook of social psychology (5th ed.) (pp. 1208-1251). New York: Wiley.
- Kozlowski, S. W. J., & Ilgen, D. R. (2006). Enhancing the effectiveness of work groups and teams. Psychological Science in the Public Interest, 7, 77–124.
- Mathieu, J., Maynard, M.T., Rapp, T., & Gilson, L. (2008). Team effectiveness 1997-2007: A review of recent advancements and a glimpse into the future. Journal of Management, 34, 410-476.
- McGrath, J.E., Arrow, H., & Berdahl, J.L. (2000). The study of groups: Past, present, and future. Personality and Social Psychology Review, 4, 95-105.
- Salas, E., Stagl, K. C., & Burke, C. S. (2004). 25 years of team effectiveness in organizations: Research themes and emerging needs. In C. L. Cooper & I.T. Robertson (Eds.), International review of industrial organizational psychology: Vol. 19 (pp. 47-92). New York: John Wiley & Sons.
- Salas, E., Stagl, K. C., Burke, C. S., & Goodwin, G. F. (2007). Fostering team effectiveness in organizations: Toward an integrative theoretical framework of team performance. In R. A. Dienstbier, J. W. Shuart, W. Spaulding, & J. Poland (Eds.), Modeling complex systems: Motivation, cognition and social processes: Nebraska Symposium on Motivation: Vol 51 (pp. 185–243). Lincoln: University of Nebraska Press.
Team Leadership Best Practices
Team leadership is an important aspect of team performance within organizations. Stagel, Salas, and Burke (2007) have summarized the findings from the team leadership literature into twenty best practices, including:
- Define and create interdependencies.
- Reinforce task interdependencies with congruent goals and feedback.
- Identify who is responsible and accountable for outcomes.
- Designate the team’s decision making authority.
- Strive to keep teams intact.
- Exercise authority to establish a compelling direction.
- Stimulate and inspire by challenging the status quo.
- Instill collective aspirations via a common mission.
- Provide consequential direction to fully engage talent.
- Promote self-goal setting, self-observation, and self-reward.
- Establish norms for how the team scans its environment for opportunities and what teams must and cannot do to seize opportunities.
- Allocate the optimal number and mix of personnel.
- Implement team-based performance-contingent rewards.
- Institutionalize multi-tiered reward systems.
- Ensure provided information is performance targeted.
- Negotiate access to sensitive information if it facilitates planning and selection of performance strategies.
- Provide and secure developmental opportunities.
- Utilize prebriefings to instill shared affect, cognition, and behavior.
- Offer novel task performance strategies.
- Engage teams in a two-way discussion of lessons learned and how they can be utilized to address challenges on the horizon.
These twenty best practices have, in turn, been consolidated into a framework of five conditions for team effectiveness based on the work of J. Richard Hackman, Ph.D. Dr. Hackman is the author of “Leading Teams: Setting the Stage for Great Performance.” The five conditions, with their corresponding best practices listed in parentheses, are:
- A real team (#1-5)
- Compelling direction (#6-9)
- Enabling structure (#10-12)
- Supportive organizational context (#13-17)
- Expert coaching (#18-20)
The authors close the chapter with the following observation:
“Team leaders who spend a majority of their time establishing these five conditions rather than directly intervening in team performance will likely find that their teams are increasingly capable and willing to adaptively respond in a coordinated manner, and thus they and their teams will ultimately be deemed more effective in the wild” (p. 197).
If you have an interest in increasing team performance within your organization, I’d suggest taking the time to read this chapter.
References
Stagl, K. C., Salas, E., & Burke, C. S. (2007). Best practices in team leadership: What team leaders do to facilitate team effectiveness. In J.A. Conger & R.E. Riggio (Eds.), The practice of leadership: Developing the next generation of leaders (pp. 172-197). San Francisco, CA: Jossey-Bass.
Characteristics of Expert Teams
Salas, Rosen, Burke, Goodwin, and Fiore (2006) have defined “…an expert team as a set of interdependent team members, each of whom possesses unique and expert-level knowledge, skills, and experience related to task performance, and who adapt, coordinate, and cooperate as a team, thereby producing sustainable, and repeatable team functioning at superior or at least near-optimal levels of performance” (p. 440).
Rosen, Salas, Lyons, and Fiore (2008), adapting a table from Salas, et al. (2006), presented a list that characterizes members of expert teams. They (pp. 221-222):
1. Develop shared mental models
- They anticipate each other’s needs and actions.
- They can communicate implicitly.
- They interpret cues in a complimentary manner.
2. Learn and adapt.
- They self-correct.
- They learn from past decision-making episodes.
- They adapt coordinating processes to dynamic environments.
- They compensate for each other.
3. Maintain clear roles and responsibilities.
- They manage expectations.
- They understand each other’s roles and how they fit together.
- They maintain clarity of roles while maintaining flexibility.
4. Possess clear, valued, and shared vision.
- They develop their goals with a shared sense of purpose.
- They guide their decisions with a common set of values.
5. Develop a cycle of pre-brief -> performance -> debrief.
- They regularly provide individual and team level feedback to one another.
- They establish and revise team goals and plans.
- They dynamically set priorities.
- They anticipate and review issues/problems of members.
- They periodically diagnose team decision making “effectiveness,” including its results, and its processes.
6. Are led by strong team leaders.
- They are led by someone with good leadership skills and not just technical competence.
- They believe the leaders care about them.
- Leaders of expert teams provide situation updates.
- Leaders of expert teams foster teamwork, coordination, and cooperation.
- Leaders of expert teams self-correct first.
7. Have a strong sense of “collective,” trust, teamness, and confidence.
- They manage conflict well; they confront each other effectively.
- They have a strong sense of team orientation.
- They trust other team members’ “intentions.”
- They strongly believe in the team’s collective ability to succeed.
8. Cooperate and coordinate.
- They identify teamwork and task work requirements.
- They ensure that, through staffing and/or development, the team possesses the right mix of competencies.
- They consciously integrate new team members.
- They distribute and assign work thoughtfully.
- They examine and adjust the team’s physical workplace to optimize communication and coordination.
References
Rosen, M.A., Salas, E., Lyons, R., & Fiore, S.M. (2008). Expertise and naturalistic decision making in organizations: Mechanisms of effective decision making. In G.P. Hodgkinson & W.H. Starbuck (Eds.), The Oxford Handbook of Organizational Decision Making (pp. 211-230). New York: Oxford University Press.
Salas, E., Rosen, M. A., Burke, C. S., Goodwin, G. F., & Fiore, S. (2006). The making of a dream team: When expert teams do best. In K. A. Ericsson, N. Charness, P. J. Feltovich & R. R. Hoffman (Eds.), The Cambridge Handbook of Expertise and Expert Performance (pp. 439-453). New York: Cambridge University Press.