S. Ginsburg

• The individual perspective focuses on an individual’s characteristics, attributes or behaviours. This is useful for evaluation purposes (most of which focus on single individuals) and for personal accountability. Most of the focus has shifted towards behaviours, as they appear to be more objective than attitudes or traits; however, behaviours are not always good indicators of motivations or intent and should be interpreted carefully. This perspective does not adequately consider or account for the context that is so powerful in shaping behaviours.

• The relational perspective considers professionalism as arising from (and affecting) relationships between individuals. This is helpful in broadening our understanding of context and in understanding how unprofessional behaviour may arise. However, it may not adequately be able to address individuals’ problematic behaviours, or the macro-social forces that act upon relationships.

• The institutional or organizational perspective allows one to consider how policies and power hierarchies can influence or even constrain individuals’ actions. This is helpful in understanding how and why people don’t always act according to their stated beliefs. However, it can obscure individual responsibility and accountability.

Defining professionalism

Although defining professionalism is not the focus of this chapter by any means, most would agree that it is impossible to evaluate something without first declaring what it is. In this case the definition is particularly important, as disagreement persists about what professionalism actually is, and what it encompasses. The Ottawa report states that the concept of professionalism varies across historical time periods and cultural contexts and that there is a need to develop or adopt concrete and operationalizable definitions from which to teach and evaluate professionalism. Further, these definitions should be negotiated with the public, as the practice of medicine relies on a social contract between the profession and society.

In practical terms this means that before a programme intends to evaluate professionalism, it should choose a definition that resonates with its members and should make this explicit so that teaching and evaluation can follow. See, for example, Wilkinson and colleagues’ efforts (2009) to blueprint professionalism assessment by considering aspects of its definition.

Why evaluate professionalism?

The evaluation of professionalism has, until relatively recently, been largely implicit rather than explicit like other domains of knowledge and skills. However, over the past couple of decades there has been a move to consider professionalism as a distinct entity that is evaluated, for several important reasons. For one thing, it signals to all stakeholders (the programme, students, patients and the public) that it is being taught and evaluated with the same importance (and consequences) as other knowledge and skills. It is assumed that this attention may help counteract effects of the hidden curriculum, along with the reported decrease in empathy that often occurs during training. And if professionalism issues are identified early in training, the logical outcome would be remediation (or removal).

Interestingly, it is only recently that evidence has been published to support this last point, but it is now clear that some professionalism issues identified early in training may predict disciplinary actions in practice (Papadakis et al 2005, Papadakis et al 2008). That said, these authors and others have pointed out that these red flags account for only a small proportion of attributable risk (i.e. most students do not go on to have documented difficulties in practice) so should be interpreted with caution. Other researchers have found that certain behaviours in the first 2 years of medical school can predict professionalism issues in clerkship: behaviours such as turning in course evaluations or being up to date on immunizations that some have referred to collectively as conscientiousness (McLachlan et al 2009, Stern et al 2005).

What to evaluate

There has also been debate about what exactly belongs in a definition of professionalism. As described above, there is no clear consensus on how to define professionalism, and therefore there is disagreement even among experts over what to include in an evaluation. As an example, the issue of communication, while certainly important for professionalism, is usually covered elsewhere or in separate evaluations entirely. The literature and research on evaluating communication skills do not often intersect with those on professionalism. The same is true of ethics and some other domains. Thus it is important when planning evaluation to consider what might be consistent with the programme’s conceptualization of professionalism but that might already be evaluated by other means.

Pitfalls to consider in evaluation

When evaluating professionalism one must consider a few important caveats, some of which apply to many areas of evaluation and some of which are particular to professionalism. In an article written over 10 years ago, for example, we argued that the assessment of professionalism should focus on behaviours rather than attitudes and should incorporate and acknowledge the importance of context, value conflicts and resolution (Ginsburg et al 2000). These issues are still important, and although some have been modified (based on new research) the basic principles are worth briefly revisiting.

The context in which behaviour occurs is extremely important to consider. We can all recognize that there is a time and a place in which some behaviours may seem appropriate and others in which the exact same behaviour would be viewed as inappropriate. Some important contextual influences include the micro-culture created by the clinical environment (e.g. the emergency department or operating room environments compared to psychiatry wards) and expectations and norms existing for different cultural or ethnic groups.

Similarly, when unprofessional behaviour arises it is often as the result of conflict between competing principles (each of which may be completely worthy), e.g. honesty versus confidentiality, individual patient advocacy versus stewardship of resources, efficiency versus comprehensiveness, and a limitless combination of any values. In evaluation, it is important to consider which values have come into conflict, under which particular context, and then to consider how and why the individual chose to act in the way he or she did.

Although we began with a call to focus on behaviours, as they seem more objective, it is important to note that research in other domains has provided abundant evidence that an individual’s behaviour, and how we interpret it, is largely shaped by environmental factors (Regehr 2006, Rees & Knight 2007). With this understanding it has been argued that we should modify our approach to evaluating professionalism, and in particular in remediating it. It no longer makes sense to evaluate an individual as if he or she acted in a vacuum; rather, we should acknowledge the factors that shape these behaviours and work to create environments that foster, rather than hinder, professional behaviour (Hodges et al 2011, Lesser et al 2010, Lucey & Souba 2010).

That said, programmes still have to evaluate individuals. What follows is a brief overview of commonly used evaluation methods that are well-described in the literature. A full explication of each would be beyond the scope of this practical chapter; however, for the interested reader, several review articles, chapters and books have been published that review these in much greater detail (Lynch et al 2004, Stern 2006, van Mook et al 2009). There is also an excellent review by Clauser and colleagues (2010) of how to consider – and build – evidence for validity when it comes to evaluating professionalism.

It is well worth considering a final word of advice from the Ottawa consensus group: that it may be ‘more important to increase the depth and quality of the reliability and validity of a programme’ of existing measures in various contexts ‘than to continue developing new measures for single contexts’ (Hodges et al 2011). In other words, use what’s ‘out there’ already and modify if needed, rather than building a new assessment from scratch.

Common methods of evaluation

In-training evaluation reports (ITERs)

ITERs and their variants (ward evaluations, faculty evaluations, etc.) are the mainstay of evaluation in most educational settings. Their advantages include familiarity, feasibility and comprehensiveness. However, they are also plagued by issues of unreliability and have not often been shown to be valid. Some issues are related to the fact that these evaluations are meant to be based on observation, yet it is widely recognized that the evaluators often do not directly observe performance; rather, they observe snippets of performance and then extrapolate, or use information from other sources (such as residents, students, nurses, etc.) (Clauser et al 2010, Mazor et al 2008).

Another problem is that they are often filled out at the end of a rotation and thus suffer from recall bias. Often the elements of professionalism are explicit on evaluation forms, but it can be difficult to decide whether a particular behaviour (e.g. poor communication with allied health) is an issue of professionalism, communication, collaboration or some combination (as modelled on the CanMEDs roles). Thus documentation can be inconsistent.

Another significant issue is the fact that clinician attendings and supervisors have repeatedly been shown to be reluctant to provide constructive or negative feedback – on any issue, in fact – but particularly on professionalism. This is despite every good intention to do so. Faculty report feeling unprepared, uncomfortable, unsure (of what they are witnessing) and, in many cases, unwilling (due to the difficulty involved, lack of support, fear of appeals or reprisals, etc.) (Burack et al 1999, Cleland et al 2008, Dudek et al 2005). These issues should not be ignored, but should be addressed with proper education, faculty development and training.

Despite these issues, there is some evidence in the literature of the utility of ITER type evaluations in picking up issues in professionalism (Colliver et al 2007, Frohna & Stern 2005, Stern et al 2005). It is also worth noting that the comments on these forms can be an even more fruitful source of information although they are not often analysed or reported systematically (Frohna & Stern 2005, Ginsburg et al 2011).

 Evaluations are more reliable and valid when based on direct observation of trainees and when rating scales are tailored to relevant performance.

Only gold members can continue reading. Log In or Register to continue

Share this:

• Click to share on Twitter (Opens in new window)
• Click to share on Facebook (Opens in new window)

Like this:

Like Loading...

Related

Related posts:

1. Clinical skills centre teaching
2. Postgraduate training
3. Instructional design
4. Basic sciences and curriculum outcomes

Stay updated, free articles. Join our Telegram channel

Join