Portfolio assessment

Chapter 39


Portfolio assessment




Introduction


In less than two decades, portfolios have gained a prominent position in medical education. The portfolio owes its popularity to its suitability for attaining goals that are difficult to achieve with other educational methods: monitoring and assessing competency development and nontechnical skills, such as reflection. In this way the portfolio is in keeping with recent developments in education, such as outcome-based education and competency-based learning. This chapter discusses the lessons we have learned from 20 years of experience with portfolios in medical education. We will focus on four topics:



We will especially focus on the assessment of portfolios. In an earlier publication we focused on the portfolio as a coaching method (Driessen et al 2010). The assessment principles and strategies we will describe in this chapter not only apply to portfolios but have broader applications. They can also be used for other complex assessments, such as assessment of projects or papers, such as a master’s thesis.



image The objective of the portfolio, and consequently its format and content, can vary markedly.



The objectives and contents of portfolios


Learners, residents, doctors and teachers are regularly asked to compose a portfolio. The objective of the portfolio, and consequently its format and content, can vary markedly. We distinguish the following main objectives for portfolios:



• Guiding the development of competencies. The learner is asked to include in the portfolio a critical reflection on his or her learning and performance. The minimal requirement for this type of portfolio is the inclusion of reflective texts and self-analyses.


• Monitoring progress. The minimum requirement for this type of portfolio is that it must contain overviews of what the learner has done or learned. This may be the numbers of different types of patients seen during a clerkship or the competencies achieved during a defined period.


• Assessment of competency development. The portfolio provides evidence of how certain competencies are developing and which level of competency has been achieved. Learners often also include an analysis of essential aspects of their competency development and indicate in which areas more work is required. This type of portfolio contains evidential materials to substantiate the level that is achieved.



image Most portfolios are aimed at a combination of goals.


Most portfolios are aimed at a combination of goals and therefore comprise a variety of evidence, overviews and reflections (Fig. 39.1). Portfolios thus differ in objectives, and the objectives determine which component of the portfolio content is emphasized. Portfolios can also differ in scope and structure (Van Tartwijk et al 2003). Portfolios can be wide or narrow in scope. A limited scope is appropriate for portfolios aimed at illustrating the learner’s development in a single skill or competency domain or in one curricular component. An example is a portfolio for communication skills of undergraduate students. A portfolio with a broad scope is aimed at demonstrating the learner’s development across all skills and competency domains over a prolonged period of education. At the end of this chapter we will describe an example of this type of portfolio.



Portfolios also differ in the degree of structuring or guidance that is provided to the learner in composing the portfolio. This dimension is characterized by the contrast between an open and a closed portfolio. A closed portfolio has to comply with detailed guidelines and regulations and offers relatively little freedom to learners with respect to the format and content of the portfolio. As a consequence, portfolios of different learners are highly comparable and the portfolios are easy to navigate.



image Portfolios with a closed structure are highly comparable while portfolios with an open structure allow learners the opportunity to display their individual learning trajectories and competencies.


A more open portfolio gives general directions, but allows the learners freedom with respect to the actual contents and format of the portfolio. The framework is described in such a way that learners are given a choice about how they present their individual learning process and learning results. A uniform basic structure is nevertheless important, because teachers and peers who have to view several portfolios should be able to easily follow the general structure of the materials. An advantage of an open portfolio is that it offers learners the opportunity to reveal their individual learning trajectories and competency development.


We have explained that portfolios can contain overviews, evidential materials and reflection. We have located these elements on the corners of a triangle (Fig. 39.1). We will now focus on the format and place of each of these elements in the portfolio.



Overviews


When learners are following more distinct routes in a curriculum it becomes increasingly difficult for learners, teachers and supervisors to keep track of their progress. In order to facilitate this, many portfolios contain overviews to be used by learners to show what they have done, where they have done it, what they have learned as a result and how they are planning to proceed. Overviews may contain the following information:



• Procedures or patient cases. Which procedures? What was the level of supervision? Which types of patients? What was learned? Were the activities assessed? Plans?


• Prior work experience. Where? When? Which tasks? Strengths and weaknesses? Which competencies or skills were developed? Evaluation by the learner?


• Prior education and training. Which courses or programmes? Where? When? What was learned? Completed successfully? Evaluation by the learner?


• Experiences within and outside the course/programme. Where? When? Which tasks? What was done? Strengths and weaknesses? Which competencies or skills were developed? Evaluation by the learner? Plans?


• Components of the course/programme. Which attended at this point? Which not (yet) attended? When? What was learned? Completed successfully? Evaluation by the learner? Plans?


• Competencies or skills. Where addressed? Level of proficiency? Plans? Preferences?



Materials


Portfolios can contain a variety of of materials. We distinguish three different types:



The nature and diversity of the materials determine the richness of the picture learners present of their learning and progress. It can be tempting for learners to include a vast amount of materials, leaving it up to the assessor to determine their value. This not only increases the assessors’ workload, but also can mean that assessors are unable to see the wood for the trees. It is therefore important for learners to be selective. A good selection criterion is that materials should provide insight into the student’s learning and progress. We will discuss the size and feasibility of portfolios later in this chapter.



image Learners can use captions to indicate the context in which the material was collected or produced, its relevance and why it was included.



Reflections


Many portfolios contain reflections. These are often organized around the competencies to be demonstrated by the learner in the portfolio. They can relate to one or more strengths and areas requiring more work for one competency, or they can be extensive reflective essays describing the learner’s personal profile. The latter type of reflection can pertain to the learner’s motivation for attending the programme, the objectives the learner wishes to attain and/or the learner’s views of him- or herself as a doctor/professional. These reflections can be used by learners as a long-term agenda. Learners support their reflections by referring to materials and overviews in the portfolio. This is important not only to convince others of the validity of the reflections, but also to focus the reflections, because learners are likely to aim for consistency of reflections and evidential materials. This makes the reflections less noncommittal. It is, for instance, not acceptable for learners to simply state that they have learned how to give a clinical presentation. They will have to substantiate this statement by evidential materials and overviews demonstrating why and how they have done this.



Success factors for portfolios



image Mentoring is of the essence for portfolio use.


Despite the simplicity of the portfolio concept – a learner documents the process and results of his or her learning activities – the portfolio has proved to be not invariably and automatically effective. The literature on portfolios shows mixed results. The key question here is what makes a portfolio successful in one situation and less successful in another situation? A number of reviews have shed light on some key factors (Buckley et al 2009, Driessen et al 2007, Tochel et al 2009).


Related posts:

  1. Clinical skills centre teaching
  2. Postgraduate training
  3. Student support
  4. Basic sciences and curriculum outcomes

Stay updated, free articles. Join our Telegram channel
Tags:
Dec 9, 2016 | Posted by in GENERAL & FAMILY MEDICINE | Comments Off on Portfolio assessment

Full access? Get Clinical Tree
Get Clinical Tree app for offline access