Preparing for portfolio assessment in art and design: a study of the opinions and experiences of exiting secondary school students in Canada, England, and The Netherlands

 

 

Fiona Blaikie, Diederik Schönau and John Steers

 

 Abstract

 

This study utilises survey questionnaires to compare 107 Canadian, English and Dutch students’ opinions and experiences of portfolio preparation for final assessment in the terminal year of secondary school.  The aim is to reveal what students value about portfolio assessment and if they see portfolio assessment as a valid preparation for their futures, particularly for those who plan to continue on studying art and design at college or university. Common approaches to assessment are examined, followed by a more focussed discussion of curriculum and assessment practices in all three countries at the time of writing (March 2004). This is followed by a description of methodology, tentative findings are presented and the paper concludes with a short discussion of some implications for art and design education.

Portfolio assessment in Canada, England and The Netherlands

Many art educators continue to support a qualitative approach to assessment utilising portfolios. [[i]] A portfolio can be defined as a focussed collection of pieces of visual art and design, often accompanied by reflective and explanatory written data.  Krueger and Wallace observe that a portfolio is usually constructed for an explicit purpose and there can be a contrast between product and process-based approaches. Furthermore, ‘… A major decision relating to the purpose of a portfolio is whether it will contain students’ best pieces of work or whether it will be developmental’ [[ii]]. Portfolio collections are assessment instruments for studio art and design in Canada and The Netherlands, while the ‘coursework’ portfolio is a significant element of external assessment in England. 

Teacher subjectivity and external examining

            In assessing art and design portfolios, subjectivity and potential negative or positive bias on the part of teachers is considered a problem.  This has resulted in external assessment, in which the judgement of an examiner, also known as a moderator, verifier or reader, tempers that of the classroom teacher.  Kárpáti et al [[iii]] refer to this as the ‘jury method’ of assessment. The jury method is in place in England and The Netherlands, with external assessment usually involving examiners or moderators visiting schools to review final portfolios.  It is not a common national or provincially mandated practice in Canada, with the exception of those schools participating in international assessment procedures such as the International Baccalaureate [[iv]] and Advanced Placement [[v]].

Group critiques and self-assessment

            Often, students’ achievements are assessed in terms of concrete art and design products as well as work habits, understanding of art concepts and the ability to be critically reflective and to self assess in written and spoken forms.  Examples of this would include the portfolio and research workbooks in the International Baccalaureate (2001), and the English ‘Critical and Contextual Studies’ approach [[vi]] integral to the General Certificate of Secondary Education (GCSE) and General Certificate of Education Advanced and Advanced Subsidiary (GCE A/AS) level examinations in England [[vii]].

             The concern in portfolio assessment with analysis and interpretation, reflection, self critiquing, and selecting examples from one’s work is important because it reveals ‘…students’ beliefs about what constitutes learning and what constitutes good work’ [[viii]].  Mitchell [[ix]] and Hall-Jones [[x]] examine how learning takes place by means of conversations between students and instructors in focussed critiques or ‘crits’.  Mitchell posits that crits help to form artistic identity and eventual enculturation into the fine art world.

            Curriculum policy in Canada, England, and The Netherlands requires teachers to engage their students in reflective critiques of art and design, including their students’ own art and design work.   The ability to reveal process is important to understanding the genesis of final art and design products, and therefore reveals much about the student’s thinking, work habits, effort and progress, as well as facilitating the student’s ability to critically self-assess by reflecting on their work process.  Opening up the process of working in art and design is enhanced and buttressed by the inclusion of preparatory research, reflective notes and development of ideas.  In this regard, Dalton [[xi]] writes that currently there is a move towards a  ‘flattening’ of power relationships, where the student’s opinion about his or her work is taken into account, and where assessment is negotiated between student and teacher.   Certainly disclosing process and engaging in self-reflection and critiques would facilitate self-assessment and negotiated assessment – but the latter does not yet characterise art assessment in any of the countries in this study. 

            There now follows a necessarily brief discussion of the curriculum and assessment practices in each country in place at the time of writing.       

 

Canada

            In Canada the provinces administer art assessment, with internal control over curriculum implementation in the hands of regional school boards.  Across the country high school art teachers generate their own criteria for studio art assessment, with some following provincial guidelines.  However Gray and MacGregor [[xii]] and Blaikie’s [[xiii]] studies reveal that there are significant similarities in teacher expectations, defined criteria and the assessment strategies adopted, suggesting that art teaching practice in Canada is more similar from school to school than it is different.

            Funding for education in Canada is determined by the political persuasions of provincial governments.  To take one example, the Conservative government in Ontario from 1995 to 2003 aimed to impress the public with an allegedly more rigorous approach to curriculum, through demanding greater teacher accountability and introducing a form of teacher testing. Since 1998, new curriculum guidelines began to appear in every subject, and at every grade level, which emphasise a more standardised and formalised approach to curriculum content, teaching and assessment.  Moreover, since 2000, children in Ontario have been assessed province-wide in grades three, six and nine in language and mathematical skills.  It is significant that the arts were never included in Ontarian provincial assessment.  In late 2003 the Liberal Party came to power in Ontario, immediately stating that they would scrap teacher testing and they are no doubt planning to bring further changes to educational practice in the province.

            The situation in Ontario appears to reflect a common situation worldwide. It has to be recognised that curriculum and assessment are politically and socio-culturally driven: what is taught and assessed is normally determined more by politicians or religious leaders than by educators. 

            In three provincial curriculum guides – those of Ontario, British Columbia, and Nova Scotia – the defined curriculum content areas, assessment and evaluation in art differ as shown in Figure 1.

 

 

 

 

 

 

 

Nova Scotia

Ontario

British Columbia

MAKING ART

1.Creative/Productive

1.Creative Work

1.Image Development and Design

2.Materials, Techniques and Processes

(Creating and Communicating Perceiving and Responding)

THEORIES OF/LEARNING ABOUT ART

2.Cultural/Historical

2.Knowledge of Elements

3.Elements and Principles of Design

4.Context

(Creating and Communicating Perceiving and Responding)

REFLECTING

RESPONDING

ART CRITICISM

3.Critical/Responsive

3.Critical Thinking

(Creating and Communicating Perceiving and Responding)

ASSESSMENT AND EVALUATION APPROACHES

-Assessment and evaluation; evaluation dominates; criteria for assessment not related explicitly to curriculum content areas defined above

-Assessment dominates; criteria related to curriculum content areas defined above

Assessment and evaluation; evaluation dominates;  criteria for assessment not related explicitly to curriculum content areas defined above

Figure 1: Comparison of approaches to curriculum and assessment in Nova Scotia, Ontario, and British Columbia (March 2004)

 

            Similarities between the provinces lie in the definitions of curriculum content: each identifies studio production, (creative work or image development) as a specific area of curricular focus. Theories of art or concepts are also defined, although in Nova Scotia the focus is on the historical/cultural, while in British Columbia and Ontario historical and contextual studies are combined with a focus on the formal elements and principles of design.  All three include a curricular component focusing on art criticism or responding to art.  In British Columbia perceiving and responding is a sub theme of all four curriculum content areas.

            In Ontario, assessment is linked specifically to the three defined curriculum content areas.  Overall levels of achievement are outlined, with an emphasis on criterion-referenced assessment of finished products.  In British Columbia and Nova Scotia, assessment is not linked explicitly to defined curriculum content areas.  Rather, links are implied within new criteria that must be defined by the teacher specifically for each project.  Although the terms assessment and evaluation are both in place, the teacher’s overall evaluative impression seems to dominate in British Columbia and Nova Scotia rather than more formal assessment arrangements.             

            These variations by region and by province characterise art education curricula and assessment in Canada; by contrast, national examinations for art can be found in The Netherlands [[xiv]] and throughout the United Kingdom.

England

            It should be noted that in this small-scale study, data was only collected from English students, hence the emphasis on England throughout this paper.

            The National Curriculum statutory Order for Art in England, first introduced in 1992 and subsequently twice modified, presented a rare opportunity to consider the philosophy, purpose and content of the subject from first principles.  However, what emerged was a far from radical conceptual framework that essentially codified an existing tradition.  In its latest version, introduced in 2000, a single attainment target, 'Knowledge, skills and understanding' has four strands.  Strand (1),  'Investigating and making art, craft and design', strand (2) 'Exploring and developing ideas' and strand (3) 'Evaluating and developing work'.  The remaining strand, (4) 'Knowledge and understanding' is expected to inform all these processes.

            The General Certificate of Secondary Education (GCSE) examinations were introduced in the mid-1980s and marked the beginning of many profound changes in the British education system.  The GCSE is administered nationally by three awarding bodies in England, one in Northern Ireland and another in Wales. (Scotland has a different examination system.) The optional GCSE Art and Design examination is taken each year by approximately 190,000 students aged about sixteen, who may or may not continue on to the pre-university, GCE Advanced (A) level or Advanced Subsidiary (AS) art and design examinations. Approximately 52,000 take the AS level examination at age 17 without necessarily continuing to the full A level a year later although roughly 32,000 candidates do so. In addition there are a range of ‘vocational’ qualifications in art and design for 16-19 year-olds.

            Although the statutory curriculum for art and design ends at age 14, the Qualifications and Curriculum Authority (QCA) – a non-departmental public body accountable to the Department for Education and Skills (DfES) – defines the over-arching specification (syllabus) and assessment objectives for the GCSE and GCE examinations. [[xv]] In this national system of art and design assessment, Steers suggests that:

…the past twenty years has seen a trend away from holistic assessment without published criteria, to a schematic process in which marks are aggregated based on published assessment criteria...Teachers now teach to specific criteria in the knowledge that if pupils provide evidence of engagement with these criteria they will be rewarded regardless to some extent of the actual quality of their work. Of necessity, assessment criteria are drafted to be generic, applicable to a wide range of specialisms and activities.  As a consequence they are not easy to apply to unorthodox outcomes and tend to inhibit those responses that engage with contemporary practice. [[xvi]] 

 

            In September 2000, in great haste and with insufficient trials, the government introduced new GCE A level examination. All subjects are now based on a modular system where three units of work are assessed for the Advanced Subsidiary (AS) qualification and a further three units of work for the full GCE A level (A2).  This has created new problems for art and design – one concern is greatly increased workload for students and teachers alike, while another is more philosophical and concerned with the effect of the examination on teaching and learning. [[xvii]] It seems that the government perceived the first year of the old A level as a time for relaxation and therefore it was necessary to prescribe greater student productivity.  However they were mistaken.  In art and design that year was often a time for student experimentation, a time to try out new ideas and for teachers to support individual creativity by making students less dependent on teacher-led projects.  This is threatened by the new exam because, in effect, every project, every piece of work from the start of the course needs to be fully realised and of the required standard.  In effect, the message is don’t bother being creative, avoid risks, play safe, do what is expected and what your teacher tells you to do.  This is damaging to students and the discipline. But three years on, following controversy about the new system, even the government seems less than satisfied and the whole English curriculum and qualifications system for 14-19 year-olds is once again under review. [[xviii]]

The Netherlands

            The situation regarding art education at Dutch secondary schools is both complicated and confusing, as a consequence of political indecision and different approaches for the school types involved. VWO schools are concerned with pre-university accreditation and preparation, while HAVO schools prepare students for the senior secondary general education school leaving qualification, which permits entry to polytechnics (where art and design is located) but not to university (where art and design courses do not feature).

            Limiting this discussion to the two senior school types there is a range of art subjects. There are four ‘old’ art subjects (two-dimensional art, three-dimensional art and textile art, and also music), and a couple of ‘new’ art subjects, known by their acronym CKV 2 and CKV 3 (cultural and artistic education). CKV2 relates to the history and theory of all art forms (western art only), while CKV3 offers students a choice from a series of practical art activities: studio art work, making music, performing dance or drama.  CKV2 and CKV3 can be taken separately or combined.

            There is a Central Practical Examination (CPE) only for the ‘old’ visual art subjects at the VWO level.  The Studio portfolio is theme based, process oriented, allows a lot of student choice, and is externally examined. This examination amounts to 25 percent of the overall mark given to the subject. The remaining 75 percent is based on: Studio work assessed by the teacher (25 percent); art appreciation assessed by the teacher (25 percent) and a Central Written Examination on arts appreciation. This examination is also theme based and it is the same for all the visual art disciplines.  

            The new subjects CKV2 and CKV3 have their own examination. The CKV2 examination is the same for students in all art disciplines (including music and the performing arts) – thus it is broader in character and more related to historical contexts. Examples of examination topics are seventeenth century Dutch civic culture; religious culture in the late Middle Ages in Europe, mass media and the arts in the late twentieth century.   Technically it is a written exam, but it is presented by way of a CD-ROM produced under the auspices of the National Examination Board and sent free to all schools. This medium provides new opportunities to introduce questions by way of video or musical fragments, or by photographs and reproductions. The CKV2 is formally a school examination, and the use of the CD-ROM exam is not compulsory, although in practice all schools use it. CKV3 for the visual arts is concerned with studio work, with some subject specific theory on techniques and materials.  CKV3 is assessed by the students' own teacher.

            With HAVO the situation is comparable to that at VWO, with the exception that for the ‘old’ subjects there is no CPE. All studio work assignments are provided and assessed by the student’s own teacher.  In HAVO the Central Examination is limited to art appreciation, and is the same irrespective of the three ‘old’ studio areas of drawing, handicraft and textile art. For CKV2 at HAVO level, the situation is the same as for VWO.  For all HAVO exams the Central Examination counts for 50 percent of the final mark, with the other 50 percent derived from teacher assessed studio work. The HAVO examination has no theme and is of a general character.

            Schönau suggests that many teachers in The Netherlands still oppose the new CKV2 and CKV3 system, disliking the combined arts approach and the split in art and design between theory and practice.  Because of lack of acceptance by schools and teachers, the Ministry has been unwilling to move forward.  In The Netherlands, the arts in schools are considered by the authorities to be specialised and talent-based – an argument for not making the arts compulsory in one of the four ‘profiles’ (groups of compulsory subjects) for which students at HAVO and VWO have been able to opt since 1998.  The arts are part of the Cultural and Social subjects profile that includes mathematics, history and four subjects selected from philosophy, geography, economics, social science, the arts, and classical and modern languages.  However is also possible to take the arts as an extra subject in addition to the core curriculum in all profiles.

            All in all, the situation is confusing but does not endanger arts provision. The future of the central examination system is currently under review and it now seems likely that the CPE will be abolished. But, surprisingly, the CPE has been introduced recently at lower vocational education (MAVO) where the first studio exams were taken in 2003. 

Method

             The aim of our study is to compare Canadian, English and Dutch students’ opinions and experiences of portfolio preparation in art and design for final assessment in the terminal year of secondary school.   We collected the data via survey-questionnaires based on a five-point Likert scale, ranging from ‘strongly agree’ to ‘strongly disagree’ in the case of seeking students’ opinions about assessment, and ‘highly accurate’ to ‘highly inaccurate’ in the case of students’ actual experiences of assessment.  

            The data refer to survey questionnaire responses from 107 students, of whom 17 (15.9 percent) did not indicate their gender, 26 (24.3 percent) were male and 64 (59.8 percent) were female. Of the students 29 (27 percent) came from The Netherlands, 17 (15.9 percent) from England, and 61 (57 percent) from Canada. 

            We chose to design questions around opinions separately from actual experiences because we wanted to distinguish between what students believe best practices in assessment should be like, and what they actually have experienced.  We mirrored questions in both categories, so that, for example, in Section A of the questionnaire we asked for their opinions on whether it is important to understand criteria for assessment.  In Section B the question re-appeared.  This time, we asked students if they do in fact understand the criteria used to assess them. Data inputting resulted in detailed spreadsheets and deductive analysis thereof.  In Canada and The Netherlands survey questionnaires were distributed to art and design students in the final year of secondary school.  In England, they were distributed to students on GCE Advanced level courses. 

            International Comparison of Students’ Opinions about Assessment

            The findings that follow below focus only on responses that reveal either significant national differences or points of agreement across all three countries.

a. Group critiques

            The statement ‘In my opinion, doing regular groups critiques of studio work (group crits) with my fellow students and teacher(s) should be an important part of the experiences of learning in art and design’ yielded some significant differences (see Table 1).

           

 

Canada

England

The Netherlands

Strongly Agree

11.7%

58.8%

44.8%

Agree

48.3%

35.3%

48.3%

Undecided

30%

5.9%

0

Disagree

10%

0

6.9%

Strongly Disagree

0

0

0

Table 1: Students’ opinions regarding the importance of group critiques of studio work

            While the numbers even out a little in the ‘agree’ category, no students in England disagreed with this statement and only 5.9 percent were undecided, 6.9 percent disagreed in The Netherlands and none were undecided. Group critiques are valued by only 60 percent of students in Canadian high schools but, by contrast, almost 100 percent of Dutch and English students believe group critiques are important.

b.         The importance of external examination/moderation

            In response to the statement ‘I think it is important that my portfolio be assessed by more than one person for the final assessment to minimise subjective bias’, there was quite a contrast between the different countries (see Table 2). We suggest that these findings relate to the fact that moderated assessment is practised commonly in England and The Netherlands, and less so in Canada.  In Canada moderated assessment typically is associated with international programmes such as the International Baccalaureate and Advanced Placement.  The responses may indicate relative levels of concern about how students perceive the fairness of assessment in each country.

 

 

Canada

England

The Netherlands

Strongly Agree

54.1%

82.4%

69.0%

Agree

27.9%

5.9%

24.1%

Undecided

14.8%

11.8%

3.4%

Disagree

3.3%

0

3.4%

Strongly Disagree

0

0

0

Table 2: Students’ opinions of the importance of external examination/moderation

            c.         The importance of ‘luck’ in assessment

            No students in The Netherlands or England strongly agreed with the statement ‘In my opinion, it is only a matter of luck whether assessor(s) like a particular portfolio’ while 14 percent of Canadian students did strongly agree (see Table 3). 

 

 

 

Canada

England

The Netherlands

Strongly Agree

14.8%

0

0

Agree

8.2%

5.9%

3.4%

Undecided

32.8%

23.5%

13.8%

Disagree

32.8%

52.9%

51.7%

Strongly Disagree

11.4%

17.6%

31.0%

Table 3: Students’ opinions on the importance of luck in assessment

            The data suggests that the Dutch students most strongly see assessment results as related to hard work rather than luck, followed by the British.   A large majority of Canadians appear to believe more or less strongly that assessment is more to do with luck and the subjective preferences of the teacher assessing their work. Again, the implication is that Canadian students have more doubts than their European peers do about the fairness of the assessments they receive.

d.         The importance of teachers hearing students’ views and opinions of their own work

           

 

Canada

England

The Netherlands

Strongly agree/agree

86.9%

100%

93.1%

Undecided

11.5%

 

3.4%

Disagree/strongly disagree

1.6%

 

3.4%

Table 4: Students’ opinions about the importance of teachers hearing their views on their own art   work.

            It is evident that most students believe that teachers should be keen to hear students’ views and opinions of their own art work (see Table 4).                         

e.         Discussion of one another’s art  work 

            More students in England and The Netherlands than in Canada agree or strongly agree with the statement that teachers should expect fellow students to express opinions about each others work in group critique sessions (see Table 5).

. 

 

Canada

England

The Netherlands

Strongly agree/agree

68.8%

88.3%

82.8%

Undecided

24.6%

5.9%

6.9%

Disagree/strongly disagree

6.5%

5.9%

10.3%

Table 5:  Students’ opinions on discussing peer art work

            In light of the data outlined in tables 5 and 6, it is very evident that the great majority of students in all three countries want to discuss their portfolio work regularly with their teachers.  In addition, overwhelmingly in England and The Netherlands, and to a slighter lesser extent in Canada, students believe that teachers should be interested in hearing students express their views about their own art work.
            Again, familiar practice probably drives opinion: those students who experience peer review through group critiques on a very consistent basis in England and The Netherlands are more supportive of it than those students in Canada where group critiques are not so widespread.

f.          Reflective writing on their own art work

            Students were less supportive of the statement that they should be required to write reflective self-critical comments about the quality of the work in their art portfolios (see Table 6). 

 

 

Canada

England

The Netherlands

Strongly Agree

13.1%

0

10.3%

Agree

34.4%

52.9%

34.5%

Undecided

29.5%

5.9%

34.5%

Disagree

14.8%

41.2%

13.8%

Strongly Disagree

8.2%

0

6.9%

Table 6: Students’ opinions on writing reflectively about their own art work

            Significantly, and surprisingly, despite the curricular emphasis on critical and contextual studies, a relatively high number of English students disagreed with this statement.

 

 g.        The importance of portfolio preparation as a useful and worthwhile learning experience

            Not surprisingly, in all three countries there was significantly strong support among students for the idea that preparing a portfolio should be a useful experience (see Table 7).

 

 

Canada

England

The Netherlands

Strongly agree/agree

90.1%

93.1%

88.3%

Undecided

6.6%

6.9%

11.8%

Disagree/strongly disagree

3.3%

0

10.3%

Table 7: Students’ opinions on whether portfolio preparation should be a useful experience

            Moreover, there was strong support for the idea that portfolio preparation should be a worthwhile learning experience (see Table 8).

 

 

Canada

England

The Netherlands

Strongly agree/agree

91.8%

100%

79.3%

Undecided

4.9%%

0

20.7%

Disagree/strongly disagree

3.2%

0

0

Table 8: Students’ opinions on whether portfolio preparation should be a worthwhile learning experience

h.         High school art and design as a good foundation for further study in art and design

There was strong agreement among students in England and Canada that their courses in art and design should provide a good foundation for further study (see Table 9). However fewer Dutch students agreed with this statement probably because they tended to opt for the subject because they enjoyed it, rather than seeing it as a first step towards a career in art and design.

 

 

Canada

England

The Netherlands

Strongly agree/agree

86.7%

88.3%

62.1%

Undecided

10.0%

5.9%

24.1%

Disagree/strongly disagree

3.3%

5.9%

13.8%

Table 9: Students’ opinions on whether high school art and design should be a good foundation for further study in art and design

 

International comparison of students’ experiences of assessment

a.         Group critiques

            The statement ‘My fellow students and I engage with our art teacher in group critiques (group crits) on a regular basis’ yielded the data in Table 10. 

 

Canada

England

The Netherlands

Highly accurate

1.6%

5.9%

13.8%

Usually accurate

31.1%

29.4%

37.9%

Undecided

29.5%

29.4%

0

Accurate sometimes

14.8%

35.3%

34.5%

Highly inaccurate

23.0%

0

13.8%

Table 10: Students’ experiences: Do they engage in group critiques?

 

            A significantly high number of students in both England and Canada are undecided on this topic and it is clear that the practice of group critiques is far from universal. The data suggest that more students engage in group critiques in The Netherlands, followed by England, with the least number in Canada – almost a quarter of Canadian students claim never to engage in group critiques. This probably reflects curriculum policy and teaching practice in England and The Netherlands, where reflective criticism is promoted to a greater extent than in Canada.

b.             Knowing the qualities teachers are looking for in their work

            English students seem to have the clearest knowledge about what qualities their teachers are looking for in their work with 82.3 percent claiming the statement ‘I know what qualities my (art) teacher(s) are looking for in my work’ was accurate or highly accurate (see Table 11).  While a high number also concurred in Canada, only 32.1 percent of Dutch students described this statement as highly accurate or usually accurate.

 

Canada

England

The Netherlands

Highly accurate

26.2%

17.6%

7.1%

Usually accurate

47.5%

64.7%

25.0%

Undecided

16.4%

11.8%

10.7%

Accurate sometimes

8.2%

5.9%

42.9%

Highly inaccurate

1.6%

0

14.3%

Table 11: Students’ experiences of knowing the qualities teachers are looking for in their work

           

c. Student input into criteria for assessment

            In response to the statement ‘I definitely have some say about the criteria used to assess my portfolio’, 44.2 percent of Canadian students declared this to be accurate or highly accurate compared with 5.9 percent in England and 6.8 percent in The Netherlands (see Table 12). 

 

 

Canada

England

The Netherlands

Highly accurate

9.8%

5.9%

3.4%

Usually accurate

34.4%

0

3.4%

Undecided

29.5%

17.6%

17.2%

Accurate sometimes

8.2%

17.6%

24.1%

Highly inaccurate

18.0%

58%

51.7%

Table 12: Students’ experiences of having input into criteria for assessment

 

            The 58.8 percent of students in England and 51.7 percent in The Netherlands who describe this statement as highly inaccurate confirms that English and Dutch students have little or no input into criteria generated, at least for external assessment, because these are nationally determined by government agencies.

d.         Luck as a factor in assessment

            With regard to students’ actual experiences of assessment, the statement ‘Luck best describes whether the assessor(s) will like my portfolio’, resulted in the following data (see Table 13):

 

 

Canada

England

The Netherlands

Highly accurate

13.1%

11.8%

7.4%

Usually accurate

19.7%

0

3.7%

Undecided

26.2%

5.9%

7.4%

Accurate sometimes

24.6%

29.4%

37.0%

Highly inaccurate

16.4%

52.9%

44.4%

Table 13: Students’ experiences of whether luck determines assessor preferences

           

            In England and The Netherlands students perceive ‘luck’ as having less to do with success in portfolio assessment than in Canada where students’ responses were spread across all categories with a significant number being undecided – 26.2 percent. Again, assessment strategies in Canada are more idiosyncratic, and determined by individual teachers; moderation is not carried out unless students are enrolled in AP or IB programmes.  The opposite is true in England and The Netherlands, where assessment is more systematic, and where some form of moderation of internal teacher judgements does take place.

e.         The usefulness of portfolio preparation

            More students in England and Canada concur that the actual experience of putting together their portfolio is likely to be useful compared with those in The Netherlands – no English or Canadian students disagreed with this statement (see Table 14).  However, 21.4 percent Dutch students claimed the statement was highly inaccurate, and another 21.4 percent were undecided: the Dutch students appear to have the least faith in the usefulness of portfolio preparation.

 

 

Canada

England

The Netherlands

Highly accurate

44.3%

76.5%

14.3%

Usually accurate

32.8%

17.6%

42.9%

Undecided

19.7%

5.9%

21.4%

Accurate sometimes

3.3%

0

0

Highly inaccurate

0

0

21.4%

Table 14: Students’ experiences of the usefulness of portfolio preparation

 

 

f.          High school art courses as a good foundation for further study in art and design   

            In England 82.3 percent and in Canada 82 percent of students claimed that the statement ‘My present course will be a good foundation for further study in art and design’ was highly or usually accurate, compared with a combined total of only 51.7 percent in The Netherlands (see Table 15). It appears that Dutch students are much less satisfied overall with the usefulness and long-term applicability of their high school art and design studies than their English peers.

 

 

Canada

England

The Netherlands

Highly accurate

37.7%

58.8%

17.2%

Usually accurate

44.3%

23.5%

34.5%

Undecided

13.1%

11.8%

31.0%

Accurate sometimes

3.3%

0

13.8%

Highly inaccurate

1.6%

5.9%

3.4%

Table 15: High school art courses as a good foundation for further study in art and design

                         

g.         Satisfaction with opting to study art and design

            Across all countries, students claimed to be generally satisfied that they opted to study art and design, responding to the statement ‘I am really pleased that I opted to study art and design’ (see Table 16) with 100 percent identified the statement as highly or usually accurate in The Netherlands. The English and Dutch students seem to be more satisfied overall than the Canadians with opting to study art and design.

 

 

Canada

England

The Netherlands

Highly accurate

54.1%

88.2%

86.2%

Usually accurate

31.1%

0

13.8%

Undecided

13.1%

5.9%

0

Highly inaccurate

1.6%

5.9%

0

Table 16: Students pleased that they opted to study art and design

Implications for curriculum and assessment

            It is clear from the data that students’ actual lived experiences influence their opinions about assessment, as well as their positive or negative responses to questions in the ‘experiences of assessment’ category.  In other words, if they are used to a particular practice, such as moderation, they tend to support it and find it worthwhile.  Students appear to reflect the national culture in their responses rather than great objectivity – not entirely surprising perhaps, but still a little disconcerting.

            In Canada, unless teachers are preparing students for IB or AP assessment, the interpretation of the curriculum is at the discretion of individual teachers.  The same is true for assessment where criteria tend to be defined by the class teacher, project by project.  With the exception of IB and AP participants, there is no formal external assessment, and therefore no outside moderation or examination.  (However, it should be noted that about half of the Canadian participants in this study are IB or AP students, which means that the results do reflect more formal views on assessment than might be the norm across all of Canada).   Conversely, in The Netherlands and England there is national control over curriculum and assessment; criteria for assessment are set, and there is external examination or moderation.

Opinions

            In England and The Netherlands there is evidence of stronger student opinion in favour of art and design assessment practices such as group critiques, peer group discussions and external examination or moderation.  Students in The Netherlands, followed by those in England, were less convinced than their Canadian counterparts that luck plays a part in assessment. In contrast, a significant number of Canadian students – almost a quarter – believe that luck does play a part in the assessment of their portfolios, probably because the assessment criteria are neither standardised nor often readily available.  There is widespread student agreement across all three countries about the importance of teachers hearing students’ views on their art work, about portfolio preparation being a useful and worthwhile learning experience, and a good foundation for further study.

Experiences

            In relation to actual experiences of assessment, the trend continues: more than half the respondents in The Netherlands claim they actually do participate in group critiques, while the fewest claim this to be true in Canada, with almost a quarter of Canadian students describing this statement as inaccurate.   The highest number of students agreeing that they know what qualities their teachers are looking for in their work are English (81 percent) followed by the Canadians (73 percent) and Dutch (32 percent) who seem to be the most unclear and confused, in spite of a national assessment system being in place.  More Canadians have input into developing criteria for assessment with their teachers, followed by Dutch and English students.  This is not surprising in light of set criteria in the latter two countries.   In response to the experience-based question as to whether luck actually does play a part in assessment, 32.8 percent of Canadians agree that it does, while only about ten percent of Dutch and English students concur.  This suggests that Canadians view assessment as more unreliable and unfair, and not a good reflection of their work and effort.

            There is widespread support for studying art and design and for portfolio preparation in general.  Almost all the English students agree that this was a useful experience (94 percent) followed by Canadians (77 percent) and Dutch students (57 percent).   The same trend continues in responses to the question about whether their school art and design studies will be a good foundation for further studies in the same field, with 82 percent of English and Canadian students agreeing, followed by 51 percent of Dutch students.  Finally, reversing this trend, 100 percent of Dutch students agree that they are ‘really pleased’ they chose to take art and design as a subject, followed by Canadian students (85 percent) and English students (82 percent) – statistics that may interesting but unresolved questions remain about the impact of assessment regimes on student attitudes to learning.

Implications for art education

            This study commenced by describing national approaches to art education and assessment in Canada, England, and The Netherlands.  It was asserted that curriculum and assessment are politically and socio-culturally driven, rather than educationally driven. It is readily acknowledged that our findings must be viewed as tentative, if only because the number of participants in this survey is low. Further research is needed to confirm the validity and reliability of this study, but it is probable that some overall trends are revealed in the data.

            Some significant gender differences in the students’ responses to our questionnaire have been discussed elsewhere. [[xix]] In summary, it appeared that it is less important to males than females:

1.       to know and understand what qualities the teacher is looking for in their portfolio work; 

2.       to know and understand the specific criteria used for assessment;

3.       to regularly discuss their art work portfolios with their art teacher; and, finally,

4.       to engage in group critiques of art work with their teacher and peer group.

However taking a non-gender specific overview it seems at first sight that Dutch students are more confused about assessment, less aware of the qualities teachers are looking for, and more unsure about the usefulness of it than their English and Canadian peers.   However this may be because, although the criteria are well known beforehand, Dutch students have very little insight about how much their work reflects the expected level. Knowing the criteria is different from knowing the expected level and how teachers will decide on the level of achievement demonstrated in a particular portfolio. Exactly because it is a national exam, Dutch students feel more insecure, because they are very aware that someone outside school will also give their opinion. It is unclear why this concern appears to be less common in England, where the strong tradition of national assessment in art and design education is reflected in mainly positive student views about the fairness of the system – other than, perhaps, because its strength derives from being a system dependent on internal assessment by the students’ own teachers, followed by external moderation. 

In Canada, the lack of national or provincial assessment practices is evident: Canadian students have experienced little or no moderation, fewer group critiques, and less awareness of clearly defined criteria for assessment.  As a consequence more Canadian students see assessment as less reliable and fair than their Dutch and English peers.  Nevertheless, the majority of students in this study are positive about portfolio preparation, about going on to study art and design further, and are really pleased they opted to take art and design.  Finally, it is clear that formative assessment, particularly group critiques in secondary schools and the opportunity to talk to their teachers about their work, is highly valued by students – as well providing an important initiation into the real life experiences of art students studying art and design at college and university levels.

            Steers has cautioned that in external assessment of art and design, creativity can be stifled by over prescription, standardised objectives and criteria for assessment. In practice a pervasive orthodoxy often threatens real creativity. [[xx]] Political decisions drive education; students’ opinions and experiences of art education are determined as much by these as by educational decisions. In all three countries, the values of art and design education, what kinds of art and design education are of most worth, and what forms of assessment may be most authentic, are discourses that must go beyond art educators. It is a debate that politicians, education bureaucrats and the wider public need to engage in as we seek to define curriculum and assessment models relevant to society’s and students’ needs in the new century.

Biographies

 

Fiona Blaikie is a painter who began her professional life as a high school art and English teacher in South Africa.  After completing a PhD at the University of British Columbia in 1992, she moved to Lakehead University, Ontario, Canada. Her research interests include curriculum, assessment, and gender.  Currently she is director of the joint PhD Programme in Educational Studies at Brock University, Lakehead University, The University of Western Ontario, and The University of Windsor.  She is also a visual art examiner in the International Baccalaureate programme. E-mail: Fiona.Blaikie@lakeheadu.ca

 

Diederik Schönau is a subject specialist in art education, and general consultant and trainer in educational assessment and evaluation at the Dutch National Institute for Educational Assessment Citogroep in Arnhem, the Netherlands. He is a psychologist and art historian who combines both fields of interest, specifically in the field of art education. He has been subject specialist for the visual arts at Citogroep for twenty years, where he produced final examinations in art education for all levels in secondary education. His research includes the assessment of studio work and he has published extensively on a variety of issues in art education, as well as in art history and the psychology of art. He was President of InSEA from 1999-2002. Contact address: Citogroep, P.O. Box 1034, 6801 MG Arnhem, The Netherlands. E-mail: diederik.schonau@citogroep.nl

 

John Steers became General Secretary of what is now the NSEAD in 1981 after fourteen years teaching art and design in secondary schools.  He was the 1993-96 President of the International Society for Education through Art and has been a member of its executive committee in several capacities. He has served on national committees and as a consultant to government agencies. He completed a PhD at the University of Liverpool in 1994 and has published widely on curriculum, assessment and policy issues. He is a trustee of the Higher Education in Art and Design Trust and the Chair of the National Arts Education Archive Trust, University of Leeds.  He is a visiting Senior Research Fellow at the University of Surrey Roehampton, London. E-mail: johnsteers@nsead.org

 

 Notes and references



[i]           See for example:

i.                     Advanced Placement  (1992) ‘Advanced placement course description in art, studio art, history of art’.  AP and The College Board: Princeton, New Jersey.

ii.                    Anderson, T. (1994) ‘The International Baccalaureate model of content-based art education’ in Art Education, Vol 47, No.2, pp. 124-129.

iii.                  Beattie, D.K.  (1994) ‘The mini-portfolio: Locus of a successful performance examination’,  Art Education, Vol 47, No.2, pp.14-18.

iv.                  Blaikie, F.M.  (1997) ‘Strategies for studio art assessment in Canada: Lana, Brenda, Sharon and Mark’, in Irwin, R., & Grauer, K., (Eds) Readings in Canadian art teacher education, Canadian Society for Education through Art: Toronto.

v.                   Boughton, D., Eisner, E., Ligtvoet, J (Eds)  (1996) ‘ Evaluating and Assessing the Visual Arts in Education’, Teachers’ College Press, New York.

vi.                 Boughton, D. (1994) ‘Evaluation and assessment in visual arts education’, Deakin University Press: Geelong, Victoria, Australia.

vii.                Steers, J. (1994) Art and Design: Assessment and Public Examinations’,  Journal of Art and      Design Education, Vol 13, No.3, pp. 287-298.

viii.                

[ii]           Krueger, B. & Wallace, J.  (1996) ‘Portfolio assessment:  Possibilities and pointers for practice’, Australian Science Teacher’s Journal, Vol 42, No. 1, p.26.

 

[iii]          Kárpáti, A., Zempleni, A., Verhelst, N., Velhuijzen, N. & Schönau, D. (1998), ‘Expert agreement in judging art projects: A myth or a reality?’, Studies in Educational Evaluation, Vol 24, No. 4, pp. 385-404.

 

[iv]          op.cit. note 1(ii): Anderson, T. (1994).

 

[v]           op.cit. note 1(i): Advanced Placement  (1992).

 

[vi]          See for example:

i.                     Taylor, R. (1986) ‘Educating for Art’, Longman: Harlow.

ii.                   Thistlewood, D. (Ed) (1989) ‘Critical Studies in Art and Design Education’, Longman: Harlow.

iii.                   

[vii]         Students in Wales and Northern Ireland also take versions of the GCSE and GCE A/AS level examinations but not those in Scotland which has its own distinct examination procedures.

 

[viii]         Ibid, note 2, p.27.

 

[ix]          Mitchell, S.  (1996) ‘Institutions, Individuals and Talk: The Construction of Identity in Fine Art’, Journal of Art and Design Education, Vol 15, No. 2, pp. 143-154.

 

[x]           Hall Jones, S.  (1996) ‘Crits – An Examination’, Journal of Art and Design Education, Vol 15, No. 2, pp. 133-142.           

 

[xi]          Dalton, P. (2001) ‘The Gendering of Art Education’, Open University Press, Buckingham.

 

[xii]         Gray, J. & MacGregor, R. (1987) ‘PROACTA: Personally relevant observations about art concepts and teaching activities’, Canadian Review of Art Education: Research and Issues, Vol 14, No. 2, pp. 13-33.

 

[xiii]        op .cit. note 1(iv): Blaikie, F.M.  (1997)

 

[xiv]         Schönau, D. (1994) ‘Final examinations in the visual arts in The Netherlands’, Art Education, Vol 47, No. 2, pp. 34-49.

 

[xv]         Full details of the National Curriculum for Art and Design as well as the specifications for art and design qualifications can be found by visiting the Qualifications and Curriculum Authority web site at http://www.qca.org.uk/

 

[xvi]         Steers, J. (2003) ‘Art and design in the UK: the theory gap’, in Addison, N & Burgess, L (Eds) Issues in art and design teaching, RoutledgeFalmer: London, p. 24.

 

[xvii]        See for example, Hardy, T. (2002) ‘AS Level Art: Farewell to the “Wow” Factor’, International Journal of Art & Design Education, Vol 21, No. 1, pp 52-59.

 

[xviii]       At the time of writing the ‘Working Group on 14-19 Reform: Interim Report’ had just been published. This may be accessed at http://www.14-19reform.gov.uk/

 

[xix]            Blaikie, F.; Schönau, D. and Steers, J. (2003) ‘Students’ Gendered Experiences of High School Portfolio Art Assessment in Canada, The Netherlands and England’, Studies in Art Education, Vol 44, No. 4, pp 335-349.

 

[xx]          Ibid, note 16, p. 25.