Square pegs in round holes: What impact do outcomes and systems have on creative subjects within Higher Education?
Creativity, ILOs (Intended Learning Outcomes), education, policy, quality, standards
This article investigates how Intended Learning Outcomes (ILOs) impact on the outcomes and systems of creative subjects within Higher Education (HE) by examining the context of assessment matters within Higher Education (HE), the ILOs within this context and the subsequent effect this model has on the context of creative courses. Drawing on the current research of key theorists and policy makers such as the Quality Assurance Agency for Higher Education (QAA), further analysis is made of the university system requirements discussed and their active working role in creative subjects. It concludes by highlighting the necessary need for change to the ILO structure to better accommodate creative courses at university level.
Having been involved both as a student and tutor within creative subjects and HE I experienced first hand the effects of ILOs on the development of creativity. Therefore, I decided it would be invaluable to the future development of ILOs and the assessment model in general within creative courses to investigate the impact it was currently having on students, tutors and the administrative staff of more traditional established education institutions. By creative subjects I mean non-traditional subjects such as creative writing that are increasingly being brought into the mainstream of Higher Education.
This article commences with an introduction to the concept of ILOs and a pointed analysis of the
educational and ideological underpinnings of the changes that have taken place
as a result of their introduction to HE systems. This will examine the pressures that have impinged on Higher Education through the ill-fitting
use of ILOs in creative courses and the bureaucracy that constrains academics.
The Context of Assessment Matters in Higher Education
The issue of assessment matters within Higher Education institutions is a difficult one, due to both inner and outer pressures. Yorke’s (1991) model of the institutional evaluation process within the sub-system of higher education helps to highlight the socio-cultural agencies and factors that ‘impinge on the measurement of institutional effectiveness as demonstrated by its practices and procedures’ (Heywood, 2000, p.76). For example, outer pressures such as society and the government have a huge effect on how institutions conduct their assessment evaluations. Politicians need to have some understanding of the effectiveness of universities to justify funding, and in addition, the public requires assurance that such studies are taken. One such study, released annually, is the university league tables that are made readily available to the public through publication in newspapers as well as education journals. These league tables highlight such issues as ‘drop-out’ rates, which have a significant effect on how the university justifies its assessment procedures.
However, a high drop-out rate does not necessarily mean inadequacy. In fact, it may mean that the university conducts a ‘cooling out’ process in some subjects that allows students who are not capable of dealing with the higher levels of study and assessment required to leave the course. This process is not only beneficial for the university, but also for the other students on the course, because it eases transition for them at an early stage, resulting in limited disruption at a later date. This may appear to be a harsh method, but for those who failed the assessments it may eventually prove to be a beneficial experience of higher education, allowing them to review their options; perhaps, for example, a change of subject is to be preferred. As Heywood outlines, the above view on league tables ‘serves as a reminder of the limitations of quantitative measures intended to provide an understanding of the benefits of higher education to the individual’ (Heywood, 2000, p.76).
Despite the conservative image that many western universities have, they have had to adapt in recent years to the above pressures imposed on them by the socio-political system in which they function, and in particular the regional and disciplinary accrediting organisations. These accrediting bodies require institutions to ‘examine their educational effectiveness and to document their impact on student learning’ (Whittlesey in Banta, T.W., 2005, p.10). Traditionally this has been achieved by focusing on input and learning processes rather than outcomes. However, the recent change in assessment attitudes is to stop questioning academic quality through the teaching by numbers template and instead start asking, ‘Are students learning?’ (Palomba and Banta in Banta, T.W., 2005, p.10).
During the 1990s, this change in attitude emerged through the introduction of ILOs: a structure of aims and objectives to state what is expected from students in the assessment of their chosen course. In a sense this ‘rigorous system of quality control’ (Heywood, 2000, p.77) has been put in place by introducing syllabus change that is new-knowledge driven. In particular, one of the biggest changes the ILO system embraced was the introduction of semesters and modular programs that are credit-based, similar to the American university system of assessment. The three main types of student learning outcomes that should be articulated in courses are: knowledge, skills and attitudes. Knowledge outcomes are written using verbs such as ‘explain’ or ‘describe’, indicating what students should know on completion of their course. Skill outcomes are written using verbs such as ‘apply’ or ‘demonstrate’, indicating what students should be able to do on completion of their course. Finally, attitudinal outcomes are written using terms such as ‘act responsibly’, indicating what students should be like on completion of their course.
Applied to the more traditional subjects, ILOs are seen to be advantageous in that, within a credit accumulation system, students need to know what exactly is expected of them, so they can decide whether or not a course is for them before they begin the programme of study. In essence, they allow students to make a fully informed decision. Having learning outcomes available to students also serves as a form of ‘self-propelled guidance since they can look up details of courses in the library and increasingly on the Internet’ (Evans, 2001, p51). In addition to this, if the students are involved in discussion and negotiation of what their learning outcomes will be, they can use this experience as part of the ‘knowledge creation’ purpose of higher education. In essence, ‘the students are being inducted into ‘what matters’ in the disciplinary culture of which their subject forms a part’ (Atkins et al, 1993, p.46).
Contrarily, there has been criticism from some academics that Intended Learning Outcomes are a ‘piece of intrusive bureaucracy which is more to do with regulation than with student learning’ (Evans, 2001, p.50). Attention has been specifically focused on the writing of ILOs and the assumed ‘hierarchy of competences’ (Evans, 2001, p.51) that follows the description of every level of performance, believed by some to give higher education a bad reputation.
In the context of creative subjects, learning outcomes have been met with a more negative response that runs deeper than the aforementioned disadvantages. It seems that many facilitators believe that ‘some forms of summative assessment are major inhibitors of creativity’ (Balchin in Jackson et al, 2007, p.173). Snyder (1967) examined the effect of the institution, the college, and the setting on the development of creativity. He concluded that students in creative fields of study may need greater freedom, in order to explore ideas:
The need for closure, for making a commitment to one of several possible solutions, may be a more crucial experience in the undergraduate education of engineers than in the education of scientists. The student scientist may have and need greater sanction and freedom for intellectual risk-taking than his engineer counterpart. (Snyder in Freeman et al, 1971, pp.41-42)
Although Snyder’s theory was first voiced over forty years
ago, Balchin more recently supports a similar theory in relation to the age of
the ILO, which suggests that the foundations of facilitating and assessing
creativity remain the same. While Snyder called for a greater need of freedom
for creative students in order to explore and test new ideas, Balchin enforces
this by saying that the summative form of assessment in higher education does
not permit failure and therefore ‘generally encourages students to play safe,
and to seek to achieve the outcomes intended by the teacher, rather than the
outcomes the student may like to achieve given more flexibility’ (Balchin in
Jackson et al, 2007, p.174).
The Active Working Role of University Requirements
It is important to define what the difference is between traditional course ‘aims’ and intended learning outcomes. Mann and Graham define aims as ‘the broad intentions and orientation of the course or programme of study’ (Mann and Graham, 2009, p.14). This definition emphasizes that aims focus mainly on what the course in question has to offer the students; for example, the aim of a course might be to ‘create awareness of structure and creative issues around writing’. On the other hand, ILOs are more specific; they are there to describe what is expected of the students, what they should be able to ‘do or demonstrate, in terms of particular knowledge, skills and attitudes, by the end of the programme/course’ (Mann and Graham, 2009, p.14), for example, ‘learn to critique own work and work of others’. They should map directly on to the assessment regime of the course.
Bodies that outline and review pointers of the knowledge, understanding and skills expected from students at differing levels of study in numerous subjects in Scotland are the Quality Assurance Agency for Higher Education (QAA) and the Scottish Credit and Qualifications Framework (SCQF). The QAA’s Subject Benchmark Statements describe what gives a discipline its ‘coherence and identity, and define what can be expected of a graduate in terms of the abilities and skills needed to develop understanding or competence in the subject’ (Quality Assurance Agency for Higher Education, 2007). These skills of discipline-specific knowledge and transferable/professional knowledge differ in their overall aims. Discipline-specific knowledge is needed to demonstrate understanding of specific skills necessary in pursuit of a particular degree. For example, for Communication, Media, Film and Cultural Studies, graduates will demonstrate the ability to ‘initiate, develop or realise distinctive and creative work within various forms of writing or of aural, visual, audiovisual, sound or other electronic media’ (Quality Assurance Agency for Higher Education, 2008, p.14). Transferable/professional knowledge can be used outside the confines of a particular subject and are seen as generic skills which will be important in any walk of professional life, such as ‘work productively in a group or team, showing abilities at different times to listen, contribute and lead effectively’ (Quality Assurance Agency for Higher Education, 2008, p.15).
While the QAA is extremely specific about the types of skills students should be expected to have gained by the end of a particular subject, the SCQF represents a more generic skills base of what would be expected from students as a normal level of achievement at each level in order that it ‘makes it easier for learners to understand the relative value of learning and credit already accumulated, and how this can be used as a stepping stone to further participation in learning’ (Cubie in Scottish Credit and Qualifications Framework Handbook, Volume 1, 2007, p.i). Generally these level descriptors cover the following five areas:
- Knowledge and understanding (mainly subject based)
- Practice (applied knowledge and understanding)
- Generic cognitive skills (e.g. evaluation, critical analysis)
- Communication, numeracy and IT skills
- Autonomy, accountability and
working with others (Mann and Graham, 2009, p.15)
The criticism from some academics who believe intended learning outcomes are a ‘piece of intrusive bureaucracy which is more to do with regulation than with student learning’ (Evans, 2001: 50) was at the forefront of the argument concerning the writing of ILOs and the assumed ‘hierarchy of competences’ (Evans, 2001, p.51) that follows the description of every level of performance, believed by some to give higher education a bad reputation in terms of simplifying learning into boxes that can merely be ticked to quantify learning at different levels of study. Conversely, the rationale for the description of performance and difference in language throughout academic levels is clearly stated in the University of Glasgow’s General Guidelines on the Code of Assessment:
The ILOs must be expressed in operational terms – that is, the ILOs must express what candidates should be able to demonstrate that they can do. This does not deprecate the significance of knowledge and understanding, nor reduce these to mere task performance. It merely acknowledges that the possession of knowledge or understanding is beyond direct observation and that an inference is required, based on evidence presented by way of some particular action on the part of the candidate. The ILOs should communicate the actions expected of candidates. (Mann and Graham, 2009, p.16) Nevertheless, there is no denying that even with all the planning and sometimes intense detail of ILOs and what the students should be able to demonstrate on completion of the course, learning is unpredictable, especially in a creative environment. Evans takes this argument further by claiming that ‘the validity of learning outcomes is based on a false premise’ (Evans, 2001, p.50) in that it presumes that the students will indeed learn what the tutor intends them to. Finally the point is not to disregard the positives of ILOs within creative subjects. For example, critique essays play an important role in accredited courses in order for creativity to develop with an understanding of the traditional theoretical components, such as applying critical thinking in order to analyse creative works with a deeper level of understanding. However, flexibility is also essential for the enhancement of creativity, and perhaps the rigid term of intended learning outcomes should be adapted to ‘learning intentions’ as ‘intentions can either be realised or not realised…the mechanistic flavour of outcomes is avoided’ (Evans, 2001, p.50).
From the very beginnings of course design, in the language used to write the ILOs and the assessment methods used to measure creativity, students are stifled by structure: the ILO framework. Even if students manage to overcome the rigid nature of ILOs what has become clear is that the very essence of learning, in particular creativity, is tremendously unpredictable. Therefore, ILOs are rendered mere predictions:
Higher education in the UK is now based on an outcomes model of learning in which teachers attempt to predict the outcomes from a process that they orchestrate. However, learning emerges from creative processes in unpredictable ways – and, unless the learning outcomes, assessment criteria and assessment methods accommodate this way of thinking, it is unlikely that a student’s creativity can be encouraged, demonstrated and evaluated through the assessment process. (Jackson et al, 2006, pp.173-174)
In conclusion, despite the positive side to ILOs which work excellently for traditional components such as the theory which underpins creative subjects like Creative Writing, they need to become more flexible in order to allow for the flow of creativity. The true beauty of creativity is its individuality: that part cannot be taught in the traditional way.
Faculty of Education
Department of Adult and Continuing Education (DACE)
University of Glasgow
St. Andrew's Building
11 Eldon Street
Glasgow G3 6NH
After many years working within the film and television
industry in New Zealand and the UK, Louise Ramage now works as a DACE tutor at
the University of Glasgow teaching basic film editing. In 2009, Louise was
awarded her MSc in Teaching Adults for research into the outcomes and systems
institutions use to assess creative subjects both undergraduate and postgraduate
Atkins, M; Beattie, J. and Dockrell, W.B. (1993) Assessment Issues in Higher Education. S.I: Employment Department Group
Evans, N. (2001) The Institute for Learning and Teaching in Higher Education: Institutions, Academics and the Assessment of Prior Experiential Learning. London: RoutledgeFalmer.
Freeman, J; Butcher, H.J. and Christie, T. (1971) Creativity: A Selective Review of Research. Second Edition. London: Society for Research into Higher Education Ltd.
Guardian, Education. (2009) "University Guide." In The Guardian.
Heywood, J. (2000) Assessment in Higher Education: Student Learning, Teaching, Programmes and Institutions. London; Philadelphia, Pa.: Jessica Kingsley Publishers.
Jackson, N; Oliver, M; Shaw, M. and Wisdom, J. ed. (2006) Developing Creativity in Higher Education: An Imaginative Curriculum. London: Routledge.
Mann, S. and Graham, C. (2009) Guidelines for Programme/Course Design and Review. Glasgow: Learning and Teaching Centre, University of Glasgow.
Scottish Credit and Qualifications Framework (2007) Scottish Credit and Qualifications Framework Handbook: Volume 1 (Online) Accessed: 22nd July, 2009. Available from: http://www.scqf.org.uk/AbouttheFramework/LevelDescriptors.aspx
Whittlesey, V. (2005) "Student Learning Outcomes Assessment and the Disciplinary Accrediting Organizations." Assessment UPdate: Progress, Trends, and Practices in Higher Education 17(4):10-13.
Quality Assurance Agency for Higher Education (2007) Subject Benchmark Statements (Online) Accessed: 22nd July, 2009. Available from: http://www.qaa.ac.uk/academicinfrastructure/benchmark/default.asp
Quality Assurance Agency for Higher Education (2008) Subject Benchmark Statement for Communication, Media, Film and Cultural Studies (Online) Accessed: 22nd July, 2009
- copy of email bulletin sent to members of ADM-HEA JISCmail list
- copy of email bulletin sent to members of ADM-HEA JISCmail list
- copy of email bulletin sent to members of ADM-HEA JISCmail list