AECEF NEWSLETTER 1/1997
ASSESSMENT OF CIVIL ENGINEERING EDUCATION
by Ben I. G. Barr
1. Introduction
The main purpose of this paper is to review some recent developments relating to the assessment of civil engineering education and to stimulate, via a number of questions, a debate regarding the growing volume of paperwork associated with the assessment process and possible further measures which may take place. Most of the views expressed have been formed as a result of the author becoming involved in this process in the UK. Other countries including the USA and Canada are also in the process of increasing the level of assessment of the education process. This discussion by AECEF is timely and recognition needs to be given to the competing pressures on academics to satisfy all interested parties - some of whom may be considered as customers (students and future employers) whereas others are responsible for raising taxes to support the total education budget of the country.
The theme of this second Workshop in the Symposium follows naturally after the theme of the first session which discussed “Quality in Civil Engineering Education”. There is considerable overlap between the two themes and discussion of this second theme should benefit from earlier discussions regarding quality. The first working paper stated that “quality assurance and quality control is always a process”. It is inevitable that this working paper should spend some time dealing with processes.
In the summer of 1996, a representative of the Higher Education Quality Council in the UK addressed the Engineering Professors’ Council on the topic of “Assessment, Accreditation and Audit” and suggested that these three elements should be considered in terms of their contribution to quality enhancement. Furthermore, concern was expressed about the level of cynicism and innovation fatigue apparent in academic departments in the UK. More significantly for engineering academics, the question, “is educational professionalism lacking?” was put at that meeting. This leads to the first question:
| Q1: | Do engineering academics need educational professionalism as well as subject professionalism? |
2. A Graduate for the 21st Century
The author’s own Institution has recently developed a policy statement for student learning. In trying to identify the requirements of a graduate in the next millennium, the following quotation, taken from the December 1993 publication of the General Medical Council in the UK, was given:
Notwithstanding these repeated exhortations, there remains gross overcrowding of most undergraduate curricula, acknowledged by teachers and deplored by students. The scarcely tolerable burden of information that is imposed taxes the memory but not the intellect. The emphasis is on the passive acquisition of knowledge, much of it to become outdated or forgotten, rather than on its discovery through curiosity and experiment. The result is a regrettable tendency to underprovide those components of the course that are truly education, that pertain to the proper function of a university and that are the hallmark of scholarship. Attitudes to learning that are based on enquiry and the exploration of knowledge are dulled by an excessive information load and by a system of examinations that determines the requirements of study as perceived by the students.
The above quotation leads naturally to the following two questions which need discussion:
| Q2: | Is there a difference between education and civil engineering education? |
| Q3: | What similarities are there between the educational requirements of future medical doctors and future civil engineers? |
3. Education and/or Training
Gaining consensus in this difficult area is virtually impossible. Different people will expect different things from the education process. Two extreme views are as follows:
Scenario 1:
Scenario 2:
Assessment of education is difficult if the objectives are not clear. This leads to a fundamental question:
| Q4: | To what extent should civil engineering education provide training for the profession? |
If we consider again the quotation from the publication of the General Medical Council of the UK, perhaps the first question to ask our doctors before he/she gives a diagnosis is whether they have been “trained” or “educated” as doctors. Another question is this general area is as follows:
| Q5: | What proportion of time within the curriculum should be devoted to the teaching of generic skills? |
4. Growth in student numbers / reduction in funding
Most civil engineering faculties are under pressure to produce more for less. This problem is probably more acute in the UK than in any other country at the present time. Recent years have seen dramatic increases in student numbers (65% for full-time undergraduates and 88% for postgraduates since 1989/90) at the same time as the unit of resource has been decreasing at approximately 2% per annum over a number of years. This had lead to detailed scrutiny of the education process with the following conclusion being drawn by the author’s institution:
In respect of its student population, the goal of the University is to promote the conditions needed for effective learning. Learning is essentially a student-centred rather than teacher-centred activity. Effective teaching is a major element in the promotion of effective learning, but it is not the sole element. Other aspects of the total learning experience and environment are also important, e.g. learning resources, private study, experimental learning opportunities, student support services etc.
Hence the assessment of civil engineering education requires the assessment of a range of activities.
| Q6: | How do civil engineering departments/faculties counter the criticisms of governments (based on expenditure) and employers (based on perceived poorer performance) regarding the quality of graduates produced by overcrowded universities? |
5. Accreditation and Assessment
Within the UK, civil engineering education is subject to both Accreditation and Assessment. Although much of the supporting documentation is similar there is a difference between the two procedures.
Accreditation of degree schemes in civil engineering is carried out by the Joint Board of Moderators representing the Institution of Civil Engineers and the Institution of Structural Engineers. The accreditation team is composed of fellow academics and practising engineers, i.e. an entirely peer review body looking at the entry qualifications of students, the curriculum, support facilities etc.
Assessment on the other hand is carried out by a team composed partly of the peer group and partly by assessors with a non-engineering background. Assessment deals primarily with the assessment of perceived quality of education at the point at which it is experience by students. Hence there is some overlap between the theme of the first Workshop and this one. At the time of writing there is some progress being made towards combining Assessment and Accreditation visits to UK engineering departments. This has arisen from the genuine concerns of departments with the double effort and double disruption caused by the separate visits. Some integrated visits have already taken place and more are planned for the future.
| Q7: | What can be done to extend the integration of accreditation and assessment visits to departments? |
6. What is to be gained by Assessment?
In the worst case, assessment can be viewed as a visit by an inspection team from whom faults and weaknesses are to be hidden. At best, assessment can contribute to a general improvement in the quality of provision through the dissemination of good practice.
Some of the principles and values underpinning the assessment process developed by the Welsh Funding Council are as follows:
The main thrust behind the development of assessment has been the desire to improve the quality of education. Quality is difficult to define. However, a good definition offered by the Welsh Funding Council is as follows:
“Quality is good when students achieve their potential by following a well-organised and challenging curriculum in a supportive learning environment with appropriate assessment and feedback”.
The phrase “achieve their potential” brings to mind another definition of education which is
“to challenge the best to do better,
to encourage the average to achieve,
to help the weak to succeed”.
| Q8: | What are the positive aspects of assessments of civil engineering education? |
7. Key elements of assessment
Four key areas have been identified as the main contributors to the student learning environment, as follows:
The key element dealing with programmes and curricula is closely related to the activities associated with the Accreditation visit. Civil engineering degree schemes must satisfy the broad educational requirements of the Joint Board of Moderators’ Guidelines. The long standing arrangements associated with JBM visits have prepared engineering departments for this aspect of the Assessment process.
For civil engineering students, subject knowledge and understanding will remain at the forefront of university education. However, a graduate for the 21st century will need a range of skills. In equipping students with such skills, higher education will be making an important contribution to economic development and progress. There is currently widespread debate as to what those skills might be. Variously, these skills may include: personal, analytical, problem-solving, cognitive, communication, team work, quantitative, IT and presentational. They should be underpinned by a social, ethical, economic and environmental awareness.
The second key element is that of Teaching and Learning which deals with the three broad areas of variety of learning opportunities, teaching delivery (plus other learning experiences) and the expectation of the standards of student work. Rapid increase in student numbers in UK Universities has brought about a radical review of teaching and learning within many UK Universities. The pace of change has been sustained partly by the need to demonstrate to Assessment Teams how Universities have been coping with increased numbers. In may universities this review has been more fundamental due to the change over from a year system to a modular/semester system. A policy statement for student learning prepared within the author’s Institution states:
The policy is also about desirable change. However, it does not advocate change for change’s sake. There is no implication in this policy that all that is traditional or well established is “bad” nor that all that is new or innovative is automatically “good”. The test of any approach must be its effectiveness, its “fitness for stated purpose”. To introduce new methods for the sake of it is as reprehensible as to adhere to traditional methods purely because they are familiar. Nevertheless, the policy aims to ensure that considerations of teaching, learning and assessments are continually in the forefront of thinking within the institution and that there is lively debate amongst staff about them.
The third key element is that of Assessment and academic support for students. Assessment of the learning process is the main thrust of this key element. Effective teaching and its associated assessment are clearly the main factors in the promotion of effective student learning. The demands of assessment may determine the learning strategies which students adopt. Assessment fulfils a “summative” function in seeking to determine the student’s level of achievement and attainment of learning outcomes. Assessment also serves a creative “formative” purpose which contributes positively to the student’s learning experience and from which the student derives essential and regular feedback on progress.
The fourth key element deals with student achievement. For obvious reasons this area lends itself to more quantifiable conclusions since it deals with topics including entry standards, completion rates, qualifications gained, learning outcomes etc. In some instances this area of the assessment process has been described as “value-added”. However, there is a dilemma here for all academic institutions. Those with high entry standards and a tough course may end up with a higher academic mortality rate than those institutions with a low entry standard, a modest course but a good completion rate!
| Q9: | What should be the key elements (and their relative weighting) in the assessment process? |
8. The future
Predicting the future is fraught with difficulties. We should recall that in Macbeth, Shakespeare wrote the immortal words: “....if you can look into the seeds of time, and say which grain will grow and which will not, speak then to me ....” Such good advice has been ignored by many eminent people including the director of the US Patent Office who said in 1899 that “everything that can be invented has been invented”, a Nobel prize winner in Physics who said in 1923 that “there is no likelihood man can ever tap the power of the atom” the US President who expressed the view in 1905 that “sensible and responsible women do no want to vote” and (on a lighter note) the President of Warner Brothers Pictures who asked “who the hell wants to hear actors talk?”
The main concern of the academic community is that the current preoccupation with assessment will lead in due course to the introduction of performance indicators. There is a feeling at large that universities have not been accountable in the past. One method of achieving accountability is via performance indicators. It is worth recording at this stage the anecdotal approach of the Department for Education towards performance indicators.
Rumour has it that the Department for Education challenged its Japanese equivalent to a boat race on the Thames. The boat race eventually took place and the Japanese team won by 100m. The Department of Education held a top level inquiry and looked at the video evidence which showed that only one was rowing in their team and seven were giving instructions whereas in the Japanese team seven were rowing and one was giving instructions. Hence they decided to appoint a director of rowing, three assistant directors of rowing and introduced performance indicators. After a suitable length of time, the Department challenged its Japanese equivalent to a re-run of the boat race. This time the Japanese team won by 500m. Another committee of inquiry was set up and its conclusion was that the man rowing should be fired as he was not working hard enough.
Rather than resist the inevitable, should we try to establish between civil engineering faculties those areas which we consider as appropriate for scrutiny via performance indicators. Some ideas for discussion during this Workshop are reported in the next section.
9. Possible performance indicators
In 1993 the Association of Universities and Colleges of Canada sponsored a symposium on performance indicators entitled “Measuring up - using indicators to manage change”. It was apparent from the symposium that there was a substantial interest across the country in the use of performance indicators as institutional management tools. Furthermore, it was concluded that most universities would be increasingly engaged in the use of performance indicators in the future.
The use of performance indicators is expanding rapidly in the United States of America. A survey carried out in 1993 by the American Council on Education concluded that 92% of American colleges and universities are now engaged in some form of assessment and use of performance indicators. Some of the 18 categories used for performance indicators in the US (reported by John Gardner in the Proceedings of the AUCC Symposium entitled “Measuring up - using indicators to manage change”) include:
Apart from the last indicator given above, all appear to be appropriate indicators for use in civil engineering education.
The last decade has also seen more emphasis on performance indication in Australia. The Australian Vice Chancellors set up, in 1988, a working party to look at performance indictors. Thereafter the Australian government funded a trial evaluation study based on the findings of the working party. The performance indicators considered in the general area of teaching and learning were as follows:
| Q10: | Should the AECEF develop its own performance indicators and/or contribute to National debates? |
Return to the page "Newsletter 1/1997"