by Manfred Federau

1. Introduction

The scope of this paper is to function as a white paper drawing the parameters and pointing out important topics for discussion at workshop 1 of the second AECEF symposium held in Odense, Denmark May 1997. It is the intention of the author at this point to avoid (however difficult this may be) stating any personal views on details of this interesting subject. It is the intention that the presentation of the paper should include these personal views in order to stimulate discussion.

2. Definition of Quality

Much time can be used to discuss academically how to define quality in education. The conference participants are asked to accept the following:

"The level of quality of a certain education (or part of it) can be measured by the degree of fulfilment of its goal".

If this definition is accepted, the following facts can be listed. They are all very obvious, but still very important for a meaningful discussion:

  1. All goals must exist in a written form.
  2. It is meaningless to discuss quality without agreement on how goals are interpreted.
  3. The quality of a certain (element of) education may be a state of things, but quality assurance and quality control is always a process.
  4. Level of quality must never be confused with academic level.

3. The Process

In order to be able to discuss how quality is established, how quality is maintained, and how quality is measured, the model shown in figure 1 will be used.

Very little time should be used to discuss the hierarchy of processes; most effort should be put into dealing with the quality elements of the department level. It is, however, important to presume the existence of goals, policies and a strategy on the level above the level in question. Furthermore the model is limited to three levels, also because the intention is to initiate a discussion of functions rather than the structure of the process.

4. Main Goals

Very often these are implicit. However, it is typical to find a great variety of views on this subject when it comes to actually codifying main goals. Examples of questions could be:

Q1: Who is the customer? Is it the student? Or the employer? Or society?
Q2: What is the product? Is it the graduates? Or education? Or both?
Q3: Should the education imparted by the department be broad or narrow, i.e. should the graduates be generalists or specialists?
Q4: Bachelor _ Master’s degree; single or dual ladder?

Q1 may not be important for quality. The combined choice of response to Q1 and Q2, however, will enable the department to give an important signal to the interested parties and to ease the search for variables. Q3 and Q4 are important for the departments choice of academic content, means of education, and structure.

5. Subgoals

Distinction between main goals and subgoals can vary; sometimes settling for only one heading - goals - seems practical. In an organisation with a faculty-department structure it would seem natural to codify main goals at faculty level and subgoals on department level, whereas a more independent unit will focus more strongly on establishing a more comprehensive system.

To supplement Q1 through Q4 the following questions could be the basis for discussion of subgoals:

Q5: Bachelor level: To what extend should the graduate be able to do routine assignments? Or: How much emphasis should be put on the practical rather than the theoretical aspects of engineering?
Q6: Should the education imparted by the department aim for a broad orientation on all aspects of the curricula offered or, alternatively focus on depth of learning within fundamental engineering principles?
Q7: Does the organisation aim to impart to its students certain qualifications or rather how to seek and maintain competencies? In other words: How important is the existence of a certain (traditional) professional content compared to general perception training - bluntly: The ability to read?
Q8: To which degree should curricula be fixed, alternatively be profiled by the students choice of elective courses, project themes, etc.?
Q9: Implicating the purpose of electives, etc. being the students need for pursuing professional objectives, should it furthermore be possible to adjust curricula to suit his or her level of ambition?

These questions are closely related - in combination the answers determine the impact on quality of professional content and capacity of the syllabi in question. Q6 and Q7 are important and could easily be listed under main goals.

Depending on degree of detail other topics could be discussed - or be filed under the heading: Policies (see below).

The description of goals should give the interested parties a clear cut impression of the department and its ideas. It should also be the basis for extraction of variables suitable for measuring and controlling quality.

6. Policies

Policies are statements - in writing - on strong principles, to which staff and students should conform. Together with description of idea, main goals, and subgoals it constitutes the department’s quality assurance. A host of reasons could be listed for not letting such statements exist only implicitly:

Since the content of policies of any department depend very much on the content of its goals, listing questions important for the discussion is not possible. The following headings, however, should show the most important areas policies must cover:

P1. Policies on educational structure.

P2. Policies on staff development.

P3. Policies on academic and professional content,

P4. Policies on means of education.

P5. Policies on internationalisation and relationship with society.

P6. Policies on social relations.

P7. Policies on evaluation.

  1. Student evaluation (Examination)
    • Principles on to use oral or written examination, approval (or a combination of these), and which form to use on each type of study element: Taught courses / project work / exercises / etc.,
    • individual or group evaluation; or when to use what,
    • procedures for oral examinations: Reading lists? List of exam questions? Preparation time?
    • Written examinations; use of aids (hardware in the form of pocket calculators, computers etc. and software in the form of books, compendia, notes, tables, examples, old solved problems etc.),
    • projects in general; procedure for supervision, i.e. how to avoid evaluating the supervisors contribution, procedure for examination, format demands,
    • diploma projects; procedure for supervision and examination, format demands, the use of student critique.
  2. Course evaluation
    • Form; oral or written, individual or in groups, questionnaire or free form, anonymous or not,
    • frequency,
    • how to act in case of persisting critical reports,
    • collegiate supervision,

The above list of policy areas is not complete; supplementary contributions are most welcome either in submitted papers or at the workshop discussion. However, the author hopes the list is comprehensive enough to create a fruitful discussion of this vital element of the process.

7. Strategy

Strategy is about how to plan changes. Even the most perfectly (read cocksurely or conceitedly) governed department or faculty will have to admit that a study of its quality level including a precise codification of goals and policies will expose a marked need for changes. In the best case it will find itself no worse than those it usually compares itself with, maybe even better - but at the same time it will realise how big an unused potential it has.

The elements of the model (figure 1) can be used to describe a major purpose of strategy: How to create a stable and well-balanced link between the course level and the department level part of the process.

The model implies that goals on course level are set at department level, that those responsible for the course(i.e. the lecturer) must use policies and a strategy to reach these goals and even conduct a course evaluation. The model also implies a quality control conducted at a higher level. Now, how does this reconcile with what is usually known as “freedom of method”. Or: What happens if the analysis exposes a need for changes in goals, in content, in structure, not to mention in method?

Q10: Will the introduction of QA/QC call for changes in the organisational structure? Where in the organisation should the “control centre” be located, i. e. who is responsible for the course of action?
Q11: Structural planning: Is the education structure flexible enough to cope with changes in curricula and syllabi?
Q12: Time planning: Should the strategy imply a quick and in depth or a slow, step-by- step introduction of changes?
Q13: Method planning: Is it possible to imply or even prescribe the use of certain means of learning and at the same time maintain the inspired effort of the teaching staff involved?
Q14: How are students and administrative staff involved in the process?

8. Evaluation

In education quality control is not a matter of measuring tolerances or a simple checking of production efficiency. It is much more complex. Every education system has a function that provides an element of quality control: Student examination. In some cases the system makes use of another control-like function: Course evaluation. It is tempting to use the issue of these functions as main mark of quality. This is, however - in most cases - a fallacy. On the other hand it would be unthinkable to conduct a quality control system without using (to put it neutrally) circumstances around examination and course evaluation as a measure. The problem is how and under which conditions this is expedient.

Q15: Which variables deduced from student evaluation can be used for education evaluation?
Q16: What are the conditions for making course evaluation serve as a means for measuring education quality?
Q17: What other means (besides student examination and course evaluation) could be used for measuring education quality?

9. Conclusion

Education is like a gutter; what you get out of it depends very much on what you put into it. This was originally said by Tom Lehrer about life. The reader of this paper will realise that quality of education is hard to define, to measure and to maintain. However, it is the hope of the author that this paper can trigger a well-structured analytic discussion.

The main goal of the author is to contribute to an improvement of engineering education, especially civil engineering education at bachelor level.

An important subgoal is the 2nd AECEF symposium being able to create a platform for the future work at this field. The main strategy is using a white paper to govern the proceedings of the symposium and the workshop outcome directly towards fulfilment of this goal. Evaluation will be part of the symposium conclusion.

The idea to use white papers as guidelines for the submission of papers and for the symposium discussions was given to the author by professor Wallace W. Sanders of Iowa State University after the general assembly of the AECEF 27 September 1995 in Prague - actually during the train ride from ÈVUT to Malostranská. The author is very grateful for this.

Return to the page "Newsletter 1/1997"