School Autonomy and External Quality Control

Peter Karstanje

Netherlands School for Educational Management, University of Amsterdam / Netherlands

Introduction

In many countries in Europe the relationship between central government and schools is changing. This shift is most striking in the Central and East European countries formerly belonging to or in the sphere of influence of the Soviet Union. Nearly everything in the schools was prescribed by central government. The shift to a democratic society and the impossibility to steer this from a central body caused a dramatic change for schools. They now are responsible for the curriculum, the personnel and in many cases also for the budget. At the same time also in Western European countries this tendency to decrease the number of central regulations and to give the schools more autonomy is obvious although not to the same extent. This ‘autonomisation-tendency’ is quite strong in the Netherlands, for example but less so in Germany and France. In England, to the contrary, there is partly an opposite tendency towards more influence of central government that now prescribes a core curriculum, a phenomenon not known before. In general however, in many Western countries a tendency towards school autonomy is noticeable.

This has considerable consequences for the system of quality care, in the first place because responsibility for quality control is more located in the schools themselves. As a consequence the traditional external quality control as a task of government and/or the inspectorate changes as well. This article focuses on external quality care. The first focus is on the school system. Secondly, external quality management of training programmes will be discussed, partly on the basis of experiences on accreditation and review in recently implemented management training programmes in Central Europe.

First the traditional quality care system in education will be briefly described followed by the changes in this system occurring as consequence of increased school autonomy. After a general introduction to the concept of quality care the external quality care of management training programmes is discussed. Special attention is given to review and accreditation.

The Traditional Quality Care System

In the traditional educational systems in most of Western as well as in Eastern Europe, the government influenced schools with regulations and rules imposed on the schools, using three instruments for quality control:

• uniform, centrally determined examinations
• state controlled teacher qualifications
• precisely prescribed table of subjects and lessons

The inspectorate had a role in controlling whether these conditions were fulfilled in the schools. There are several assumptions behind this type of quality control:

• a uniform set of generally accepted goals is valid for all schools. In so far as goals are not uniform, different school types are created, each with an own set of schools, to make it possible that every type of school has a uniform goal-set;
• goals are valid or seen as important only as far as they are measurable by means of (central) examinations;
• goals are determined by governmens. The clients of the school (pupils, parents) or the customers (further/higher education, business and service organizations) do not play a significant role in the formulation of these goals.

This system existed for general education in the Netherlands until 1968. After that date teachers obtained a little more freedom to work on the goals they wanted to stress. This was done by the introduction of a school-based examination determining 50% of the qualification obtained by the students. The other 50% is determined by the central examination. This was a first important step in giving some autonomy of schools regarding the quality control of the student-based outcomes. However it was not primarily oriented to the quality of the school itself. Such aspects as the educational processes, the curriculum, and the facilitating or inhibiting conditions of the school organization remained out of the picture.

A definition of good quality was in fact ‘following the rules strictly’. A control system (by the inspectorate for example) served to ensure that the quality requirements were indeed met.

This type of system was quite centralized, which implies that the national government played an important role. This could be justified by the fact that the government not only financed the educational system, but also saw it as a task to make sure that the cultural heritage or the national ideology was transmitted in a proper way.

This system of quality assurance did not exist in England or in the United States, where education was more a matter for local authorities or the more independent institutes such as the universities. In France and Eastern Europe on the other hand the centralization could be characterized as very high.

This traditional quality care is now being gradually replaced by a new system that locates responsibility for quality care primarily within the school. This shift is mainly caused by a shift of ‘empowerment’: autonomisation of schools.

‘Autonomisation’ in the Education Sector

In the eighties in many Western countries views concerning the state influence on education and especially it’s quality assurance developed gradually as it changed in Central and Eastern Europe suddenly. The growing costs of state-regulated education, the changing views on education as part of a market, the impossibility of steering complex educational changes from the centre and the need for flexibility and adaptability of educational institutions all led to a movement towards a greater autonomy of schools and of institutions of higher education. Autonomous institutes are better able to compete, to adapt to the market, to innovate and to improve.

Among other things, this ‘autonomising’ implies:

• Financial independence for schools and freedom in spending, with virement between heads of budget (e.g. to exchange expenditure from the budgets for personnel costs and material).
• Competition for pupils becomes more important since the number of pupils is the base for financing the school.
• Less detailed prescription of lesson plans. Instead of four lessons in English per week during the school year five, now the general aims are formulated and it is the responsibility of the school to determine how many lessons they give and when. Or the number of lessons in the whole school period is prescribed and it is up to the school to determine how many lessons are given each year.
• Less detailed prescriptions of the curriculum which increasingly takes the form of general indications or global aims.
• Other requirements of teachers and other personnel of the schools: increasingly competencies and abilities are seen as important rather than formal qualifications. These competencies are to be determined by the school management and not by the central government
• Possibilities for differentiation in financial regulations and legal status for personnel. This necessitates the freedom for a school or group of schools to negotiation with unions.
• School management gets more the role of employer as a consequence of which the representative body of personnel gets another role: it represents now the employees.
• A new role for the inspectorate is necessary, it not being any more the controllers of the reliable execution of regulations.

These developments differ between sectors within a country. In the Netherlands for example, the autonomisation started in higher education, followed by the sector of further education (secondary vocational education) and is now being implemented in the first phase of secondary education and will end up with primary education. Not only within countries but especially between countries striking differences between the degree of autonisation can be observed. In Southern European countries, for example, the autonomisation tendencies are hardly observable. In some Scandinavian countries, on the other hand, as well as in Central Europe, these tendencies can be seen quite clearly.

Consequences of Autonomisation

The described tendencies for and implications of autonomisation necessitate at least three conditions:

• an enlargement of scale. Small schools are too vulnerable for fluctuations in number of pupils (income) and do not have the freedom of movement as far as personnel management is concerned as do big schools;
• a strengthening of the role and task of the school management;
• the necessity of school-based (internal) quality care and a complementary system of external quality care.

Further attention now will be paid to the third condition.

• Several reasons can be mentioned for the necessity to develop a new school-based system of quality care:
• In auditing quality it is not sufficient any more to trust to the classical instruments (central examination, qualification of teachers and table of lessons) that were used in the centralized system. Schools now have to look at different standards of quality appropriate to their goals.
• The clients (pupils, parents) and customers (further education, services and companies} are now seen as more important and have more interest in the quality of the schools.
• Accountability of schools is more important now, since schools will tend to differ. But as noted above the method of holding schools to account must be different as compared with the old system.

The increasingly more autonomous institutions now have had to assure their own quality. Thinking from the free market philosophy it is quite logical that an organization, whether it be a firm or an school, has to have a good internal quality system. Otherwise the service could become unattractive for the clients with bankruptcy as the ultimate consequence.

One could also argue that the clients who invest either finances or time and energy by following a programme, have the right to know beforehand the quality of the programme, a right that the sponsor or resourcer of the programme has as well. This has led to the quest for an external evaluation or accreditation of institutes and programmes besides the undisputed internal quality control. This external evaluation had to replace the ‘old fashioned’ system of control.

Before turning to the external quality some general notions on quality care will be discussed.

Quality Care

A quite general definition of quality care is “the total of activities in an organization oriented toward systematic control and improvement of the quality of the organization”. This definition is quite vague unless the concept of ‘quality’ is explained. In many books on quality management one can find a definition such as “quality is the extent to which the attributes of the organization fulfil the needs of the client”. This definition is not quite sufficient for the educational sector. Educational organizations have more than one type of customer or client, frequently with contradictory requirements. Vocational schools, for example, experience different requirements from large as compared to small companies, not to speak about the requirements of further education. Schools for general education have to reckon with different types of institutions of further or higher education. The pupils and/or their parents have their requirements on what a school should offer them and it may be clear that within these client groups great differences can be found as far as their requirements and definitions of good education (high quality) are concerned. This multi-actor problem causes great difficulties for schools. The solution many choose in this dilemma is to have a somewhat vague mixture of the old examination requirements and adaptations to the requirements of some client groups: while giving each teacher or group of subject teachers the freedom to choose what they want to offer. In that situation quality care is impossible.

In place of giving an elaborate definition of quality care, a model depicting the central components of a quality care system, seems more adequate.

The discrepancy between the envisaged situation (the goals) and reality can be identified through information contained within the information system that every system of quality care should have. This is the ‘quality control’ or ‘quality guard’ part of the system. If the real situation does not conform to the goals there is a problem that must lead to action in order to bring the real situation closer to the envisaged position. This is the ‘quality improvement’ part of the system. The integration of quality control and quality improvement is quality care.

Mission and Goals

This model of quality care is based on agreement on the goals. If there is no agreement on the goals of the school such a system cannot work. The problem of different requirements from different actors can be solved only by a careful process of ‘mission-building’. A mission must be formulated in such a way that it gives direction to the internal decision-making processes in the school. If there is a difference of opinions, for example, it must be possible to use the mission as a final decision-making tool. Besides this internal function of the mission it has an external function as well in giving the (future) clients of the school clarity concerning what the school stands for.

In defining it’s mission the school team must allow for the different requirements of the different internal and external actors: the requirements of branch organizations for vocational education, the official governmental requirements, regional demands or information from former students (not to forget the concerns of teachers and parents) may have to be included. In other words mission formulation requires a careful process of strategic management.

The other components of the model will not be discussed further now but some brief points may be noted:

• The information system must be developed by the school itself. The points of attention, norms and criteria must correspond to the mission and goals.
• It is important that the school team has a good understanding of means-end chains in school life.
• The organization of the school must facilitate communication, exchange and co-operation.
• The results of quality care must be regularly recorded in an internal quality report.

External Quality Care

The described mechanisms and trends have led in the first place to a shift from external to internal quality care. In the past the inspection process had to control the existence of the three ‘classical’ instruments (examinations, qualifications and table of lessons). A system of internal quality care, however, does not imply that external quality care is unnecessary. Schools, being now fairly autonomous, need an external agency to help them especially in preventing to be to ‘selfish’ or to behave ‘unrealistically’. Therefore a new instrument has been used, the so-called visitation committee or review committee. Such a committee, either composed from ‘peers’ or from specialists in the field of schooling has as it’s main functions to analyze periodically the educational institutions to be ‘visited’, to reflect on the internal quality report and to be the agency to which the school gives account.

Primarily the work of the visitation committee should recognize and start from the mission and goals the school has formulated. But of course the committee has a role in reflecting on these goals as well. Besides this reflection, being a ‘critical mirror’ for the school, the visitation committee can also give recommendations for improvement. Since such a visitation is periodical, it can function as a formative evaluation and should not be summative. Therefore the role of the Minister of Education regarding this visitation procedure is in the Dutch Law on Higher Education quite modest. In a case of a university which has received adverse reports, only after some attempts to improve have failed may a (juridical or financial) sanction may be given. In practice, the Minister was quite premature in threatening to stop the financing of certain teacher training colleges if they were not improved two years after the visitation that turned out to be quite negative. Here a possibility of interchange of formative and summative evaluation is illustrated, with a possible consequence that educational institutes will not be honest any more in producing their internal quality report but will ‘cheat’ by window-dressing reports, which makes the whole system useless.

The same problem arises when visitation reports are published in cases where the education institution will get a bad name. This dilemma is very difficult to solve.

The main task of the inspectorate has changed from controlling the regulations towards ‘meta-evaluation’, checking whether the internal and external evaluation fulfils the requirements formulated for such evaluations. Here we can also see problems in the transition period. Inspectors, used to their previous functions, now have to have rather different attitudes and skills. Sometimes, however, inspectors like to continue their old controlling role.

Management Training Programmes

Management training programmes are a new phenomenon in many countries. The explanation for the raising need for professionalisation of educational managers can be partly found in the increase of school autonomy. It is not only that more decisions have to be made at the school level but the decision making structure has became more complex, other internal and external actors are involved in determining the main goals, and school-related quality improvement requires a different set of skills as compared to making sure that regulations are followed.

For management training programmes the quality control problem is at least as essential as for the school sector. The judgements of the clients (mainly defined as the participants in the courses, and those who appoint the school leaders) are of great importance. This aspect of quality control could be reduced to internal structures and processes. Almost every management training programme bases its quality control on the needs of the client groups, the satisfaction of the participants and the success of it’s graduates. Information from these customers should lead to adaptation of the programme and its processes. However, there is a trend towards more than only internal quality control. Especially, management training institutes are inclined to stress external quality control, preferably by means of accreditation (and especially international accreditation).

The author has been working for five years with people from several (mainly East-Central and East European) countries on the establishment and implementation of programmes for training in educational management. The first visits always consisted of meetings with representatives of different institutions along with governmental officials. Especially the latter, but also university people and school leaders, were very interested in whether the programme to be developed would be internationally accredited. This question always surprised me, coming from a country where accreditation is not so common.

The reasons for this interest might be the following.

1. Especially during the first years after 1989 there was a strong orientation to the West, many people thought (partly erroneously) that the knowledge and experience from Western countries was definitely better than what came from their own country. So international (or Western) accreditation would give a high status to the programme to be developed.
2. Accreditation is a common phenomenon. The universities accredit the institutes for higher vocational training, which accredit again the institutes lower in rank. In some countries this still exist. In Ukraine for example the university of economics accredits the work of the national in-service training institutes providing educational management programmes.
3. Teachers and school leaders usually profit in the form of higher salaries from following a course of study, especially when it leads to a degree. The higher the degree, the higher the increase. It is understandable that for the clients of management courses, international accreditation could increase the status of the course.
4. There was an ‘outburst’ of initiatives to establish new institutes. Since the quality of these programmes was very dissimilar, accreditation could be a means for distinguishing the better programmes from the mediocre ones.

Who Are the New Controllers?

An interesting question here is, who should be the agency of external control? Different solutions are possible. One solution frequently chosen, namely the university, can serve as an example or case to illustrate the problem and to come to a line of thinking on this question. While establishing programmes for educational management, the people usually involved want to develop useful programmes. That means, among other things, practicality, an appropriate relationship between theory and practice, skills and competency training and so on. On the other hand these programmes usually have to be embedded in the university structure. To be accepted within the university environment, such a programme has to fulfil the usual requirements, such as basic courses on statistics, research methods, psychology, sociology, philosophy and pedagogics as well as examination requirements of a certain number of pages and papers with a minimum size. The question is whether these requirements, coming from a completely different view on what constitutes quality, are really relevant for a programme in educational management. So usually there is a tension between the expectations and goals of the clients and the requirements of the ‘host organization’ (in this case the university). This phenomenon is certainly not only restricted to Central or East Europe. In England, the Netherlands and other countries these constraints are felt as well.

This problem, that will be easily recognized by people responsible for establishing and implementing management programmes, illustrates that different interests of several parties are at stake with external assessment: those of the clients, the traditional institutions, sometimes the government and not to forget the people who are concerned with the teaching and management of the programme.

Accreditation Versus Peer Review

External quality assurance, however, is not just accreditation. Accreditation can be described as “a process of quality control and assurance, whereby, as a result of inspection or assessment, or both, an institution or its programs are recognized as meeting minimum acceptable standards” (Adelman, 1992, p. 1313). Typical for accreditation is the approval: the declaration that an institute or programme meets a standard. It is a matter of pass or fail, just as a final examination for a pupil. It is a special American way of external evaluation. There are not very many Western countries, however, where accreditation of schools and institutions of higher education is in use. In many European countries external evaluation is done by peer review. This means that an institute carries out a self-evaluation which is the basis for an external group (usually ‘peers’ or independent colleagues) who gives comment on this review, after visiting the institute. These comments are reported and form the basis for an improvement project. After a certain period this procedure is repeated, so the review committee can see whether there is improvement and what should be done further to enhance the quality of the institute or programme. In essence this cyclical procedure does not differ from accreditation (except the fact that accreditation implies an additional judgement).

In the United States of America the accreditation procedure is similar to the review process: the accreditation committee visits the institution to be accredited after having read the self-evaluation. The result can be either accreditation, conditional accreditation or rejection (the latter is seldom done). If the accreditation is not given immediately, more information or explanation of questions which were not clear in the self-review may be requested, or some improvement required on certain aspects that did not meet the criteria of the accrediting institution. Sometimes it is also decided that an in-between visit is necessary to check before the next accreditation (usually after a period between five and ten years) whether and how improvement can be seen. So accreditation also has a cyclical character, focused on improvement rather than on acceptance or rejection as such.

East-Central European countries are usually oriented more on the accreditation model as it is used in the United States, rather than on the European peer review system (Kells, 1995, p.18).

In England the external quality assurance in higher education is quite different from that in the USA. As an answer to the threat of the government to submit the universities to inspection, the universities established a committee for national quality. The task of this Academic Audit Unit was to check the extent to which the institutions controlled and improved their own quality. The autonomy of the different universities was carefully respected and the audit unit limited itself to an evaluation of the evaluation mechanisms in the universities (meta-evaluation). In 1992 the Higher Education Quality Council (HEQC) continued the work of the Academic Audit Unit. It is instructive to read their Guidelines on quality assurance 1994. The first part encompasses a list of 35 points of attention divided into four parts:

1. Framework for quality (e.g. on characteristics of an effective quality assurance and control system).
2. Entry to higher education (e.g. admission policies and requirements, selection process and facilitating student entry).
3. Quality of the student experience (e.g. on external and internal programme approval, evaluation of teaching and learning, staff appointment, staff development and appraisal).
4. Student outcomes (e.g. student progress, student assessment, external verification of standards, external examiners).

The second part consists of an annotated list of suggestions for each of the topics of the first part with not only literature, but also institutions or persons where one can find more information on that topic.

Functions of External Quality Assurance

The involvement of external agencies or actors in quality assurance has different purposes. Frazer (1995, pp. 14-15) distinguishes five purposes that can partly be connected with accreditation and partly with external review:

1. Identifying excellence. When criteria for quality control are known, it is possible to find out how far these criteria are met.
2. Checking that externally defined threshold standards are being achieved at a minimum level.
3. Checking that internally set goals are being achieved, whereby there must be some judgement about the values of the goals.
4. Checking that there is value for money (efficiency).
5. Identifying and disseminating good practice.

These purposes reflect a quite typical accreditation-oriented view since the author assumes that there is an externally defined minimum standard and that the external agency is able to judge the institution to be accredited against this standard. There is a problem, however, connected with the accrediting agency. How far is this agency familiar with the goals of the institution to be accredited? Or does it have an own set of goals? Is this agency familiar with the client groups the institution is working for? The above mentioned controversy between the goals of the management programmes and the university goals illustrates this problem.

A more European view on purposes is described by Liket (1992, p. 62) who also attributes five functions to external evaluation:

1. Legitimation and recognition that new activities are important and guaranteed for the future.
2. A mirror to the institute by means of feed-back.
3. Transfer of good practice.
4. Innovation as a consequence of feed-back.
5. Stimulating policy to improve the conditions for such a programme or institute.

Liket’s view on purposes is not fundamentally different from Frazer’s list, but it reflects a different emphasis. While Frazer is especially concerned with accountability or external judgement, Liket is more oriented to corrective change and enhancement of quality.

This difference in view is depicted by Kells (1995, p. 19) who distinguishes several views on evaluation systems for higher education based on his work on external evaluation in many countries. Kells sees the purposes of the process on a continuum ranging from ‘improvement’ to ‘public assurance’ and ‘governmental goals’. These different purposes are connected with (among other things) the framework for judgements and the primary procedures. Figure 2 depicts a part of Kell’s scheme.

In line with the growing autonomy of educational institutions, Kells sees two general trends over time:

Figure 2 - Kell’s Scheme of Accreditation Processes

Primary attributes	← Spectrum →
purposes	improvement	public assurance	government goals targeting resources
framework for judgements	stated intentions	peer opinions	norms comparisons
primary procedures	self-evaluation process	external peer review process	indicators ratings published

1. Schemes depict more internally-driven concerns, more emphases on self-evaluation and self-regulatory activity (the left side of Figure 2).
2. Schemes are less related to the influence of government and more related to improvement and to better management. Feedback from clients and more focus on the improvement of teaching, learning, service and management tend to occur increasingly as government backs away. Kells (1995, p. 20) says about this phenomenon:

“It seems that universities act more maturely in these matters if they are treated as trusted adults and less like children and if they are wise enough to seize the responsibility for the evaluation scheme and for self-regulation. In addition, the more institutions are expected to manage themselves, the more useful the results of self-regulatory evaluation seem to be”.

Besides, accreditation on the basis of fixed, external norms and standards has a great danger for the developmental potential of the accredited institution or programme. If the people involved are not committed to the external criteria it will be very difficult to move the institution in the intended direction. Therefore a self-evaluation according to standards that are perceived to be important for the people of the institution is far more helpful than just laying the programme or organization on the procrustus-bed of the external accreditors.

Framework for Judgement

Regarding the framework for judgement, the problem of whose framework should be used has been mentioned. If we look at management programmes there could be a clear answer. The clients of these programmes are important in deciding what is good and what should be changed. The typical university requirements could partly be opposite to the clients’ interest. Ideally the management training institute should have a good procedure for identifying the clients’ goals and to find out whether these have been met by the programme. The review committee reviews whether this procedure guarantees a good fit between the clients’ goals and the outcomes of the programme. This is the way the British Higher Education Quality Council, for example, promotes: the review committee does not use specific goals as criteria and standards, but looks at whether the institute to be reviewed has the right procedures and uses them properly to assure that their goals fit those of their clients and that they have a procedure to know whether these goals are indeed met.

To complicate matters further, there is usually not only one client group. Management programmes serve different groups of school leaders, prospective school leaders, sometimes consultants or members of the educational administration. If one accepts the importance of client groups in external evaluation, it is important to recognize the involvement of these groups in the review, either by requiring the institute to pay attention to these clients in the self-review, or by incorporating these clients in the review committees, or both. Writing about accrediting Van Vught and Westerheijden (1994) argue in this respect for a system of ‘multiple accrediting’ indicating that all actors interested in the quality of an institute or programme have the right to approve this if the institute or programme fulfils their requirements. On the other hand the institutions have the right to choose those accreditations that fit best their profile.

International?

A question not addressed so far concerns the international character of the external quality assurance. Is it really helpful to have foreigners in the external review committee, or even to have an international review? An important reason for involving foreigners is to avoid either competition or co-optation problems, which could emerge especially when the national professional community in the field of study is small. There is also a good reason to ask people from abroad if the level of development of the colleague organizations is still not in balance. Having reviewers from abroad may be helpful, provided they do not impose their specific norms and standards but use proper procedures based on self-review.

Conclusion

External quality assurance and control of educational management programmes and institutions for educational management training can have an important function, primarily for the improvement of the programmes. If improvement is the main goal, a process of self-evaluation followed by a review with feed-back is preferable to accreditation, especially for recently developed programmes. An international review can be given preference if the national context is too imbalanced. The problem of multiple actors can be tackled by involving more actors in the self-review procedure or by involving those actors in the review committee. Governments that shifted towards autonomy of educational institutions should adapt the quality control and improvement procedures to the new autonomy, with the primary responsibility for quality assurance being in the hands of the institution itself.

References

Adelman, C (1992), “Accreditation” in B R Clark and G Neave (eds), The Encyclopedia of Higher Education. Oxford: Pergamon Press.
Frazer, M J (1995), “Issues of quality”, Higher Education in Europe, 20 (1-2): 13-17.
Higher Education Quality Council (1994), Guidelines on Quality Assurance 1994. London, HEQC (344-354 Gray’s Inn Road, London WC1X 8BP, England).
Karstanje, P and T Baráth (1994), “Implementation of a master’s programme on educational management”, in Hämäläinen, K and F v Wieringen (eds), Reforming Educational Management in Europe. De Lier: Academic Book Centre: 233-257.
Kells, H R (1992), Self-Regulation in Higher Education. London: Jessica Kingsley Publishers.
Kells, H R (1995), “Building a national evaluation system for higher education: lessons from diverse settings”, Higher Education in Europe, 20 (1-2): 18-26.
Liket, Th M E (1992), Vrijheid & Rekenschap: Zelfevaluatie en externe evaluatie in het voortgezet onderwijs. Amsterdam: Meulenhoff.
Vught F A van and D F Westerheijden (1994). Naar een Stelsel van Meervoudige Accreditering? Utrecht: Adviesraad voor het Onderwijs.