Which Way Are You Going, Quality?

Justina Erčulj

National Leadership School, Slovenia

Introduction

The management of quality has emerged as a key development issue for education in the 1990s and beyond. In spite of this fact, there has been little agreement about what ‘quality’ denotes and even less agreement about how it can best be achieved and managed. I guess that there are very few countries in Europe (if any at all) where some systematic approach to quality has not yet been introduced since stakeholders have been putting pressure on schools, calling for a high quality service. Various attempts to introduce quality standards within the educational context have raised numerous questions and caused serious problems of translating ‘hard industrial products’ into ‘soft educational processes’. Simultaneously, a number of research studies concerning school effectiveness as well as about school improvement practice have been published. It seems that this body of knowledge has been and can be used “to direct the development of schools toward a desired level of performance” (Stone, 1997) but again with a level of prejudice and problems; because understanding quality, effectiveness, improvement, excellence and other notions related to achieving more or less the same aim are highly dependent on the context in which they are being used.
This paper discusses three dilemmas concerning the phenomenon called ‘quality’:

• Can or should quality be understood regardless of the context from which it emerges?
• Who can assess quality in schools?
• Should quality be seen as a general or school–specific issue?

Indeed, the list of dilemmas and doubts could be longer. These have been raised partly through reading literature on the topic but mainly on the basis of my personal professional experience. As a head teachers’ trainer I cannot avoid the questions related to different approaches to quality in education. As the Network of Learning Schools project leader I have – together with my colleagues – initiated the process of school improvement and we have faced some problems related to it. As a professional working in education for 20 years I have critically monitored the current situation regarding the introduction of different approaches to quality in education in Slovenia. This will be explained in the last part of the article.

Some Dilemmas About Quality

Quality Defined

The first dilemma opens with the attempt to define quality. It seems that the term is one which we all understand until we are asked to explain it. We all agree that it is indispensable for a successful organisation. We may even define ‘quality schools’ but the definition of quality itself remains problematic. One of the reasons that a unique or a universal definition cannot be relied on probably lies in the fact that it is “difficult ornot at all possible to measure” (Hopkins, 1987). The “slippery concept” (Pfeffer, 1993) is especially problematic in countries with a strong positivistic tradition where nothing which cannot be measured can be used or introduced at the national level. Therefore Sallis’s (1993) definition, which can often be found in the literature, namely that quality is “what makes the difference between things being excellent or run–of–the mill” may be taken as covering and at the same time opening the whole range of ambiguities related to the issue. However, such ambiguity is connected to many concepts which are mainly dealt with at the practical level. Quality definitely is one of them. Since it has not been moved exclusively to the level of academic research it has not lost its emotional resonance but its understanding remains highly dependent on the context from which it emerges. National culture, tradition, organisation of the school system, level of centralisation, degree of school autonomy, teachers’ readiness and awareness are just some factors which influence the understanding of quality. Besides, quality is a dynamic phenomenon which is an additional reason not to look for a more precise definition. If it was an easy term to define it would not have engaged authors for so many years.

Who Can Be the Judge?

Another dilemma refers to the question of accountability. There are at least three different views of who schools are accountable to.
The customer–centred approaches as more–or–less successful adaptations from industry and commerce imply that customer perception is the key to understanding quality. It is customer satisfaction which counts and defines when quality is achieved. In this context quality is considered as a relative concept. It is defined and demonstrated by a producer having a system known as a quality assurance system. The notion of keeping the customer satisfied goes along with a market–driven approach. While the ‘marketization’ of schools has achieved at different levels in different countries, the approach itself has been introduced with different intensity. If schools do not have to compete for pupils and students (for example rural primary schools in Slovenia) the approach has not had any significant impact. However, linking quality to customer expectations has been criticised on a number of grounds. For me, one of the most serious concerns is related to the question of customers’ professionalism. What, for example, might happen if parents or students choose to specify solutions or products which are not acceptable in the professional sense?
The standards–centred approach is perhaps the best known and the most often discussed among practitioners as well as among researchers and educational administrators. It is sometimes perceived as an alternative approach which attempts to define quality in terms of an objective framework of pre–determined standards. It is not about an absolute concept but about “quality in fact” (Stone, 1997). As long as products conform to their manufacturers’ specifications and standards they exhibit quality. Here two issues must be considered: What or who is the product? Who sets the standards? In education the product, however defined, cannot be produced to measurably consistent standard without considering the processes involved. ‘Student’ was defined as a product during the initial attempts to introduce the existing standards (such as those of the International Standards Organization) to schools. Another view arguing that the student is the primary customer, not the product, led to the understanding that the programme and/or the process of teaching and learning may be qualified as products because the interaction between the customer and the supplier alters the quality of service.
The second question about who sets the standards is more a political one. If they are set at the national level by educational authorities they may be perceived as just another type of control.
Using standards to assess quality may indicate quality at a certain period of time, while it does not say anything about how this state has been achieved: neither does it guarantee further development. The problem with standards has other roots, too. It is a contextual dilemma as to whether ‘industrial’ ISO standards can meet the specifics of education, or more precisely, to what extent quality of teaching and learning may be assessed. Furthermore, too strong an emphasis on standards may become a threat since concentration on achieving targets may distract attention away from each schools’ raison d’ ętre: teaching and learning. At the same time they may be attractive for the same reasons: standards do not necessarily threaten poor performance of teachers so even schools with a very unsupportive learning environment may be assessed positively. At the national level introducing standards is a comfortable way of declaring that the state cares about quality of education and about taxpayers’ money. Again we face the question of what is the ultimate goal of standards: control, ranking, quality itself?
Professional assessment based on inspection is the traditional mechanism for quality management in education. In spite of a general trend within industry and commerce to dispense with inspection there are numerous advantages to this approach (Stone, 1997): inspection/assessment concentrates attention on quality issues, the results allow a level of comparison between standards in different institutions, institutions are held accountable for the standard of service they provide and observations are made by trained outsiders whose perspective may be new. However inspection also has numerous limitations: management and staff attention may be diverted at the expense of other developments, attention can focus on grades, head teachers may feel obliged to adopt a style based on inspection and audit against the best interests of the institution, responsibility for quality improvement may be moved towards the inspectors themselves especially among less committed staff. These advantages and disadvantages are related to the system of inspection in the United Kingdom, while they are irrelevant for those countries where inspection has a more restrictive role.
This overview of these three approaches does not answer the dilemma stated in the question. It just points to the problem of accountability of schools and to the issues arising from it.

General or School–Specific?

This question seems to be the key issue in the debate about quality in schools. Partly, the question has been answered along with the introduction of standards. This section will be more focused to the dilemma often identified in literature dealing with school effectiveness and school improvement. Although both school effectiveness and school improvement researchers have been involved in attempting to improve the quality of education, we face historic differences and even tensions between them . It is not my intention to present both paradigms because a substantial body of literature has been provided by researchers as well as by practitioners. I will just refer to some crucial points which may contribute in the effort of resolving the dilemma.
Since 1960, when the debate about whether “schools make a difference” (Hopkins, 1987) started, researchers have been trying to provide a comprehensive list of factors which seem to contribute to such difference. It would be superfluous to copy any of them here: I would rather point to four issues which have appeared as crucial in most studies.
A prominent place is given to the centrality of institution’s values: whether staff seem to know what the purpose of their school is, what they want to achieve and where they are going. Another important feature is that the school vision, its aims and objectives have been created by the whole staff.
Good relationships among staff and among staff and pupils and/or students have been emphasised. This does not mean that there are no conflicts or disagreement. On the contrary, people are encouraged to express different opinions but these are accepted as part of human intercourse which enable people to learn from one another. There are also plenty of opportunities for pupils and/or students to establish good relationships with adults.
Teaching and learning are perceived as the central processes of every school. This is not as widely found as one might expect and so it has to be constantly emphasised. Besides, teachers expect their students to achieve while in turn they find themselves challenged in the classroom.
Finally, these processes have to be managed. It is not just that the role of the school’s leadership is underlined. More recent studies have gone beyond the head teacher to emphasise the role of senior management. Evaluating and improving the school’s current performance also seems to be one of the central tasks of the effective leader.
The arguments against performance indicators does not differ much from those related to introducing standards. However, compared to many standards they have an important advantage: they have been developed within educational settings. At the same time such lists may be regarded as temptation for some countries. They can be simply taken from one context and planted into another. It is especially dangerous for those countries with little or no research in this area which may lead to “decontextualised empiricism” (Lindgard, 1998). Assessing all schools by externally–introduced performance indicators usually results in “sociological binarism” (ibid.) – on the one hand we talk about organisational functioning, on the other hand ‘soft’ pedagogical issues like identity and subjectivity are put in the centre. It has also been controversial, with each school’s own identity functioning in its own specific context. In this respect I would agree with McLaughlin’s (quoted in Gray and Wilcox, 1995) ideas derived from her observations related to the concept of restructuring in USA. Although schools are accountable to the national educational bodies, policy cannot mandate what matters. She also points to the importance of taking into account local variability while uniformity may only be an exception.
I find more opportunities and advantages in developing more ‘contextually specific’ school improvement strategies if real quality change is to occur. It involves a number of advantages over externally–imposed or introduced performance indicators. The underlying assumptions about school improvement put individual schools at the centre of change. Moreover, it may be regarded as a process through which the school’s capacity for managing change is strengthened while learning becomes the prevailing value in the classroom and beyond. It is not difficult to agree that school improvement is a process which ensures commitment of teachers to change, that it enhances formal and informal interactions, that these schools are able to link educational aspirations to identifiable priorities. But it is difficult to transplant this bottom–up, school–oriented process into school systems with high level of centralisation. The paradox about how central policy can be implemented and monitored while leaving some latitude for professional judgement at the school level has been resolved in some countries by giving schools more responsibility for their own management. However, more autonomy in schools, especially if the focus is given to financial autonomy, does not guarantee that real improvement is going to occur. Other conditions, such as changes in accountability and placing the paramount emphasis on classroom practice have to be met. For many countries it will probably take some years to reach the optimal level of the relationship between “macro–level policies and micro–level behaviour” (Reynolds, 1996). It can thus be concluded that in spite of many educationalists’ agreement at the general level. that it is the process of school improvement which bring real change in the learning conditions, certain pre–conditions are necessary before it may be effectively initiated.
The question as to whether quality may be regarded as general or as school–specific issue is not an easy one and cannot be answered unanimously. The answer depends mainly on the context of the county itself. It also depends on the readiness and capability of those who support these two processes and on their sensitivity for the readiness and capability of those who will be implementing them in schools.
What seems to be crucial (especially in the countries with a short tradition of searching for quality in education) is the danger of the uncritical adoption of various approaches either from research or from practice. The mere application of findings from one context to another has already demonstrated significant contextual differences.

Pulling the Threads Together

None of the questions stated in the introduction can be answered unambiguously. It may even seem that they do not relate. They have been discussed here in order to present uncertainties and variables usually connected with quality. However, it is claimed here that the way towards quality in education depends on at least three variables:

• the way quality is understood in a particular country,
• the question of accountability of educational institutions,
• the level of (de)centralisation of a particular educational system.

A simplified combination of these variables would lead to the following conclusions:
The school effectiveness paradigm is likely to be supported by those countries whose educational system is still highly centralised and where quality is understood in terms of fulfilling standards developed by external institutions: for it gives educational authorities the opportunity as well as the power of developing criteria of quality schools and to employ mechanisms for assessment of schools according to these criteria.
School improvement, on the other hand, may only flourish in those countries where power and responsibility for quality is given to schools and where customers (internal and external) are taken into account, but regarded as partners, not as a threat.
It is evident that the extreme possible combinations are presented here, and simple conclusions are drawn. It is not my intention to develop models or to suggest the results of other possible combinations. The purpose of opening and discussing these three dilemmas has been to cover the range of ambiguity and vagueness concerned quality in education and to point to the possible danger if a single uniform model is developed for different countries whose educational systems differ in many aspects.

Slovenia at the Crossroads of Different Approaches

The situation regarding quality in education in Slovenia may be described by such words as uncertainty, ambiguity, obscurity, searching (and even fighting) for a best–fit approach. Different institutions are involved and different approaches employed. The questions dealt with in this paper reflect to a certain extent the situation.
Quality in education has not been an issue for educational system in Slovenia since 1991 – at least not an overt and discussed issue among educationalists. Moreover, unitarism was determined by legislation: the same values for all schools, a national curriculum, no choice for students or for parents except the free choice of post–compulsory schools. Inspection was the only mechanism employed to control schools, both their functioning and teachers’ performance. The emphasis was on control by counsellors from the Board of Education. This dual contradictory role of the counsellors was changed in 1990 since when the Board of Education has performed a counselling role only. For five years schools were ‘lost in space’ until a state inspectorate was established anew in 1995. Schools’ performance depended on capability, sensitivity and readiness of head teachers and individual teachers.
During this period many ad hoc projects were started in order to increase the quality of schools. Some of them (i.e. introduction of a second foreign language, formative assessment in the first two grades, computer literacy) have become nationally acknowledged. Many others faded away even before they were completed. They were very poorly evaluated so the real impact on quality has not been established. Some schools were literally striving for new projects beyond any reasonable level because educational authorities promoted good schools according to the number of projects being developed regardless of their compatibility with schools’ specific context.
Simultaneously, the awareness of the importance of quality in education has increased and a number of mechanisms have been introduced. The question whether schools make a difference has been finally answered positively. I will focus to four processes which have resulted in differentiation among school.
Covert marketization of schools started at the end of 1980s. Decrease of number of births first influenced the number of pupils in primary schools in larger towns while the collapse of certain industrial bases led to the same situation in vocational and professional schools. Schools started to care more about their performance at the organisational level. They introduced more attractive extra–curricular activities, new technology, foreign language at the age of 10, etc. Stated simply, they responded more actively to their clients’ needs. These schools are strongly market–driven even nowadays so the customer–centred approach increasingly is prevailing.
The introduction of external examination first in primary and later in secondary schools (national baccalaureate) may be seen as a quality control mechanism used by the state. Although the results should not be used for differentiation of schools (they are not given in public) they are used as a tool for promotion of schools. However, there has been much evidence (i.e. Goddard and Leask, 1992; Gray and Wilcox, 1995; Reynolds et al., 1996) that mere comparison of examination results is too simplistic, but since they are the major selection factor for entering the universities, grammar schools are usually ranked according to these results.
Alongside the educational reform of primary schools the claim for quality has been expressed at the Ministry of Education. It is not surprising that they have been in favour of introducing performance indicators. The project. called ‘The Mirror’ was conceived at the Board of Education. At the moment about 30 schools participate in the project with three purposes (Lorenčič, 1999):

• to improve quality in education;
• to introduce indirect evaluation of schools,
• to protect schools.

The results concerning its effectiveness have not so far been made public so my discussion is based on personal information and on a paper presented at the Management in Education annual conference held in Portoroz, Slovenia.
Schools which have entered the project of self–evaluation against the given criteria have reported certain difficulties (Zorko, 1999):

• all performance indicators are not relevant for all types of schools,
• some teachers are not willing to be evaluated by pupils/students and parents,
• lack of knowledge and support in the process of evaluation.

An additional danger concerning the introduction of performance indicators in our country lies in the attempt at ranking schools. If this is to happen then the project (seen from today’s perspective) may have more negative than positive consequences. Too much engagement in fulfilling the required standards defined by performance indicators rarely results in the improvement of teaching and learning processes, especially when this part of the school’s performance is treated as equally important as for example the number of extra–curricular activities.
The project called ‘Network of Learning Schools’ was launched almost simultaneously with ‘The Mirror’ by The National Leadership School (šola za ravnatelje). It is based on foreign practice and on experiences with whole staff training. Although the three–day training of whole school staffs has been regarded as an important step forward in ‘speaking the common language’ they have often remained at the level of an event (Erčulj, 1999). When the pressure was off ‘people reverted back to old behaviour’ (Fullan, 1993) encouraged by the culture of individualism which is probably the prevailing teacher culture in most schools, not only in Slovenia.
Eight primary schools form the network, the main aims of which are to encourage and maintain the process of school improvement, to develop collaborative learning within and among schools and to share good practice. It is based on team learning stimulated by cross–curricular school development teams. The priorities for improvement have been identified at all schools while the implementation phase has not yet started. So far the reactions coming from schools are positive. Teachers are especially enthusiastic about participation in setting priorities for the schools’ development. They have also reported the importance of working together, talking about common problems and knowing what other colleagues actually do. Our initial fear that the team member may not be accepted by the rest of the staff has vanished after the first reports from schools. However, this interaction is still carefully monitored because (at least in our country) it is not understood that schools can use their own potential to run staff development activities. Another issue of concern is how to sustain the initial enthusiasm for collaboration. One possible answer is to use network as a linkage among schools not only in terms of sharing experiences but also as a generator for further activities. At this phase we act as initiators and supporters of the process but the project will achieve its aim if schools themselves feel commitment to continuous improvement. So far, the eight schools have proved at least that quality cannot be mandated. It must become a whole–school philosophy if real improvement is to occur. It would be too optimistic to say that the project will persuade educational authorities that there is an alternative way towards quality. In a sense it actually does not conform to the Slovene centralised educational system. Therefore it cannot even be regarded as the best–fit approach but as a way of quality assurance for schools which have already developed a special culture which is closer to collaboration than in other schools.

Towards a Comprehensive Approach

It will not be easy to find a comprehensive approach in searching for quality in education. We have at least two very different ‘disciplines’: school effectiveness and school improvement. Regardless of what we believe in at personal or at national levels there is still a lot of work for researchers and practitioners in both fields. This is especially true for countries where little tradition in quality in education has existed. The fact that “schools are subjected to quality criteria and bear greater responsibility for realising educational quality” (Wieringen and Ax, 1999) cannot be avoided nor denied. Therefore there is no doubt that each country will search for a common mechanism which will give educational authorities an insight into the quality of schools. However, without carefully developed outcome measures the introduction of performance indicators taken from foreign literature can deteriorate to mere assessment, which can only serve the state especially when they are regarded as prescriptive. This will happen as soon as schools are ranked in any sense according to the performance indicators.
On the other hand, school improvement also involves some traps. Reynolds (quoted in Ribbins and Burridge, 1995) claims that school improvement practitioners need change by adopting an ‘outcomes’ perspective and testing the actual effects of their strategies on whether they generate improved student learning. The issue of the influencing of teaching and learning processes is still one of the concerns related to our project ‘Networks of Learning Schools’. Although the areas for improvement are based on the previous knowledge of school effectiveness as well as on the contextual specifics, the evaluation of the results will have to establish improvement in a very pragmatic way. Otherwise the project will be regarded as just one of the many introduced in our schools with no significant effect – at least without an evaluation which will show the effects.
The ideal scenario would bring both approaches together into the “third wave initiative” (Stoll, 1996) with a variety of strategies that could cover the diversity of schools, while the school improvement practice should rely on nationally–based school effectiveness research. This does not mean that there is nothing to take from the substantial base of knowledge already developed in the UK, Canada, USA, The Netherlands and elsewhere (to list only those most frequently mentioned in connection with quality in education). There are some enduring truths in the processes that have been found to be effective in any country.
Many debates will have to be opened and enhanced in this area, at least in our country. As long as there is no central agreement on what ‘quality’ means, whether it refers to schools in general or to the specific context in which schools operate and until the question of accountability has been resolved there is little scope for one comprehensive approach. The idea of setting the scene for a wider cross–country strategy for quality assurance seems more an illusion than a real future prospect. Each country will have to find its own best–fit approach according to numerous factors implied in the ambiguities surrounding the understanding of quality. This does not mean that we have to start from the scratch. We have certain advantage that we can borrow some concepts from other countries but the danger of uncritical transfer remains the core issue.
However, the concern for quality is not just a professional issue. It has become a part of political discourse. Unfortunately, the debates are more focused to who will set the quality criteria than to who will support schools in their process of improvement. If professional and political links do not flourish then all the dilemmas posed in this paper will be diminished to a flat discussion about who and not about how.

Post Scriptum

I have started from the premise that there is no single view about what constitutes high quality of education. There are no easy answers which would satisfy all. What has been said is that the apparently easy solution of judging quality by outcomes or defining commonly valid and applicable standards or effectiveness criteria will not assure the quality of the process. In fact they can easily neglect the importance of a process taking many years. Quick–fix ‘recipes’ are unlikely to be effective in the long term. Yet if the longer–term view is taken, society requires the assurance that quality education will be provided. The central question about what principles should underpin such a system and what structures are required to support the education service in this work will engage educational administrators, researchers and practitioners. However, each country will probably search for its own solution, dependant on the tradition and awareness of those involved in these processes which cannot be mandated and quick solutions cannot be expected. It is about “long developmental journey with no easy short cuts” (Fullan, 1995).

References

Bolam R and F van Wieringen (eds.) (1999), Research on Educational Management in Europe. Munster: Waxmann.
Erčulj, J (1999), “Quality, Effectiveness, Improvement – Advantages, Traps and Challenges for Different Approaches” in G Vukovič (ed.), European Community and Management. Kranj: Moderna Organizacija.
Fullan, M G (1993), Change Forces. London: Falmer Press.
Fullan, M G and A Hargreaves (1995), What’s Worth Fighting for in Your Schools? London: Paul Chapman Publishing.
Gray, J and B Wilcox (1995), Good School, Bad School. Buckingham: Open University Press.
Goddard, D and M Leask (1992), The Search for Quality. London: Paul Chapman Publishing.
Hopkins, D (1987), Improving the Quality of Schooling. London: Falmer Press.
Lindgard, S et al. (1998), “The search for effective school” in Slee, R, G Weiner and S Tomlinson (1998), School Effectiveness for Whom? London: Falmer Press.
Lorenčič, I (1999), Quality Assessment and Quality Assurance of Pre–University Education. Paper presented to the Annual Conference of Slovene Head Teachers, Portoroz, 1999.
Pfeffer, J (1993), referred to by E Sallis (1993), Total Quality Management in Education. London: Kogan Page.
Reynolds, D et al. (1996), Making Good Schools: Linking School Effectiveness and School Improvement. London: Routledge.
Ribbins, P and E Burridge (1994), Improving Education: Promoting Quality in Schools. London: Cassell.
Sallis, E (1993), Total Quality Management in Education. London: Kogan Page.
Slee, R, G Weiner and S Tomlinson (1998), School Effectiveness for Whom? London: Falmer Press.
Stoll, L et al. (1996), “Merging school effectiveness and school improvement” in Reynolds et al., Making Good Schools: Linking School Effectiveness and School Improvement. London: Routledge.
Stone, J (1997), Increasing Effectiveness: A Guide to Quality Management. London: Falmer Press.
Wieringen, F van and J Ax (1999), “Quality of Educational Organizations” in R Bolam and F van Wieringen (eds.), Research on Educational Management in Europe. Munster: Waxmann.
Zorko, S (1999), Effectiveness Criteria – Pro and Cons. Paper presented to the Conference Management in Education, Portoroz, 1999.