Education in Hungary 2000

Chapter 8 - The Quality and Success of Education

Given the largely increased local and institutional independence in the nineties, issues concerning the quality and success of education were raised with more and more emphasis. The external monitoring of schools declined and standardised testing had not been introduced by the end of the decade. Different investigations indicated the fall of testing-based school performance, and the growth of the already present discrepancy between location and performance. Several international comparative analyses suggested that one of the weak points of Hungarian education is quality assurance. The concerns over the quality and success of education were only raised by the modification of regulations on content and the resulting transitional conditions.

As a result of these processes, an intense professional and social conversation evolved on the possible reasons of the inadequacy of quality and on the available government support in the interest of quality assurance. The aim to improve quality appeared in the election-campaigns in 1998, and the new government programme, passed in 1998, explicitly drew up plans to establish a system of quality assurance. The issue received special attention in the Amendment of the Public Education Act in 1999. At the same time there began a remarkable process of adaptation of quality assurance and evaluation tools already in use in other countries. The private sector had an increasing role in the development of the quality assurance system of public education, which was consciously backed by government policy. Most of the players are members of the professional sphere of education, and there is a smaller segment attached to the business rather than the public service sector in the quality assurance business.

Quality and success have always been important issues of public education. However, having accepted quality assurance as a key term in public education, Hungary has taken a significant step in the formation of views taken by the public on the success of education. Quality assurance, as a term, is used in two senses both in and outside Hungary. On one hand, it refers to organisational and management techniques on the institutional level, whilst on the other, it refers to all devices and mechanisms which serve to ensure the successful operation of public education as a whole. In judging quality and success, it is especially important how professional objectives are set, as these are referred to in the evaluation phase.

8.1 The Comenius 2000 Programme

The Ministry of Education initiated an independent quality assurance development scheme called Comenius 2000 in 1999. The scheme is based on a detailed professional concept, which differentiates between three models of quality assurance and quality development on both the institutional and local governmental levels. The first involves cooperative work-models, and the ability of providing the entailing self-evaluation and self-analysis. The second involves local-institutional adaptation of existing quality assurance systems and the improvement of structural-development abilities. The third involves the diffusion of ideas to other institutions (Table 8.1). From the three models, the second deserves special attention, due to the fact that through its application, and with external professional support, the institutions and local governments can establish systems of quality assurance similar to the ones used in the business world and other public service sectors (ISO or TQM, for example).

Table 8.1    Levels and models of the Comenius 2000 Scheme for Quality Assurance Development in Public Education

In other areas there is less effort made to ensure the success and quality of public education. In the case of curricular programmes or, to use the more recent terminology, pedagogical systems, made available to schools, the problem of quality assurance has not been truly solved. Although there exist generally accepted requirements for textbooks, piloting and the publication of the results of piloting to teachers are both very rare. The development of item-banks in testing is an interesting initiation in connection with quality assurance. Concerning the success of evaluation, there is a growing need for reliable and quantifiable indicators to base the evaluation on. For example, Hungary is an active participant in the Indicators of Education Systems programme of the OECD, and in an indicator-development programme initiated by the European Union, based on the decision made by the Ministers of Education in Prague in 1998.

8.2 Achievement in Hungarian Public Education: the Results of the Monitor Survey

From the indices of school achievement, student performance measured by standardised testing deserves special attention. Surveys on student performance based on standardised testing methods have been conducted in several examination centres for years. Data gathered from surveys based on sampling procedures are available retrospectively for several years, which makes it possible to analyse the chronological change of performance, and also allows for conclusions to be drawn in relation to the development of school-gained knowledge. Previously, surveys using standardised testing primarily gave information on the degree to which students had mastered the body of knowledge in a subject. Recent investigations, however, tend to target the sizing up of general competence, i.e. they put the emphasis on so-called instrumental knowledge instead of factual. Such series of surveys, called Monitor, have taken place on a regular two-year basis since 1986. One of their greatest advantage is that they provide an analysis which gauges performance in the comparison of allowed time.

Reading Comprehension

The most important conclusion of the 1999 Monitor survey, as already indicated by 1997 data, is that the steady fall of reading comprehension results since 1986 has stopped. The overall results, denoted in standard scores and calculated together for both years, were similar in 1995 and 1999. As far as text-types are concerned, findings show that results were somewhat lower with document-type texts, and higher with the other two types and with general cognitive abilities. The favourable and unfavourable changes thus compensate each other, and on the whole, reading comprehension performance is basically identical with the results from four years ago.

Mathematics

In the case of mathematical abilities the 1995 and 1999 Monitor surveys make comparison relevant in two domain parts, both in connection with applied mathematics: in performing algebraic-calculation and text-based tasks. In contrast with reading comprehension results, the 1999 results in this domain fell behind the 1995 results. Although the difference is little, statistically speaking, the trend raises concern. The spread of performance grew in mathematics as well.

ICT Skills and Attitudes

In the case of ICT, the 1999 survey makes 1995 data comparable in two domain parts: in software-related and applied knowledge. Both domains present a significant improvement in performance: the standard scores of 1999 are 62.8 points higher on average than the 1995 scores. This reflects the effect of regular computer use and the widespread use of certain programmes and appliances. It also shows that interest in information technology has shifted to younger age groups, and that attitudes towards ICT have clearly changed in a positive way. For example, the average eighth grader (12-13 year olds) spends 5.5 hours a week in front of the computer by his own admission.

Natural Sciences

The knowledge level of students in natural sciences (biology, physics, geology) has dropped to a smaller, yet significant degree. The decrease was larger in the case of biology and geology, which fell almost similarly, while this drop was somewhat smaller in the case of physics.

Civic Education

Civic education and attitudes have been surveyed for the first time in the 1999 Monitor samples. The survey conducted with 14 year old students included 202 multiple choice questions. Of all the items, there were 38 cognitive questions on democracy and the economy, and 164 attitude questions on democracy and politics, national identity, social cohesion and divergence. Of all cognitive items, 26 were based on factual knowledge (correct answers spread between 40.1 to 88.9%) and 12 measured abilities (correct answers showed a smaller spread: 46.3 to 74.8%). It is typical feature that the majority of cognitive items where correct answers were below 50% were somehow related to the operation of the economy. This might lead to the conclusion that the students’ economic knowledge is fairly unsteady, although analysis of the data also revealed some lingual-cultural correlation. There was an item on the non-democratic functioning of the state, which proved to be difficult, as only 43.3% of students marked the correct answer. The answers to attitude-discovery items clearly indicated that for most young people, and very similarly to the adult population, democracy denotes a welfare state, and that within civic rights they consider social rights superior to political ones. Only 47.1% of students could answer the question: what activities of political organisations are illegal in a democratic state? The analysis of attitude-items raises the dilemma: to what degree do these answers reflect the efficiency and success of education, and to what degree do they merely show the personal choice of values of individual students?

8.3 Factors Determining Student Performance

Student performance largely depends on non-institutional factors, such as the place of residence, family background and the socio-economic features of the students’ environment. The effect of these is particularly significant in Hungary. Therefore the surveys pay increasing attention to gathering and analysing so-called variables which might influence student performance. Of these variables, the settlement-type variable has received special attention in the nineties, since its analysis reflects the fact that the gap between city schools and rural schools is widening. The results of the 1999 Monitor survey indicated that the discrepancies between different settlement types have grown further towards the end of the decade. The previous distance between the capital and county cities seems to be closing, however, the falling of rural schools behind city schools is continuing (Fig. 8.1 and Table 8.2). Apart from this, the differences between the performance of student groups may be primarily explained by the qualification of parents. The difference between the performance of eighth-graders with parents of the lowest and the highest qualifications, measured in standard scores, reaches 20%, which is a lot higher than what we find in developed countries.

Figure 8.1    The change in the standard scores of eighth-graders in reading and the types of settlement, 1991, 1995, 1997, and 1999

Source: Monitor surveys

Table 8.2    The change in the standard scores of eighth-graders in reading and mathematics and the types of settlement, 1991, 1995, 1997, and 1999

8.4 Student Performance on an International Level

The international comparison of student performance may be based on the IEA TIMSS-survey of 1995. The survey examined 9 (third and fourth grade) and 13 year old students (seventh and eighth grade), and also school-leavers who leave public education once and for all. We shall now only pay attention to school-leavers. This category covers three groups: school-leavers of grammar and vocational schools (also called 12th graders in Hungary), and tenth-graders of vocational training schools (who, strictly speaking, do not belong to the category of school-leavers, but for whom public education has finished in the sense that in the last year they do not usually study general subjects.) Students involved in the survey covered 65% of the total population (the survey excluded those who left the education system after primary school, or possibly have not finished primary school, or went to short vocational training school).

A special test, which made the study of mathematical and natural science education possible, was devised for the school-leavers. The results indicated that Hungarian students perform well below the international average. In the domain of mathematics Hungary finished in the 14th place out of 21, and in the case of natural sciences in the 18th. Clearly, the results of this age group are not as good as those of younger generations. The data draws our attention to the assumption that, regarding student performance, problems particularly occur in upper grades. The ranking did not change even if only the best-performing 25% of students from all countries were taken into consideration. As for the divergence of the performance of different age groups from the international average, we can distinguish between four groups of countries. In the first group the performance of school-leavers is below, and that of younger age groups is above average. In the second group the performance of younger age groups is weaker and that of older students is stronger than the international average. In the third and fourth groups the performance is better than the international average in all age groups, but whereas school-leavers in the third group show a decreasing tendency, those in the fourth are improving (Fig. 8.2 on the next page). Hungary clearly belongs to the first group: although students in lower grades achieve results in mathematics, for example, that are getting better and which are well above international average as they progress in the system, performance in final grades is well below the international average. Results in natural sciences show a similar tendency. As long as the 90s trend of growing student intake rates continues, the situation is likely to worsen in the future. This phenomenon raises concern.

Fig 8.2    The differences in the performance of different grades in mathematics and the international average, 1995 (standard scores)

Source: Vári, 1998

8.5 Other Indices of Success

Success in school has other indices than student performance. Such are the rate of admission into higher education, the results of inter-school competitions at secondary level, or the rate of students taking foreign language examinations. This data has been regularly gathered and analysed extensively for all secondary schools, for years. Similar indicators related to primary education are not yet available. Educational statistics also contain indicators that are appropriate for the evaluation of success, such as data on end-of-year failures or drop-outs. In the past years there have been efforts to devise other indicators, such as the index of the frequency of deviant behavioural patterns observable in schools, or the degree of consumer satisfaction with the institutions.

The rate of students admitted into higher education is an index which is relatively easy to measure. This, however, only reflects one dimension of success. Admission into higher education and successful performances at entry tests may be studied according to several indices. The most frequently surveyed rate is the number of students admitted compared to the number of school-leavers (the F/L rate). Data gathered between 1991 and 1998 indicate that the typical F/L rate of secondary schools in different settlements, and in the regions nation-wide had slightly improved, as a result of the increasing number of seats and the decreasing number of school-leavers. The rate was 27.8% when calculated on the whole period between 1991 and 1998, and 29.3% between 1994 and 1998. At the same time, the F/L rate of separate settlement types and regions is nearly stagnating. For example, the highest rate is typical of Győr-Moson-Sopron, Szabolcs-Szatmár-Bereg, Bács-Kiskun and Hajdú-Bihar Counties, where it is constantly above 30%, whereas the lowest is in Budapest and Nógrád County, where it is below 25%. In regards of settlement types, the highest rate is in schools located in the eight largest cities (over one hundred thousand inhabitants), and the lowest is in schools located in the eight smallest settlements (below 5 000 inhabitants) (Table 8.3).

Table 8.3    The average rate of the number of students admitted into higher education and the number of school-leavers (the F/L rate) and settlement types, 1991-1998 and 1994-1998

As for the admission rate, there seem to be large and growing discrepancies between different groups of secondary schools. While an average of 44.7% of students leaving general secondary schools were admitted into higher education between 1991 and 1998, the rate was 24.6% with mixed secondary schools (schools which integrate general and vocational education), and 16.3% with secondary vocational schools. The discrepancies between the F/L rate of differently structured general secondary schools is worth noting: the highest rate is in six-grade general secondary schools and the lowest in four-grade schools, but the index is almost as low in eight-grade general secondary schools as in the latter (Table 8.4).

Table 8.4    The average rate of the number of students admitted into higher education and the number of school-leavers (the F/L rate) and the types of secondary schools, 1991-1998

Another indicator of the achievement of secondary schools is the participation and possible successful performance in the OKTV and the OSZTV (the former is a competition between secondary schools, the latter is the same for vocational schools). From data gathered for twelve years, we may conclude that students ranked in the OKTV came from 444 secondary vocational schools (50% of Hungarian secondary schools), out of which 208 are general and 128 secondary vocational schools. 90% of the total scores were won by 154 schools (17% of all schools). Another index for success and efficiency is the rate of students who have passed foreign language examinations. This index has improved remarkably in the decade. Whereas out of all students applying to universities and colleges in 1991 only every third had a state language exam, the index changed to every second by 1998. In order to arrive at a reliable analysis of the success of schools, there would be a great need for other indices: for example ones which measure the success of students in finding a job on the labour market, participating in professional training as adults, and integrating into society. The reason is that for a large proportion of institutions these indices would be the primary standards for measuring success. From this standpoint it is worth noting that Hungarian schools perform rather well, compared to international standards, when it comes to students’ knowledge of subjects, however, their score is fairly low in indices that measure the school system’s success in guiding students towards the world of labour.