Part III: Achievement, Assessment, and Evaluating Learning

Comparative School Achievement
National Case Study Report
International Study of School Achievement
Reflections
The Two Faces of Examinations
Tradeoffs in Examination Policies: An International Comparative Perspective
Secondary School Examinations: International Perspectives on Policies and Practice
An International Perspective on National Standards
A Comparative Assessment of Assessment
An International Comparison of End-of-Secondary School Examinations

Source: Harold J. Noah and Max A. Eckstein,
"Tradeoffs in Examination Policies: an international
comparative perspective," Oxford Review of
Education, 15 (1989), pp. 17-27.  Reprinted
by permission.1 

TRADEOFFS IN EXAMINATION POLICIES: AN INTERNATIONAL COMPARATIVE PERSPECTIVE

Controversy over examination policies is commonplace in the contemporary world. It has been exemplified in China's abandonment of secondary school and university entrance examinations during the Cultural Revolution, and their reinstatement 10 years ago; in the disputes over the form and purposes of the Bac in France, disputes that from time to time threaten to undermine the very continuance of government; in the concerns expressed in Japan that whatever the benefits its 'examination hell' may bring in the way of stimulating student and teacher efforts, they are being bought at the price of severe tension placed on young people and their families; and in the current vigorous discussions in England over the institution of the GCSE, the changes proposed for the GCE Advanced Level examinations, and the introduction of periodic national assessments of pupils' progress throughout their school careers.

Argument ranges over the entire spectrum of matters associated with examinations -- from narrowly technical problems of examination procedures, through questions of broad educational significance, all the way to issues that touch fundamental ideological concerns and political preferences. Not that any particular question about examinations can be neatly categorised under a single such heading. Even the most arcane technical question can, and not infrequently does, carry with it implications for education as a whole, and even political choices; and these, in turn, can quickly involve specialised psychometric problems.

The specific terms of debate vary significantly from nation to nation and from decade to decade, but some recur. None of the policy problems are easy to solve, and some are so difficult that they can well be called dilemmas. In consequence, each nation's system of examinations may be regarded as representing a set of provisional compromises among competing values. While seeking to increase perceived benefits in one direction, a nation almost inevitably gives up some benefit or exacerbates some problem in another direction. It is in that sense, therefore, that we view extant examination systems as configurations of tradeoffs, arising out of the dilemmas of evaluation policy. ²

For example, consider the characteristic of examination uniformity. Uniform examinations across the entire nation facilitate comparability and even handedness of treatment between different groups. But uniformity exacts its price: regional and local interests may feel slighted, the centre's purposes are likely to be served at the expense of the peripheries', and opportunities to adjust the examination to recognise the different needs of regions or groups at different stages of school development are inevitably reduced.

Or, consider the extent to which options are permitted. A large measure of optionality brings the clear benefit of adapting the examination to the subject preferences and aptitudes of individual candidates. But optionality inevitably weakens the sense of a national curriculum and a national culture. A credential based on a familiar standard set of compulsory subjects is easy for employers and admissions officers to interpret; they can be puzzled indeed by the complex regulations and weighting schemes used to equate the essentially non-equatable assortments of examination subjects offered by candidates.

Or, consider the choices for the format of the examination. Oral examinations were once quite common at the end of secondary school, because they offered an opportunity for assessment based on interaction between the examiners and the candidate, and thus permitted examiners to shape standard questions to individual candidates. Nowadays, oral examinations are rare, mostly because the cost is considered too high, but also for fear of loss of objectivity and comparability across candidates. Precisely in order to gain such objectivity, a few nations have turned to multiple-choice, machine-scorable examinations, which also have the significant benefit of costing very little per additional candidate, once the substantial initial expenses of constructing and pretesting have been met. Yet many believe that in the end these benefits come at too high a price, encouraging styles of teaching and learning that they would prefer to avoid.

These and other tradeoffs can be illustrated from the experience of the eight countries in our study.

The United States

More than most countries, the United States has embraced the device of machine-scorable examinations, usually in the form of collections of multiple-choice items. This has been done largely because the commitment to widening the clientele served by examinations has been so strong, and the resulting numbers of candidates so large. Large numbers of candidates in turn made it economical to invest considerable resources in formulating, pre-testing and revising a very large bank of items, from which actual question papers could be constructed. The option of retaining the traditional extended-answer type of examination was rejected, for reasons of cost and complexity of organising a grading system that would be seen to be equitable. The choice has exacted some substantial educational costs: the development of written language skills among the student population is not a high priority; careful construction of an answer gives way to learning test-taking tricks and the tactics of guessing; in practice, short-item questions tend to emphasise recall-type learning, rather than analysis and problem-solving. These drawbacks are widely conceded, but the price has been paid and the tradeoff has been made relatively willingly and uncomplainingly in order to secure the important political value of a more accessible and objective examination system, as well as the ability to deal reasonably inexpensively with the consequent flood of candidates.

A second noteworthy feature of the United States examination scene has been the rejection of the slightest hint of a centralised system of examinations in the hands of the national government. Nor, indeed, do most of the 50 states offer a secondary school leaving examination or university selection/entrance examination. Instead, the job is left to what are essentially private organisations, such as the Educational Testing Service (which provides the Scholastic Aptitude Test on behalf of the College Entrance Examination Board), and the American College Testing Program.³ Though the provision of examinations by these organisations has introduced a certain element of coherence to an educational system that would otherwise be exceptionally fragmented, their non-public status has nevertheless helped maintain the states' rights and even parochial bases of American education and they have done little to help raise general educational standards in less advantaged parts of the country.

However, some change may be in the offing. The Educational Testing Service has recently (1987) been handed responsibility for conducting the National Assessment of Educational Progress (NAEP), and for publishing and analysing the results. At the same time, NAEP procedures are to break with past practice, and in future are to make possible the publication of interstate comparisons of cognitive achievement of students at ages 9, 14 and 17 years. In consequence, some observers are forecasting that the United States is, perhaps unwittingly, veering toward establishing a de facto national achievement test that will eventually drive the adoption of a de facto national curriculum. ⁴

Japan

At the other end of the spectrum of control and coherence of education in general and examinations in particular, Japan until 1954 operated a very economical system of selection for higher education entrance, on the basis of a single, nationwide, standardised examination. In view of the extreme importance of the decisions that were being made on the basis of this single examination, the quantity of resources spent on providing it was remarkably low. Between 1954 and 1976 various other programmes of selection were tried, and in 1976 the present two-stage system was introduced, primarily in order to improve control by colleges and universities over the make-up of their entering classes.⁵ The first stage, the Joint First-stage Achievement Test (JFSAT) is retrospective and seeks to test mastery of the secondary school curriculum; the second stage examination is constructed, offered and graded separately by each higher education institution. These second-stage examinations are partly retrospective, but in many cases they also attempt to be prospective, trying to forecast candidates' potential fit to the future course of study. One effect of the two-stage plan has been to transform the JFSAT into a preliminary qualifying examination, and although the plan has enabled the institutes of higher education in Japan to retain a good measure of control over their student recruitment, the tradeoff has been the significantly higher resource costs that are now involved in selection for post-secondary education in Japan. A large share of these resource costs is borne by candidates and their families, who invest time and funds in one-on-one coaching, after-school schools (the famous juku), and the expenses of travel to distant cities to sit for the second-level examinations. Nor is the total of these costs negligible: they can run to the equivalent of many thousands of US dollars for one family.

Of the eight countries in our study, Japan and the United States are the only two to have adopted a virtually exclusive multiple-choice, machine-scorable format for the university entrance examinations. The Japanese appear to have been persuaded, along with the Americans, that such tests are more objective, provide higher levels of comparability across candidates, and are generally more efficient to administer by examiners who are facing hundreds of thousands, if not millions, of candidates. Perhaps more than in the United States, the Japanese have paid a heavy price for these benefits, producing tests that require candidates to memorise vast quantities of 'facts', thus downplaying originality and flexibility of thought.

In addition, there are important intangible costs arising from intense competition for places in the best universities and the resulting academic and psychological pressure on candidates. Indeed, the competition is so intense and the pressures are so great during the secondary school period that the universities complain that students arrive burnedout, determined to make up for their lost youth, and unwilling to continue to study hard. The contrast with the United States is sharp. There, the complaint is about the lack of challenge that many high schools offer to their students, and the shock that college freshmen can receive when confronted for the first time with major demands upon their time and intellect.

France

Over the past four decades, France has placed increasing reliance on the school system (as distinct from employers) to provide its generally buoyant economy with trained labour. As a consequence, more young people are carried further in school to a degree that would have been very difficult to predict from the France of the 1950s. Moreover, as youth unemployment has become ever more worrisome, the schools have been pressed to tailor their curricula and organisation to the desires of employers, a development seen in many other countries, too. ⁶ One consequence of the vastly increased numbers and new types of candidates finishing a full secondary education has been a recognition of the limitations of the academically oriented Bac examination of, say, 1950, in the changed circumstances. ⁷ The French solution to this problem has been to retain the Bac in form, but to furnish it with substantial new content. In a major effort of educational adaptation, a Bac that had been organised on a narrow humanities- and mathematics-oriented basis and was easy to comprehend, has become an extraordinarily differentiated and complex examination system, with a host of séries, lignes, and options (38 in 1988, compared to just four before 1950). Although in some respects the French system has now moved some distance towards the English specialised model, this should not be exaggerated, as the Bac retains a large core of general education subjects required of all candidates. Nevertheless, one can no longer speak of a single nationally comparable examination administered to all candidates. Instead, a strongly demarcated hierarchy of prestige has emerged, with the mathematical options at the head, and the vocational options forming the tail. We should also note that, in spite of the persistence of a common core of subjects, the highly differentiated Bac has provoked fear that France is dissipating her intellectual patrimoine. Whether this is a benefit or a loss is, we suppose, a matter of taste, but taken simply as a matter of fact rather than of values, French culture générale has become a little less générale. The changes made in the Bac represent bold and on the whole successful moves, but they have been achieved at some price.

The most obvious has been a loss of comparability across candidates, who take widely different assortments of subjects, different papers in nominally the same subject, with different weights given to the results, depending on the particular option.

In addition, the limited devolution of administrative authority from Paris also extends to the administration of the Bac. Each of the 23 académies selects its own assortment of questions from the centrally approved list, and has some latitude to set its own standards of grading. Some académies have even acquired reputations for their relatively lenient standards. An examination system that began with strong commitment to strict comparability across the entire country (including even overseas departments and dependencies) has had to yield up that important value.

Last, as has occurred in so many other countries, the opening of broader access to the Bac has produced a flood of students for the French universities. Yet the university system has been held on an exceptionally tight budget rein by successive administrations. The result has been the development of scandalously poor conditions of study for many students, especially those in the universities situated in and around Paris. Competition for entrance into the better provided areas of the universities and particularly into the grandes écoles has intensified to the point that the Bac has become a kind of first-level qualifying examination, with the decisive examination being either the concours, or the examination for admission to a grande école. This represents a devaluation of the Bac, perhaps an inevitable cost of its democratisation and of its extension to the vocational tracks.

Federal Republic of Germany

The expansion of secondary education in the FRG took place a full 15 years later than in France, and, as with the Bac, the Abitur examination has been significantly altered to cope with the increased numbers and the changes in the kinds of candidates.⁸ However, the Abitur was less radically altered than the Bac, perhaps because the need in the FRG to secure some measure of agreement among 11 quite differently oriented Länder governments always tends to put a brake on change. In addition, while the Bac is marked by a generally high degree of central direction, a determining characteristic of the Abitur is its school-based control. In matters of format, both France and the FRG (in distinct contrast to the United States and Japan) have retained a traditional approach, relying on extended answers to questions, even to the extent of maintaining a certain reliance on oral examinations. Especially in the FRG, the combination of local control and written and oral examinations raises questions about the extent to which grading standards are kept consistent even within a given Land, let alone across the Länder. Because the FRG makes less effort compared even with France to ensure such standardisation, an important element of chance and arbitrariness has developed.

Since 1979, the demands made on candidates for the Abitur have been reduced. In particular, they have been permitted to offer selected subjects at lower levels of difficulty. The changes have encouraged a vast expansion of the number of young people completing their secondary education with the Abitur, but again, as in France, certain costs have been paid for this advance.

First, and most dramatic, has been the need to introduce restrictions on the right of the Abitur holder to enrol in any university faculty of his/her choice, the so-called numerus clausus. The lines of study affected are those that carry high social prestige and/or offer the opportunity of high earnings in the future (and which are also extremely expensive for the state to provide), for example, medicine and dentistry. Since the legal entitlement to a university place was probably the most cherished aspect of the Abitur credential, the price paid for expansion has been high.

In addition, admission to the faculties and departments under numerus clausus is determined via a highly complex points system, that takes into account not only the marks received in the Abitur (with differential weightings for different subjects for candidates choosing different sets of major and minor subjects), but also school grades. Even so, because competition is so intense for admission to some faculties, tiny fractions of a percentage point in the final placement can become critical in the admissions process. Another cost of the changed system and the introduction of numerus clausus is that some candidates with the highest scores are entering the most favoured and prestigious faculties, even though their interests, aptitude or previous educational specialisation may lie in other directions. They are simply reluctant to 'waste' high standing in the admissions ranking on a place in a less prestigious line of study. This clearly is not a welcome development, especially in such fields as medicine.

These problems have caused the Council of Ministers of Education to reconsider the changes made in the Abitur. In the autumn of 1987 decisions were taken that amounted to restoring some of the older standards and regulations, especially limiting candidates' freedom to select subjects at lower levels of difficulty.

People's Republic of China

Examinations for entrance to higher education and higher level technical training in the Secondary Technical Schools serve primarily to control access to severely limited resources, already strained by a shortage of well-prepared teachers, inadequate buildings and equipment and out-of-date libraries. In the current drive to modernise the Chinese economy and the armed forces, the pendulum of Chinese higher education admission policy has once again swung to an extreme position. Ideological faith and socialist 'good works' now count for little; nor is peasant origin any longer so helpful. Instead, admission depends on success in the examinations at the end of senior secondary school. 'Expertness' is valued more highly than 'redness', at least for the time being. This policy of placing student ability above political orthodoxy as the major criterion for advancement is a political risk the current administration has chosen to run. ⁹ More even than in France, the basic stance of the examination system in China is one of rigid central control and uniformity of administration and content, although some devolution of authority has been made to Beijing, Shanghai and Tientsin. Partly this characteristic builds upon traditional state practices, but partly too it is based upon a desire to select on a strictly equitable basis the best and the brightest of Chinese youth for university education. The number of candidates is exceptionally large (in 1988 2.7 million prepared for the national college admission tests), of whom only a quarter will be accepted for study. ¹⁰ Overall, about 2% of first graders go on to higher education. The combination of intense competition and virtually nationwide uniformity of the examination leads to pressures on students that are every bit as severe as in Japan.⁹

The Chinese authorities have introduced substantial elements of multiple-choice and short-answer questions into what had previously been a traditional extended-answer type of examination. They have not yet moved to machine-scorable formats. Given the large numbers they are presently dealing with, the costs of grading and administration must be burdensome. As the number of candidates increases in future years, the temptation to move to machine-scorable tests and the attraction of the low marginal costs associated with such formats must grow ever greater. At least one student of examination policy in China has predicted that the time is not far off when the pressure of numbers on the examiners will become irresistible, and a vast school population of multiple-choice, machine-scored examinees will be added to those of the United States and Japan. ¹¹ If so, there is a distinct danger that the changed format will reinforce the already strong emphasis in Chinese schools on rote learning and the recall of 'facts'.

Soviet Union

As in so many other respects, the Soviet Union provides a sharp contrast to China. Though there is significant influence exerted from Moscow, each of the 15 republics is responsible for setting the content and standards of the secondary school examinations for the leaving certificate, the attestat zrelosti. Schools work within the Republic guidelines, but in turn enjoy a good deal of local discretion. The teachers who have prepared the students dominate the process of setting the questions and evaluating the responses. Paradoxically, in a society and a school system that are in most respects characterised by substantial central direction, the school completion examination is not. Thus, the Soviet Union has settled for a curious compromise between the rhetoric of centralised planning and the practice of local discretion. ¹² The tradeoff for such local discretion is a substantial loss of comparability of marks, and this has led the VUZY (universities and technical institutes) to insist on applicants sitting for special, institutionally set and graded entrance examinations, very much along Japanese lines. As in Japan, the examinations are highly competitive and can impose substantial travel costs on students. There is virtually no coordination among the VUZY concerning the dates on which they will hold their examinations, examination syllabi are idiosyncratic, and grading formulas, cut-off points, and so forth, confidential. Glasnost has much work yet to do in this comer of Soviet life!

A consequence for many Soviet youngsters has been to turn higher education admission into a process incorporating large elements of game-theory, almost textbook examples of decision-making under conditions of imperfect knowledge and uncertainty. Apart from the persistent reports of discrimination against certain ethnic and religious groups, influence peddling and corruption, the system appears to lack important elements of overall fairness and objectivity.

Sweden

In 1971 Sweden introduced a new form of upper secondary school, designed to continue the education of a very large fraction of the post-compulsory age-group, and integrating three formerly separate types of schools (gymnasium, vocational and continuation school). The various study courses of those schools were largely imported unchanged into the new integrated school, represented by 27 so-called 'lines' of two or three years duration. Some of these lines are highly academic (especially the natural sciences line); others are more vocationally oriented; yet others are 'general'. In the mid-1970s, Sweden discarded a limited but usable final secondary school examination system in order to reduce the strain on pupils, produce more valid and reliable predictors of university success, and (it was hoped) correct socio-educational inequities in assessment. In place of the final examination, the Swedes installed a combination of marks gained during regular classroom and home work and in nationally set tests administered at intervals during the school career. Meanwhile, in 1977 a major reform of the higher education system revamped admissions criteria. Work experience and age (maturity) were given strong weight in the admission decision, and completing less demanding upper secondary school lines was not by itself sufficient for consideration for admission to higher education. Additional study and credentials were demanded.

Abandonment of final examinations was also motivated by the desire to improve the diagnostic and predictive value of tests of individual student achievement and to give teachers national benchmarks against which to set their own pedagogical efforts.

The Swedes have been willing (and able) to incur rather heavy costs to achieve these goals. The new system requires time-consuming collaboration among teachers in a given school, and across schools in a region. Exceptionally detailed record-keeping is required, and the Swedish National Board of Education is charged with the responsibility of preparing and standardising the tests given in the basic school subjects at various points along the school ladder. Although Sweden may have abandoned its final secondary school examinations, there has been no abandonment of tests and examinations in general. Indeed, one might well argue that there is now more examining and evaluation based on tests and examinations than ever before.

In 1987, it was announced that examinations and testing in the upper secondary school will be complemented by an assessment programme for the compulsory school years, to begin with the 1988-89 school year and to take in successive grades. All grades will be covered and reported on at three-year intervals. This so-called 'National Evaluation to Give a Holistic Picture of School Activities' will not confine itself to the academic side of the school, but will include data on the social and home environments of the pupils, their health, their social and emotional development, and their attitudes.¹³ What is being proposed is a massive national enterprise, carried out to an exceptionally thorough degree, and demanding the expenditure of very significant resources, both human and material.

External examinations have thus been replaced by continuous assessment, nationally planned but locally executed. In all of this, the authorities are seeking to ensure that the decentralisation of school administration in Sweden will not lead to unacceptable degrees of inequality of provision. Regular evaluation of the entire Swedish school system is supposed to provide the data on which any necessary remedial actions can be based.

Such a thorough-going system of sustained scrutiny raises a question for Sweden which has current relevance in many of the states of the United States, and which soon may become important in England: How much attention by way of tests and examination can a school system stand before it becomes over-routinised and overpreoccupied with frequent, probing testing? Is there a danger of turning education into mere instruction? Our own view is that the Swedes are indeed risking the payment of a very high price for their commitment to constructing a comprehensively detailed data base of the performance of their schools, their teachers, and their school children.

England and Wales

Recent developments in English secondary school examinations, the GCSE, A and AS levels and the introduction of plans for regular in-school assessments have been very contentious. Parallels with Sweden are striking, though the two countries have arrived at their present policies starting from distinctly different traditions of educational administration.

Without going into the details, the events of the last couple of years represent an abrupt acceleration of what has otherwise been a glacially slow process of transferring authority over the schools from local to central government. In the interest of establishing national standards, voluntarism and localism are being forced to give way.

Ever since Robert Lowe, Kay Shuttleworth and Payment-By-Results in the mid-nineteenth century, it has been clear that examinations and testing could be used effectively in England as a lever to change the way in which the schools operate. Since the demise of the General School Certificate and its associated London Matric regulations, it has not been necessary for a given student to take any particular subject or to follow any particular syllabus within that subject, except in so far as he or she wanted to take an examination, or the school demanded it. The cost in terms of lack of coherence in school curricula and indefiniteness of expectations of what the schools should be doing has now been judged by the government to be too great to be supported any longer. The new structure of examinations is intended to help implement what amounts to a national curriculum, though that new structure is not likely to be accompanied by any change from the traditional extended-answer format. Perhaps it is worthwhile noting that a much greater effort has been made in England (both within and across the examining boards) than in either France or the FRG to ensure standardisation of grading criteria. For this reason, some of the more serious doubts about the fairness of marking that are voiced in these last two countries have been absent here.

The changes already implemented or foreseen for the end-of-secondary school examinations in England should be seen as part of a proposed comprehensive assessment procedure throughout the nation and for the mass of the school population.¹⁴ This last will be a major innovation in the English context, and will be one that is likely to come, at least initially, with a high price tag attached in terms of professional morale. Morale among teachers and head teachers in England has already fallen to levels not seen in the entire period since 1870, when the state-supported education system was established. Some will argue that comprehensive assessment procedures are necessary, because only if it can be publicly demonstrated that the schools are returning value for money will the teaching profession be accorded the respect, appreciation and material rewards that it deserves. And, it is argued, a recovery of such respect is the precondition for a major upturn of morale in the profession. This justification for what has become known as 'accountability' is heard on all sides in the United States, and is very strongly sounded in Sweden. However, it fails to explain how more intense scrutiny of the work of teachers is going to help them achieve that essential characteristic of respected professional status, a large degree of personal autonomy in deciding how professional practice shall be carried out. For this reason, it will be advisable in England to take careful note of Swedish experience. Because they are very far ahead along the road of examining, testing and evaluation that is currently being charted in England, any drawbacks (or remedies) the Swedes discover en route should be quite instructive here, too.

Conclusion

In this paper, we have sought to show how important characteristics of examination policies and practices in each of eight countries can be usefully viewed as compromises, or tradeoffs, between desirable alternatives. Such tradeoffs are inherent in trying to negotiate four contemporary and well-nigh universal dilemmas of examinations policy. Each dilemma has been created in large part by the pressures of universal secondary education, and the consequent expansion of higher education.

Dilemma 1. A traditional examination that was used in many countries to select from a small secondary school élite an even smaller élite to enter higher education has been recognised to be inadequate to cope with vastly greater numbers and different types of candidates. Yet it has proved difficult to widen access to the examination without some devaluation of the credential gained.

Dilemma 2. Examination uniformity is desired in order to promote comparability of marks, as well as to help create or conserve a common national culture. Yet, reconciliation of these desiderata with demands for diversity to meet special subnational needs and with the increasing diversity of the secondary school curriculum has not been easy to accomplish.

Dilemma 3. Because of the powerful influence that examinations have on the way teachers teach and students learn, it has been difficult to make use of the new technology of testing without adversely affecting the purposes and styles of schooling.

Dilemma 4. As secondary school coverage has grown, education becomes an ever more critical national concern and its costs come to represent a significant charge on tax revenues. Governments see the schools as part of the struggle for international competitiveness, and they are sensitive to demands that taxpayers should receive 'value for money'. Hence the emphasis on 'accountability'. But governments' readiness to use examinations as a device to monitor and change education tends to undermine the professional autonomy and status of school personnel. Yet strengthening professional autonomy and status may well be a necessary condition for successful operation of an effective school system in the contemporary world.

While not every one of the eight countries examined here represents these dilemmas and the consequential tradeoffs with equal force, taken together they demonstrate their importance in any comparative consideration of examination policies and practices.

NOTES

1. A version of this paper was presented at the 1988 annual conference of the British Comparative and International Educational Society, Bristol. It will be published in the proceedings of that conference and permission to print it in Oxford Review of Education is gratefully acknowledged. A companion paper providing comparisons of national examination systems is Max A. Eckstein and Harold J. Noah (1988) "Changing forms and functions of secondary school examinations: an international comparative perspective," paper presented at CEDAR conference, University of Warwick, England (mimeo.). Support from the Spencer Foundation for the conduct of the research on which both papers are based is gratefully acknowledged. [BACK]

2. We acknowledge the valuable contributions made to our thinking on the topic of tradeoffs in examination policies by Juan Manuel Moreno, Universidad Nacional de Educacion a Distancia, Madrid. [BACK]

3. John A. Valentine (1987) The College Board and the School Curriculum (New York, College Entrance Examination Board). [BACK]

4. "...although NAEP claims not to be in the curriculum business, it may enter if NAEP assessments are used for interstate comparisons. Subject matter that is assessed on these 'national achievement tests' is critical because it is likely to represent a 'national curriculum'." Steven F. Ferrara & Stephen J. Thornton (1988) Using NAEP for interstate comparisons: the beginnings of a 'national achievement test' and 'national curriculum', Education Evaluation and Policy Analysis, 10, p. 209. See also: Alice J. Irby (1987) "Administration and management of examinations: practices in the United States", paper presented at a World Bank seminar, Lisbon, Portugal (mimeo.). [BACK]

5. Tadashi Hidano (1987) "Admission to higher education in Japan," in: Stephen P. Heyneman & Ingemar Fägerlind (Eds) University Admissions and Standardized Testing: principles, experience, and policy options (Washington, D.C., The World Bank), pp. 9-25. Also: Japan, Ministry of Education, Science and Culture (1988) "A new entrance examination system," Monthly Journal of Ministry of Education, Science and Culture. [BACK]

6. Harold J. Noah and Max A. Eckstein (1988) "Business and industry involvement with education in Britain, France and Germany," in: Jon Lauglo & Kevin Lillis (Eds) VocationaIizing Education: an international perspective (New York, Pergamon), pp. 45-68. [BACK]

7. "Ministry tackles tough exam with pleasing results," The Times Educational Supplement 22 April 1988, p. 14; Jacques Colomb (1987) "Examinations and selection for higher education" (Washington, D.C., World Bank) (mimeo.); Ministère de l'Education Nationale (1988) Baccalauréat de l'enseignement du second degré (Paris, Centre National de Documentation Pédagogique). [BACK]

8. Andreas Flitner (1987) Für das Leben, oder für die Schule?, pp. 140-163 (Weinheim, Beltz). See also: Hartmut von Hentig (1980) Die Krise des Abiturs und eine Alternative (Stuttgart, Klett-Cotta). [BACK]

9. People's Republic of China (1986) A Brief Introduction to Higher Education Enrolment Examinations in China (Beijing, The State Education Commission of the People's Republic of China); Keith Lewin and Wang Lu (1988) "University Entrance Examinations in China: the Quiet Revolution," paper presented at the annual conference of the British Comparative and International Education Society, Bristol (mimeo.). [BACK]

10. "Beijing Journal," The New York Times, 12 July 1988. [BACK]

11. S. Heyneman, Remarks at the conference of the Comparative and International Education Society, Atlanta, Georgia, April 1988. [BACK]

12. Wolfgang Mitter (1976) Secondary School Graduation: university entrance qualification in socialist countries: a comparative study (New York, Pergamon). [BACK]

13. Swedish National Board of Education (1987) National Evaluation To Give a Holistic Picture of School Activities (Stockholm, Author). [BACK]

14. Jo Mortimore & Peter Mortimore (1984) Secondary School Examinations, Bedford Way Papers 18 (London, Institute of Education, University of London); Caroline Gipps et al. (1986) The GCSE: an uncommon examination, Bedford Way Papers 29 (London, Institute of Education, University of London); Michael Kingdon & Gordon Stobart (1988) GCSE Examined (London, Falmer). [BACK]