Part III: Achievement, Assessment, and Evaluating Learning

Comparative School Achievement
National Case Study Report
International Study of School Achievement
Reflections
The Two Faces of Examinations
Tradeoffs in Examination Policies: An International Comparative Perspective
Secondary School Examinations: International Perspectives on Policies and Practice
An International Perspective on National Standards
A Comparative Assessment of Assessment
An International Comparison of End-of-Secondary School Examinations

Source: Excerpted from Max A. Eckstein and 
Harold J. Noah, Secondary School Examinations:
International Perspectives on Policies and Practice
(Yale University Press, 1993): 36-40, 68-74,
218-224, 226-232, 243-245. Reprinted by permission
of Yale University Press.

SECONDARY SCHOOL EXAMINATIONS: INTERNATIONAL PERSPECTIVES ON POLICIES AND PRACTICE

[From Chapters 2 and 3: The Candidates and their Schools]

Sweden

Sixten and Ingrid live in the northern outskirts of Stockholm, close to a lake in the suburb of Stocksund, where almost everyone seems to have a boat of some kind. Both students are in the final year of their upper secondary school courses, though Sixten's has been a long (four-year) course, while Ingrid is finishing a two-year program.

They entered school at age seven (after two years spent in kindergarten) and completed the compulsory, comprehensive, "basic" nine-year school attended by all young Swedes. They then transferred to their present school, a Gymnasieskolan (an integrated upper secondary school). Admission was virtually automatic, taking place without entrance examinations or other formalities. In the country as a whole, very few (only about 10 percent) of the graduates from the basic school decide not to continue their education into upper secondary school. In Stocksund the percentage of noncontinuing students is minuscule. This is the point at which specialization begins in the Swedish school system. There are twenty-five different upper secondary school "lines" to choose from, some academic and some vocational. Courses last for two, three, or four years, according to the type of course.

Ingrid is eighteen years old. Even though her parents had higher hopes for her career, she chose a relatively undemanding two-year line, nursing. Her father, an official in the central government's National Board of Education, died when she was eight years old, leaving her mother to finish her qualifications as a general medical practitioner while raising their only child. In her early teenage years, Ingrid became very beauty- and fashion-conscious and spent long hours immersed in glossy magazines -- Swedish, American, German, and French. Although this has given her a certain facility in foreign languages, she was left with little spare time for formal school study, and her grades suffered. If she had wanted to follow her mother's path into medicine, she would have needed to enroll in one of the three-year lines: natural sciences or perhaps liberal arts (classical variant). But with her interests and undistinguished school record, this did not seem realistic.

Ingrid is not especially interested in much of her schoolwork. She quite enjoys some of the romantic literature she has to read in Swedish and English language classes, as well as parts of her studies in biology, since she sees its relationship to her future in nursing. Other work in the standard common curriculum for all students includes continuing study in Swedish language and literature, English, history, mathematics, and physical education. The directly vocational portion of her course comprises theoretical courses in nursing and a practicum in the Karolinska hospital, which she thoroughly enjoys, except for the more mundane housekeeping chores she is asked to perform.

In contrast, Sixten is an academic high-flyer. Neither of his parents was ambitious for advanced education or social status for themselves -- his mother drives a taxi and his father is a maintenance worker in Svenska Handelsbank --but both have marked his lively intelligence and encouraged him in his schoolwork. His grades in the basic school were superlative, and his ambition is large. On entry into the Gymnasieskolan he was undecided whether to apply for the four-year natural sciences line or the four-year technology line. In the end he opted for technology. By the end of his second year, he knew that he would specialize in the final two years in one of the most prestigious and difficult of all lines: chemical engineering.

For the first three years, Sixten continued with Swedish, English, mathematics, and physical education. He also took civics in the first year, and studied a second foreign language (German), history, basic chemistry, and technology for the first two years. In the third and fourth years, he added an array of chemistry courses (physical, organic, electrical engineering, biochemistry, chemical technology, technical chemistry), either for one year or both.¹ In addition, now, in his final year, he is preparing his special project paper on some of the chemical engineering problems to be faced as Sweden proceeds to deactivate its nuclear power reactors.

There are no final leaving examinations at the end of secondary school for either Sixten or Ingrid. Instead, they have taken tests and examinations throughout their courses of study. For example, Ingrid has already taken tests set by the National Board of Education in Swedish language and literature in her final (second) year. The English-language achievement test is scheduled for March 15. Sixten took similar tests in English, German, and chemistry in his second year and in Swedish, mathematics, and physics during his third year. The tests are typically composed of a mixture of multiple-choice and extended-answer items, three hours for each subject. The dates for these tests are set for the entire country by the National Board of Education² and are spread over the academic year. This is done in order to make the testing process part of the ordinary school routine and to reduce the tension usually associated with a marathon period of testing crowded into one or two weeks at the end of the school career.

In addition to these "national" tests, a minimum number of other written tests in subjects is required, usually two or three a year. These may be set by the individual subject teacher alone or by all the teachers of that subject in a school, or occasionally for an entire school district. Grades are assigned on a five-point scale from a low of one to a high of five.

In this, the final year of school, all that remains is to complete the course of study. The award of a leaving certificate is made on the basis of satisfactory achievement over the entire course of study taken as a whole. Ingrid is making the grade, though not easily. She has improved her work since basic school and now has good hopes of finishing school with a respectable leaving certificate. This requires at least a three in all major subjects. Sixten, even with all his talent for study and exposition, is having a hard time keeping up with the enormous amount of work that is expected of him. He will soon be twenty, and he has been stretched academically for the past four years, ever since he left the relatively relaxed basic school. There he had grown used to getting fives without too much trouble. It was an effort to keep up his general education subjects for the first three years. If anything, his study load in this final year is even heavier, especially as he aims to submit an outstanding special-project report. But he is devoting all of his time to technical subjects where hoped-for high grades will be important for his application to a university. His parents are justifiably proud of the grades their son is bringing home, a mixture of fours and fives.

Swedish government policy in education aims at equalizing access to post-secondary education, irrespective of the location of students' homes and schools. While parts of Sweden, especially in and around Stockholm, are highly urbanized and suburbanized, the country is large (one thousand miles from north to south) and mostly quite sparsely populated. Both students feel that living as they do in Stocksund, with its generally higher standards of income and education, they probably have to try harder in order to achieve the best marks in their schoolwork and their examinations than students from less favored parts of the country.

With all the undoubted importance of the tests, examinations, and marks, neither Sixten nor Ingrid feels the competitive urgency experienced by the Chinese and Japanese students we shall meet later. Their records have been accumulated over several years and include an exhaustive record of their experiences and accomplishments, grades on school, district, and national tests, and comments by teachers. They also include notes on periodic discussions about career preferences. Although Sixten particularly feels some anxiety about his immediate future, compared with typical secondary school completers in most other countries he and Ingrid are able to take matters very much in stride. The final year of school is important, but not absolutely critical to their futures.

They are confident that they will be able to pursue whatever careers they choose without too many hurdles placed in their way. They can always try a second time, and even a third, if their application for higher education or further training is turned down. There are ways of entering alternative career paths if they should change their minds later on.

Most important, perhaps, Ingrid and Sixten are not moved by much fear of unemployment after school. Ingrid is well aware that there is a severe shortage of nurses in Sweden and knows she will find a warm and ready welcome in a hospital once she graduates. She has had enough of being a full-time student, she feels, and has no intention of going on to a university, which is just as well, as her grades make it unlikely that she could be offered a place. Entry into hospital nursing will provide further training and a salary, outstanding working conditions, and a job she likes. However, if she changes her mind later in life, the door to higher education is not closed. Since autumn 1991, students who graduate from the upper secondary school with low grades have had an opportunity to take a proficiency test for university entrance to demonstrate their improved academic aptitude.

Sixten has his sights set on continuing his studies in chemical engineering at the university. But his leaving certificate will also confer on him the title of Certified Upper Secondary School Engineer. So if by remote chance he does not get a university place immediately, he expects to be able find employment without difficulty (after doing his military service). In any event, as soon as he is twenty-five, he can apply again to enter the university under the 25-4 rule (minimum age twenty-five; minimum work/military service of four years), which reserves a number of places for people like himself. His prospects, he has every reason to believe, are very good.

These two young Swedes have been educated in a system that strives to provide wide educational opportunities to all. Tests and examinations continue to exist, but a deliberate effort is made to ease the psychological strain on students and to make the tests as nonthreatening as possible. The central government and the localities (municipalities) collaborate in providing the examinations, with the close involvement of teachers. A student who at first does not succeed can try again. The examination results are important, but do not virtually control an individual's future, as we shall see is only too frequent elsewhere.

Japan

Kohji and Miyoko's teachers and parents have consistently held them to high standards of comportment, study, and work, and have encouraged them in an impressively competitive spirit of academic achievement. They live and go to school in Kyoto, one of Japan's most important cities. Kohji's father is an investment manager in Sumitomo, a Japanese bank. He has worked there ever since he graduated from his university twenty-five years ago, and he expects to remain with the firm till he retires. Miyoko's father runs a small automobile repair garage, employing four mechanics. In both families, the mothers are at home, even though Kohji's mother had completed four years of college in foreign languages (English and German). The two youngsters are now in the final year of the kotogakko (senior high school, grades ten to twelve) in company with students much like themselves, with excellent school records and success in examinations. Each day, Kohji and Miyoko travel over an hour to the other side of town to get to their school, but they feel that the privilege of attending this institution is well worth the effort. Their school has an outstanding reputation based on its record of student success in the university entrance examinations.

Transfer from junior to senior high school is, as a rule, based in part on the results of achievement tests at the end of grade nine, and in part on high school entrance examinations. Miyoko and Kohji were admitted on the basis of their superior marks after three years in junior high school. Local educational authorities follow slightly different practices, but for some years now, most have tried to steer a middle course, trying to avoid the elitism that comes with strict selectivity when transferring students to senior high school and at the same time placing a limit on the extent of student heterogeneity within a school. Local policies thus make use of several criteria for transfer: examination results, parents' preferences, and the desired range and mix of student abilities in each school.

School attendance is not mandatory beyond grade nine and parents of high school students must pay fees. Nevertheless, the majority of sixteento-eighteen-year-olds are enrolled in the general, academic full-time course of study in a senior high school, and about 95 percent of the age group complete secondary schooling through grade twelve. A small number are part-time students and some are in technical schools.

If Kohji had not done well enough in the entrance examination to gain entrance to this more selective public school, his father would undoubtedly have made the effort to pay the substantial fees at a private school. There he would receive better preparation for the university entrance examinations than the other public secondary schools in town would provide. Miyoko's parents, however, would have given this a lower priority in the family budget, reasoning that girls do not need to have the same educational opportunities as boys.

The public school system in Japan is directed from Tokyo by the central Ministry of Education (popularly known as "Mombusho"), but which delegates administrative responsibilities to prefectures and towns. But major educational policies, such as curriculum structure and content, textbook approval, school organization and schedules, teachers' credentials and pay, are all determined and controlled by the ministry and are uniform across the nation. General policies are determined by national committees, discussed and voted upon by the Diet, and often questioned and vigorously opposed by the national teachers' union.

Kohji and Miyoko's curriculum for the first two years in senior high school differed little from everyone else's: Japanese, social studies (history, geography, and civics), mathematics, science, health and physical education, arts (music, art, calligraphy), and a foreign language (English). Miyoko, like all the girls, also takes home economics. Only in the final year have a few options become available, providing some opportunity for additional, more advanced work in either science and mathematics or the humanities, related perhaps to their target for the future. Miyoko chooses an elective social studies course, and Kohji takes additional mathematics at a more advanced level. These choices account for only 10 to 15 percent of their school timetable, however, and most of their classes are the same as all other students', using the standard textbooks and following the ministry's prescribed syllabi.

Kohji and Miyoko's memories of the more relaxed primary years, when school was fun, have faded. Junior high school had been something of a shock, with its increased pressure to perform, regular formal tests, and emphasis on preparation for the third-year examination. Some of their peers became demoralized by the change of climate, some even taking to such antisocial behavior as bullying their weaker classmates (ijime). Senior high school has intensified the pressure.

The thirty-five-week school year opens at the beginning of April, with classes running from eight to three, five days per week, plus a four-hour day on Saturdays. With six periods in a complete day, this provides for thirty-two classes a week of fifty minutes each, plus school clubs and activities. After school closes, Kohji and Miyoko, like more than half of their class, go directly to a juku. This is a private school where students may choose from an array of craft, aesthetic, and enriched study classes. But a major focus of this extracurricular system is intense instruction to reinforce what was taught in school and training in the techniques of taking the public examinations. The fees are high, but their parents do not begrudge the expense. Kohji arranges through his juku for his scores on practice tests to be sent to a company that specializes in predicting examination success. The company will also advise him about which universities he should apply to, given his scores and other personal details. Kohji attends the juku three afternoons during the week, Miyoko on Saturday afternoon. Only when they get home will there be time to sit down to their homework. They do not find this routine unusual since they have been attending such after-school classes since sixth grade: Miyoko attending music lessons, and Kohji classes in general school subjects. It is simply a part of the daily program and duties and the normal expectation among friends and family. Indeed, if they were not enrolled in juku, they would feel rather left out of things.

Much of the final year in high school has been directed at preparing for the entrance examinations for higher education. By now Kohji and Miyoko are accustomed to the daily grind, and they have become most efficient at the memorization and rote learning that characterize their schooling. The pace of work and intensity of study, which have been heavy all along, have become even greater -- about sixteen hours a day³ -- and there is less time for relaxation, for meals, and even for sleep. There has been no time for social activities for quite a while; now even chatting for a few minutes after school is seen as wasting time and provokes feelings of anxiety and guilt. They hear stories about classmates who have succumbed to physical and mental illness due to the pressures. The press adds to the stress by reports of student suicides -- but they have no personal knowledge of anyone who has been that despairing.⁴

Kohji has been advised by the company that has analyzed his juku scores that he stands a chance of gaining admission to Tokyo University (Todai), a national, public university and one of the most highly regarded in the country. Miyoko also hopes to attend a public institution, though she recognizes that given her family's unwillingness to send a girl to a four-year degree-granting institution, she cannot aim very high. The first step is to register to sit for the Joint First Stage Achievement Test (JFSAT), organized by a special department of the Ministry of Education. The JFSAT is not needed for graduation from high school, nor do most private colleges and universities require it of applicants. But it is necessary as the first stage of application to the public institutions. So, halfway through the school year, in October, the two students submitted their application forms to sit for the JFSAT. Now, at the end of January, in common with about four hundred thousand of their peers across the nation, Miyoko and Kohji are taking the JFSAT at their school. Candidates take tests in six or seven subjects. There are some variations: for example, Kohji sits for a more advanced paper in mathematics, while Miyoko chooses a European History option.

The results of the JFSAT and the dates of the entrance examinations set by the universities are announced. Many of Kohji and Miyoko's contemporaries have obtained good results on the national examination. The Todai entrance examination will be conducted by the university in two months. Kohji signs up with a travel agency that is assembling a group of high school seniors to travel to Tokyo for the examinations and a few additional days in the capital. The tour package includes rail fare, hotel accommodation, meals, transportation from the hotel to the university's examination halls, and a sightseeing trip before the return home. The entrance examination will last two days, and applicants may choose to be tested in two or more subjects -- some choose as many as five. Kohji will be examined in four subjects, including an advanced test in mathematics. Todai usually has about four applicants for every place offered, so this will be an exceptionally competitive examination, much harder than the JFSAT.

Miyoko takes only the JFSAT, for she does not expect to be able to apply to a college or university that requires a special entrance examination. For those who apply and are accepted, there is the prospect of four relatively relaxed years, beginning the social life they had little time for during the demanding years of secondary school. Now there should be time for dating, joining clubs, going to movies, and having fun, since the demands of college work are nothing compared with what they have endured in high school preparing for the entrance examinations. Moreover, for those who have been admitted to their chosen department in their chosen university, the future is already secure: they will have little difficulty obtaining a job that will earn them security and a good income.

Although Kohji did well in the JFSAT, he does not secure a place at Todai. He is utterly disappointed and ashamed. He feels he has let down his teachers, his school, and his family even though everyone knows how intense the competition is. Despite the hard work and late nights of study, he can only conclude that he did not make a sufficient effort. He resolves to try again and to study even harder. About a quarter of all candidates in the examination repeat it. To join the ranks of the repeating candidates (so-called ronin) is not dishonorable, and Kohji has been told, too, that almost half the applicants admitted to Todai and to medical colleges across the nation, which are in especially great demand, are ronin.⁵ The chances of getting into a good university program are certainly enhanced by an improved second set of results. Driven by the desire to erase his failure and motivated by his interest in a math/computer science career, Kohji immediately enrols in the yobiko (a crammer solely for university entrance examinations) to prepare for another attempt a year hence.

The yobiko fees and the cost of the study materials will be high, but Kohji's parents consider it well worthwhile. Good results mean entry to a top national university, where fees are low, quality of staff, facilities, and education are, superior, prestige is high, and career opportunities are outstanding. Provincial universities do not offer these advantages, and set a clear limit on what careers will be available and the extent of advancement that may be expected in the future. Although some private universities have high status and thus offer better opportunities, they are very expensive; some of the others are viewed as second-rank or below.

If worst comes to worst and the results are not good enough for Kohji to be accepted in the course he wants, he may even take the examinations yet again, for there is no limit on repeating and students on a waiting list may apply for admission to a university as often as they wish until they succeed. However, though first-time repeaters are frequent, and secondtime repeaters are common, few persist beyond that point.

For a young man, every effort must be expended to enhance the chances of entrance to the best university possible, for his whole future depends on it. But this is not so for daughters. In Miyoko's family, discussions over the examination results and her future are much briefer and take a different turn. While not outstanding, her marks are quite good and she has earned the praise of her family and friends. Though it is no longer rare for girls to continue their education, most still do so in junior colleges and in programs traditionally considered suitable for females: primary school teaching, nursing, business skills (secretarial work), and the "softer" subjects like the arts, humanities, and social sciences. Secretly, Miyoko had hoped to take a humanities degree in the provincial four-year college, and her marks are more than adequate for this. But her father insists that she attend the nearby junior college and study business skills, so that is where she will go for the next two years. If her marks are good, if her ambition and intentions remain strong, and if she can persuade her parents to give their permission, she may yet be able to attend the provincial university and obtain the degree.

The pressure exerted by examinations in Japan is extraordinarily high. They control the lives of students (and their parents) to a degree that is very rare in the other countries of this study. Indeed, it is not too much to say that education in Japan, particularly after sixth grade, is defined as preparation to take examinations. Secondary school students are provided with a very clear set of objectives, and they are expected to make the effort necessary to achieve them. Only lack of effort, not lack of ability, it is commonly accepted, stands in the way of their success. So, it is to study, more study, and yet more study for tests and examinations that Kohji and Miyoko have devoted themselves for more years than they can remember.

[From Chapter 10: Findings, Options, Trade-offs, and Dilemmas]

Findings

Our study shows that in a number of respects, national examination systems are converging toward common forms and practices. Examination systems that have traditionally been decentralized give strong indications of greater centralization, and those that have been centralized in their control appear to be becoming less so. In every country the secondary school examination system has expanded to embrace a far greater proportion of the population of senior secondary school students. Expansion has been part consequence, but also part cause, of the ongoing differentiation of examinations that traditionally have been quite limited in their range of subjects. The changes have provided candidates with an increased range of choices among subjects and levels of difficulty. Often this has taken the form of including a greatly increased number of employment-related options. Convergence is evident, too, in trends among systems that have relied exclusively on open-ended, extended-answer formats to introduce multiple-choice, machine-scorable items and among those that have emphasized objective tests to introduce more elements of extended answers.

Last, the study highlights the two faces of examination systems. Examinations play a dual role in broader educational change. They can serve both as levers and obstacles to educational change, even within a given country. Although governments are prone to seize upon examination systems as handy instruments for pushing schools and teachers in desired directions, equally common is the resistance mounted to such initiatives by school administrators, teachers, employers, and even parents, using the examination systems to defend the status quo.

Impact and Reach of Examinations In none of the countries studied does every student coming to the end of secondary education sit for an external examination. Large groups are often unaffected. Some have been selected out of the examination-preparatory tracks. Others attend a school that is simply not engaged in the business of preparing students for the examination. Nevertheless, the influence of an examination system permeates a nation's education system. It affects performance expectations, curriculum content, and definitions of scholastic success and failure. Especially for those who prepare for the examination, the impact on school and personal life is likely to be profound, as in Japan, France, Germany and England/ Wales, where successive years of upper secondary schooling are characterized by a crescendo of concentration and effort. But even in other countries, students' lives in and out of school focus upon examinations. There are exceptions, however. Sweden makes a deliberate attempt to minimize the impact of the tests, and in the United States preparing for and taking the SAT and the ACT is far from the defining experience of the typical student's high school career.

Controlling the Examinations We have observed many variations in the distribution of authority and responsibilities for national examinations. No nation has a single, centrally determined examination for all possible purposes. Control and ownership of the examinations can be in the hands of the central government, or shared among subnational units, or in a "quango," or even, as in the United States, in a private not-for-profit corporation. In France, China, Japan, and Sweden, the end-of-secondary-school examination systems are controlled mainly from the center by a special agency of each country's ministry of education. But even in these formally centralized arrangements, there are strong elements of devolution to regional or local authorities. Conversely, even when the system is formally decentralized, usually some oversight is entrusted to a central authority. Thus, in the Soviet Union each republic has formal authority over the attestat zrelosti, but took its cue from Moscow. In England/Wales, several largely autonomous regional examining boards are in control, but they collaborate on administrative matters and examination content, and they are increasingly subject to supervision by the central government's Department of Education and Science through the School Examination and Assessment Council. In Germany, though the Länder retain formal sovereign powers over their Abitur examination systems, they enter into voluntary agreements that in practice limit their discretion. In all these countries, control and ownership of the examinations are in the hands of public bodies, either a government department or semi-independent public agency.

The situation is quite different in the United States. Here, although a few states arrange for end-of-secondary-school achievement examinations, none has national currency, nor does the federal government provide a national examination system for the schools. The United States is exceptional in its reliance on private organizations like the Educational Testing Service and the American College Testing Program to provide end-of-secondary-school external tests and examinations, free of federal direction or control.⁶

Apart from who "owns" the examinations, there is the question of the role of teachers -- mainly, though not exclusively, secondary school teachers -- in performing such specific tasks as constructing test papers, proctoring the examination sessions, evaluating responses, and monitoring the whole process. The latter function can influence teacher-student relationships profoundly. When, as in Japan, China, England/Wales, the United States, and France, all grading of external examination answers is done anonymously, without teacher participation in grading their own students' answer papers, students and teachers can become allies in the business of outsmarting examiners. This is more difficult in Germany and the Soviet Union, where a student's own teachers play a large, overt role in evaluating his or her examination work. In addition, the German and Soviet practice indicates the presence of considerable trust, whether justified or not, in the skill and judgment of teachers, a positive reflection on their professional status. Swedish practice contains elements of all of the above.

The Examinations Examination systems strongly influence the school curriculum, to the extent that in England/Wales, France, China, and Japan the subjects examined and the syllabi for the examinations virtually determine the school curriculum and the objectives of teaching. The format and content of examinations reflect what an educational system considers to be the knowledge and skills of most worth. They also reflect and stimulate curriculum changes. New curriculum elements tend gradually to find their way into examination papers, usually at the expense of traditional ones.

Tests incorporating significant practical work, such as laboratory exercises or artistic work -- so-called performance examinations -- are common in some places, rare or absent elsewhere. Multiple-choice formats are becoming more common, but still remain the exception rather than the rule in the eight nations of this study. Although all examinations test recall and comprehension, more advanced intellectual activities like problem-solving, critical analysis, and creative expression may also be assessed. New forms of evaluation are emerging that involve more than a timed paper-and-pen test. Rather, the new forms take into account an accumulated corpus of the candidate's work in the subject. Some advocate reporting results in more detailed ways, beyond traditional letter or number grades, to reflect fuller "profiles" of achievement.

The Demands Examinations Place on Candidates, Compared Examination systems define a nation's expectations of students and its standards for educational success. Although all systems impose requirements upon candidates, these vary considerably. Some, like France and Germany, call for many subjects, an extensive syllabus, and long examination papers; others are less burdensome, for example, the United States. All systems offer certain options to candidates in choice of subjects and in level of difficulty, some of them very limited, others more numerous. In France scores for a subject examination are weighted differently in the final overall grade according to the candidate's specialization. In Germany and recently in England/Wales, candidates may take a subject as a major or a minor, with different examination papers based on different syllabi.

Comparing the relative difficulty of examination systems involves evaluating their general requirements, the content and the format of examination papers, timing and time limits, and the available options. Although such an exercise includes some objective data, like the number of examination hours, much rests on subjective judgments. We confined our analysis to the two most frequently tested subjects, national language/literature and mathematics. We conclude that end-of-secondary-school examinations are very burdensome in Germany and France, especially when they are compared with the relatively undemanding tests in Sweden and the United States. The content of the tests in Germany and France places high cognitive demands on candidates, extended answers are always required, and the time allotment is prolonged. In all respects, the examinations in Sweden and the United States are much less burdensome. Examinations in England/Wales and Japan lie between these extremes, but toward the more difficult end of the scale; China and the Soviet Union toward the less difficult end. Our conclusions refer only to the national external examinations and not to the university entrance examinations given as a second stage in Japan and the Soviet Union.

Success Rates Compared Some examination systems, like Japan's JFSAT and the United States' SAT and ACT, provide results simply in the form of a numerical score, with no indication whether the score means pass or fail. Other systems, for example, France, England/Wales, and the Soviet Union, provide certification of passing or failing as well as a numerical or letter score. In those countries where the examination serves primarily as a selection instrument for post-secondary education, success is defined in practice as gaining entrance to higher education. In China the score is made known to the candidate, but it is meaningful only as it relates to the cutoff point for admission to college. Therefore, comparative and international study of success rates presents some difficulty.

Differences of organization and practice in secondary school systems, in admission standards to higher education, and in the announcement of results must be taken into account, and complicate comparison even more. Our broad conclusion is not just that success is defined differently but that success rates differ quite markedly across nations and that each nation appears to regard its own rates as somehow part of the natural order of things. Difficult examinations help reduce the percentage of the age group passing, but preselection tends to ensure that success rates with respect to the number of candidates are kept high. The gap between success rates in the United States and Sweden at one end of the scale and China at the other might better be termed a gulf. In the United States and Sweden the doors to post-secondary education are wide open. A high school diploma is not difficult to achieve and there are ways of getting admitted to some college, somewhere, without even completing the standard high school course of studies. This stands in stark contrast to China, where the percentage of the age group entering universities is very small. Our estimate is as low as 2 percent of the age group. We conclude that success rates reflect both national resource levels and the willingness to make them available for secondary and post-secondary education. In this way, success rates can be regarded as artifacts of national policies, for they go far beyond the narrow business of determining how much candidates know and how well they can demonstrate knowledge.

The Politics of Examinations Because we know them to be powerful instruments of control, examination systems can become a focus of attention, even of political debate and struggle. This has been particularly evident in China, England/Wales, and Germany, and the political temperature surrounding examinations is rising sharply in the United States. On the other hand, even though the Swedes have made the greatest change of all by abandoning the traditional end-of-secondary-school examination and moving to a system of periodic local and national assessments, they have engaged in little dispute over examinations. In the Soviet Union, too, although many educational topics have been vigorously discussed, the examination system has generated little if any attention.

Possibly the most important cause of political engagement with examinations in recent years has been the very large increase in the number of students completing secondary education. Examinations that once involved only a tiny fraction of the population now interest a much wider public. Disputes over structure, format, and content of examinations often reflect differences in basic educational philosophy, even standard political ideology. As the number of stakeholders is growing, so are the voices of groups focusing attention on their particular interests and points of view. One important result is an increasing demand for examining bodies to be accountable for decisions and to follow full-disclosure principles as part of normal practice. They are only too frequently caught in a crossfire of charge and countercharge. In the United States the examination authorities, ETS in particular, are criticized for constructing "unfair" tests that are systematically biased against ethnic minorities and women. In some instances, the examiners are accused of overemphasizing the traditional aspects of the examined subjects; in others, the charge is that the examinations have thrown out too much of the traditional canon in favor of faddish approaches and material. In history and language and literature, particularly, the "modernization" of curricula and examination syllabi has been accompanied in the United States, France, and England/Wales by expressions of concern that national identity, national pride, and knowledge of the national heritage have been undermined by the introduction of world history, multicultural history, non-Western literature, language for communication, and the like.

The Persistence of Examinations In spite of the conflicts surrounding them, examinations appear to have great survival capacity. Indeed, they have done more than persist, they have positively flourished. Why? First, they fulfill a number of essential functions for the school system and for society at large, namely (a) assessing student achievement, (b) controlling curriculum and instruction, (c) allocating rewards in education and employment, (d) motivating students and teachers, and (e) validating/legitimating knowledge and educational activity.

In addition, social changes have forced the bureaucratization of relationships that once could be satisfactorily maintained on a personal basis. When only a small proportion of the population stayed in school long enough to become candidates for the end-of-secondary-school examinations, everyone knew everyone else, and attendance at a particular school or the recommendation of a teacher could be relied upon to communicate to university professors or prospective employers a comprehensible and reliable evaluation of a student's preparation and performance. In contemporary society, schooling has become a mega-enterprise, public and complex, touching virtually every citizen and, most important of all, widely regarded as critical for individual and national prosperity. Examinations are expected to provide objective, reliable measures of accomplishment on which rational and efficient decisions about the future of individuals can be based, along with national benchmarks that enable citizens to assess the contribution made by the schools to the total national economic effort.

Finally, examination systems have demonstrated great adaptability in the face of changing circumstances. As the content of work has demanded more specifically technical knowledge and skills, so school curricula and examinations have moved to incorporate more applied and vocational elements into subject matter that was once almost exclusively academic and abstract. To meet accusations of subjectivity and unfairness in grading, examination systems have developed tests that lay claim to objectivity. And, reinforcing the trend from norm-referenced to criterion-referenced assessment, there are now signs of movement away from reliance on one-shot, time-limited tests toward the incorporation of such nonexamination criteria as portfolios, profiles, and school records in the final evaluation of candidates. Sweden is furthest along this road, using both continuous assessment (a mix of centrally and locally set examinations, required and optional) and grades from course work done during regular school activities

Options and Trade-offs

Comparative and international observation can go beyond the obvious "good points versus bad points" approach. A series of "What if?" possibilities can be phrased in terms of the trade-offs implicit in specific examination arrangements in particular countries. This should appeal particularly to policymakers and administrators, who are required to negotiate an acceptable path between opposing, even contradictory, goals as they try to achieve one set of objectives without paying too high a price in terms of other desirable outcomes. As in other aspects of a country's social life, examination policies and practices reflect compromises among such competing values and goals. While seeking to increase perceived benefits in one direction, a nation almost inevitably gives up some benefit or exacerbates some problem in another direction. It is in that sense, therefore, that we can view extant examination systems as configurations of trade-offs. Here, for example, are three such sets of trade-offs evident in many countries:

-- Administering a uniform examination to all, rather than allowing candidates to choose which examinations and subjects to take, and at what level of difficulty, facilitates comparability of scores, but shortchanges individual, regional, ethnic, cultural, and other differences.

-- Machine-scorable formats are economical, but their use risks neglecting the development of those skills that are difficult to test using these formats.

-- Moving away from one-shot, time-bound examinations to assessments based on the results of work done over extended periods of time is likely to provide more authentic evidence of a student's capacities, but raises problems of grading standards, fairness, and comparability that are very difficult to solve.

Uniformity versus optionality represents one of the most critical sets of trade-offs in the realm of examinations. Some nations, for example, Japan, China, the Soviet Union, and the United States, in the SAT and ACT, offer limited options in what is essentially the same examination for all candidates, whereas other countries, particularly England/Wales, France, Germany, and Sweden, offer candidates a large measure of choice among subjects, syllabi, kinds or degrees of specialization, and levels of difficulty.

Uniform national examinations facilitate comparability and evenhandedness of treatment for different groups. But uniformity exacts its price: opportunities to adjust the examination to recognize the different needs of regions or groups at different stages of school development are inevitably reduced; regional and local interests may feel slighted; and the center's purposes are likely to be served at the expense of the periphery's. These are the educational and social costs of a single examination, of special concern in countries covering a varied territory or comprising a large and heterogeneous population. By the same token, a large measure of optionality brings the clear benefit of adapting the examination to the subject preferences and aptitudes of individual candidates. Examinations can in this way also be tailored to different educational and occupational prospects and to the broad requirements of society and the economy.

But optionality inevitably weakens the prospects for a national curriculum and the sense of a national culture. It raises problems of comparability and may even provoke concerns about equity. A credential based on a familiar standard set of compulsory subjects is easy for employers and admissions officers to interpret; they can be puzzled indeed by the complex regulations and weighting schemes used to equate the essentially nonequatable assortments of examination subjects offered by candidates.

The retreat from uniformity to diversity is well illustrated by changes in examination policy in France and Germany. The changes made in the baccalauréat represent bold and on the whole successful moves, but they have been achieved at some price. The most obvious has been a loss of comparability of candidates, who take widely different assortments of subjects and different tests in nominally the same subject, with different weights given to the results, depending on the particular option. Only in name, certainly not in practice, is there a single national end-of-secondary-school examination administered to all candidates in France. Instead, a strongly demarcated hierarchy of prestige has emerged, with the mathematical options at the head and the vocational options at the tail. Furthermore, devolution of some limited responsibilities for selecting examination questions and awarding marks has led to the view that standards are not the same across all académies. An examination system that began with a strong commitment to strict uniformity and comparability across the entire country, including even overseas departments and dependencies, has yielded to the need to accommodate a wider spectrum of the student population and broadened definitions of what should be learned at school and assessed.

In addition, despite the persistence of a common core of subjects, the differentiation of the baccalauréat has provoked fear that France is dissipating its intellectual patrimoine. Whether this is a benefit or a loss is, we suppose, a matter of taste, but taken simply as a matter of fact rather than of values, French culture générale has become a little less générale.

As with the baccalauréat, the Abitur has been significantly altered to cope with the increased numbers and heterogeneity of candidates. It has changed less radically than the baccalauréat, however, perhaps because the need in Germany to secure some measure of agreement among eleven quite differently oriented Länder sets limits on the possibilities of change. In addition, whereas the baccalauréat is marked by a generally high degree of central direction, a determining characteristic of the Abitur is its school-based control. Especially in Germany, the combination of local control and written and oral examinations raises questions about the extent to which grading standards are kept consistent even within a given Land, let alone across the eleven Länder. Because Germany makes less effort compared even with France to ensure such standardization, an important element of chance and arbitrariness has developed.

Even more than in most centralized countries, the basic stance of the examination systems in both Japan and China is one of strict central control and uniformity of content, characteristics that build upon traditional state practices. Few options are extended to Japanese candidates. They may choose mathematics tests of different levels of difficulty, additional tests in social studies, and a particular foreign language. In China, although the detailed administration of the examinations and the final selection process are delegated to the provincial authorities, they are required to operate within narrowly drawn guidelines. The intent is to ensure a strictly meritocratic selection of candidates for entry into higher education. Although criteria other than examination scores are taken into account, especially evidence of political reliability, work experience, and good health, performance on the National Unified College Entrance Examination remains the major criterion for selection. Considering that the number of candidates is exceptionally large -- in 1988, 2.7 million prepared for the national college admission tests, of whom only a fifth were accepted for study -- the Chinese system has produced a truly impressive degree of comparability among candidates. But this insistence on a uniform examination has severely limited the range of studies in the secondary schools, as well as the freedom of Chinese teachers to experiment with new materials and methods. To that extent, educational development has been held back in China.

The Soviet Union provides a sharp contrast to China in practice, though not in formal terms. On paper, the center, Moscow, is dominant and only token variations from standard uniform syllabi and examinations are permitted; in practice the republics and the localities enjoy considerable latitude. Each of the fifteen republics has responsibility for setting the content and standards of the examinations for the attestat zrelosti. Schools work within the republic guidelines, but in turn enjoy a good deal of local discretion. The teachers who prepare the students dominate the process of setting the questions and evaluating the responses. Even though many teachers are members of the Communist party, this degree of autonomy presents a curious disjuncture with the tight control the party and the central government exert over school curricula and textbooks. Paradoxically, in a society and a school system that are in most respects characterized by substantial central direction, the school-completion examination is not. Thus, the Soviet Union has settled for a curious compromise between the rhetoric of centralized planning and the practice of local discretion.

Once again, the trade-off for such local discretion is a substantial loss of comparability of marks, and this led the universities and higher technical institutes (the VUZy) to insist on applicants sitting for special, institutionally set and graded entrance examinations, a second stage very much along Japanese lines. In the final years of Soviet power, the universities have received formal encouragement to establish their own entrance criteria and examinations.⁷ Diversity and optionality achieved in this manner entail significant costs and aberrations. The location of the examinations can impose substantial travel costs on students. There is virtually no coordination among the VUZy concerning the dates on which they will hold their examinations; examination syllabi are idiosyncratic; and grading formulas, cutoff points, and so forth are confidential. Admission into higher education has consequently become something of an exercise in game theory, requiring candidates to exercise decision-making skills in conditions of imperfect knowledge and uncertainty. The system appears to be lacking important elements of overall fairness and objectivity, to say nothing of the persistent reports of discrimination against particular ethnic and religious groups, influence-peddling, and corruption.

In Sweden the changes in the examination system since the mid-1970s have gone hand-in-hand with changes in the upper secondary school curriculum. These offered students a much wider choice of courses and devolved administrative authority over the schools from the center to the municipalities. Sweden discarded a fairly uniform national end-of-secondary-school examination, replacing it with a combination of marks gained during regular classroom and home work and in nationally set tests administered at intervals during the upper secondary school career. The goals were to reduce the strain on pupils, produce more valid and reliable predictors of university success, and correct socioeducational inequities in assessment. These were judged to be as important as insisting on a high degree of comparability via a uniform examination.

The new system by no means abandons attempts at comparability across candidates. Teachers keep detailed records of their students' achievements, and much time is given to moderating grading standards, so that variations are minimized within both schools and regions. All of this makes important demands on educational resources, but so far the Swedes have been willing and able to incur the necessary costs in order to try to reconcile the respective demands of individual diversity and national comparability.

There is some evidence that the system may not be working to provide a wholly satisfactory degree of comparability. Beginning in September 1991, high school seniors could choose to take the Swedish Scholastic Aptitude Test (SSAT), a day-long assessment of verbal, quantitative, analytic and problem-solving abilities, and offer their scores in place of the usual high school records when applying for admission to higher education.⁸ Previously this route had been open to mature-age students only.

If the trend in Sweden has been to devolve authority from the center to the local government units, events in England/Wales since 1987 represent an abrupt acceleration of what had been a slow transfer of authority over the schools from local to central government. In the interest of establishing uniform national curricular standards, voluntarism and localism have been forced to give way.

Since the demise of the General School Certificate and its associated London University Matriculation regulations, it has not been necessary for a given student to take any particular subject or to follow any particular syllabus within that subject, except insofar as he or she wanted to take an examination or the school demanded it. The former GCE 0 level and CSE offered, and the present GCSE and A level examinations all continue to offer, a truly startling array of options. The government decided that this laissez-faire policy had produced such incoherence in school curricula and such indefiniteness in expectations of what the schools should be doing that it had to go. Hence the quite revolutionary decision to introduce regular national assessment of each student's knowledge and skills, which together with the new GCSE examinations is intended to help implement a national core curriculum.

So far, the government has declined to interfere with the GCE A-level examination system, which continues to offer a rich variety of subjects and examination syllabi to schools and candidates. However, problems occasioned by lack of comparability have been reduced by cooperation among the examining boards to identify core topics in syllabi in such basic subjects as mathematics and literature. In addition, a much greater effort has been made in England/Wales than in either France or Germany, for example, to ensure standardization of grading criteria, both within and between examining boards. For this reason, some of the more serious doubts about the fairness and comparability of marking that are voiced in the latter countries have been restrained in England/Wales, though by no means absent.

In the United States the benefits of local control and optionality have been secured, but only at the price of absence of widely recognized achievement standards. State and local control of the schools in the United States has produced a bewildering array of instructional practices, far exceeding even the variety formerly available in England/Wales. In addition, very few of the fifty states offer an external secondary-school-leaving examination or university selection/entrance examination. A high school graduate's academic achievement is thus reported on the basis of schoolbased, teacher-awarded course grades, with little or no effort made, as in Sweden, to ensure comparability of grading standards among schools or even in many instances within a school.

To compensate for the resulting extreme lack of comparability, the "national" examinations of the Educational Testing Service and the American College Testing Program have deliberately provided few options to the individual candidate. In this way a limited amount of comparable information about college applicants is provided. On the other hand, employers hiring the non-college-bound usually have only the applicant's high school diploma as evidence of academic achievement. Sometimes, but by no means commonly, an employer will have a copy of the applicant's high school transcript, but even then may be hard-pressed to divine what a particular grade means with respect to academic ability. Most employers have responded to this uncertainty by ignoring transcripts altogether. So far, the United States has steered away from instituting an officially sanctioned, uniform system of examinations, though recently, as a result of rising concern about achievement standards, the possibility of some form of national school-leaving examination has come under consideration.⁹

The Persistent Dilemmas of Examination Policy

While the example offered by other countries gives strong support to the notion that a national, external, end-of-secondary-school examination is useful, and even necessary, its introduction into the United States should not be regarded as a panacea for the country's educational ills. In addition, although there are clear advantages accruing to Germany from its Abitur, to France from its baccalauréat, and to Japan from its JFSAT, the critical policy question when trying to adapt foreign institutions to domestic purposes is: How can the United States secure these advantages, while avoiding, or at least minimizing, the disadvantages that may accompany them?

Entirely satisfactory solutions for such policy problems are rare indeed. Moreover, as the circumstances of an examination system change, so a new set of compromises among the competing goals is likely to be required. For these reasons, the choices that must be made are best regarded as persistent dilemmas that examination authorities have to learn to live with, leaning sometimes to one side, sometimes to the other. Among such dilemmas are the following:

Dilemma 1. How to retain the function of the examination as a stabilizing element in the educational and social systems, while:

1. providing flexibility and opportunities for change in content and format of the examination;
   
   
2. introducing new, nonexamination criteria of assessment, for example, course work and portfolios;
   
   
3. expanding the number of candidates.
   
   

Dilemma 2. How to promote examination results that are comparable and understandable, while:

1. introducing new, more sophisticated ways of reporting results;
   
   
2. introducing nonexamination criteria of assessment;
   
   
3. introducing new subjects and differentiated levels of subjects.
   
   

Dilemma 3. How to maintain the value of the credential earned by examination success, while:

1. increasing the number of successful candidates;
   
   
2. introducing nonexamination criteria of assessment;
   
   
3. reducing the burdens and pressure on students;
   
   
4. controlling examination costs.
   
   

Dilemma 4. How to ensure a sufficient degree of regional authority and school or individual autonomy, while:

1. implementing national standards of content and quality;
   
   
2. recognizing the claims of internationalism;
   
   
3. using examination results to monitor school system performance.
   
   

Dilemma 5. How to use the examination to select for subsequent education, training, and jobs, while:

1. also certifying satisfactory completion of secondary education;
   
   
2. not discouraging too many candidates, that is, picking winners without creating too many losers;
   
   
3. providing flexibility of examination content, in particular, access to new subjects.
   
   

Dilemma 6. How to incorporate the new information technologies in the examination system and secure the promised benefits of greater efficiency and cost savings, while safeguarding teaching and learning from undesirable side effects.

Dilemma 7. How to maintain and increase the professional autonomy of teachers, while:

1. using examination results to monitor school system performance;
   
   
2. strengthening national curricula and national standards.
   
   

Dilemma 8. How to raise standards of performance, while:

1. increasing the number of successful candidates;
   
   
2. using nonexamination criteria of assessment;
   
   
3. reducing the burden and pressure on students;
   
   
4. reducing examination costs;
   
   
5. not discouraging too many candidates.
   
   

Each of these dilemmas represents a potential problem associated with use of the examination system to control either individual destinies or what occurs in schools. In consequence, as the United States quite properly considers reform of the testing and examination system as a way to lever education to a higher level of performance, policymakers need to be constantly mindful of the likelihood of unintended, undesirable consequences of their decisions. Other countries have often "been there before, " and the consequences of going in this or that direction are available for inspection.¹⁰ If for no other reason, the record of other countries' examination experience repays attention.

NOTES

1. All students in upper secondary school continue the common curriculum-Swedish language and literature, English, history, mathematics, and physical education. The particulars of the curriculum will differ, however, depending on the course chosen. The longer programs allow for more advanced work and some specialization in the third and fourth years. [BACK]

2. As the decentralization of the Swedish school system continues, the authority of municipalities has increased while that of the board has been sharply reduced, though it retains an important coordination role. New arrangements for implementation of the new policy are currently under way. [BACK]

3. John Greenlees, "Pupils Go Crackers over Study Aid," Times Educational Supplement (June 17, 1988): 15. [BACK]

4. Thomas Rohlen notes that suicide rates are not higher for Japanese high school seniors facing examinations than for their counterparts in other nations, but that examination pressure as a cause of suicide is probably higher. Some of the yobiko (cram schools for repeaters) have their own clinics. Thomas P. Rohlen, Japan's High Schools (Berkeley: University of Berkeley Press, 1983): 327-334. [BACK]

5. U.S. Department of Education, Japanese Education Today (Washington, D.C.: U.S. Government Printing Office, 1987). [BACK]

6. Some states do limit the otherwise unfettered discretion of the testing organizations. For example, New York State requires ETS to provide each SAT candidate with an item-byitem printout of the grading of his or her responses. [BACK]

7. Bulletin of the State Committee of the USSR on Public Education. Professional education series. 1/1991; Poisk 6/91. [BACK]

8. Kenny Bränberg et al., "The Influence of Sex, Education and Age on Test Scores on the Swedish Scholastic Aptitude Test," Scandinavian Journal of Educational Research 34, no. 3 (1990): 189. [BACK]

9. See, for example, the work of the New Standards Project, developed jointly by the National Center on Education and the Economy, Washington D.C., and the University of Pittsburgh's Learning Research and Development Center. In addition, the National Council on Education Standards and Testing, created by the U.S. Congress in early 1991, is likely to have a major impact on the debate over standards of achievement and modes of assessment in the United States. The preferred solution seems at the moment to be the establishment of a set of nationally recognized standards, identifying a core of subjects and their content, which state educational authorities could adopt, while adding whatever they wish to the state's own examination to meet local preferences. [BACK]

10. Albert Shanker, president of the American Federation of Teachers, calls attention to the experience of other countries, as follows: "People who predict disaster [if the United States were to adopt national standards and assessments] should look at how systems work in other industrialized countries. Some of these systems are better than others, but none is a disaster, and there's no reason to believe we'd have one here. In fact, there's every reason to believe we could make such a system work." "National Standards and Exams," The New York Times (March 1, 1992): 4.7. [BACK]