STATEMENT BY LEO G. BYRNE, DIRECTOR OF RESEARCH, SCHOOL DEPARTMENT, HARPER & Row, PUBLISHERS

Mr. Chairman and distinguished members of the House Select Subcommittee on Education:

I am pleased to be able to submit to you some observations and recommendations relative to the National Institute of Education which is to be established under the draft legislative measures designated as H.R. 33, and H.R. 3606.

I have been watching the progress of the legislative hearings and the evolution of associated items since the spring of the current year when I became Director of Research of the School Department of Harper & Row, Publishers, Inc. Prior to assuming that responsibility I functioned for almost four years as Editor in Chief of the same publishing department. My involvement in school publishing extends back to 1950 and includes experience in writing, editing, and marketing. Each of these activities called for constant contact with classroom teachers, curriculum specialists, academic personnel, and school administrators.

That background, I believe, gives me a base of authority for submitting certain recommendations and raising particular questions which may have been overlooked among the items brought to your attention.

Because you have heard a great deal of testimony I will limit myself to two areas, specifically, that of current educational research and that of current educational programs with particular reference to the place of the modern textbook in education and the role of the National Institute in helping producers of educational materials compile the best possible material.

CURRENT EDUCATIONAL RESEARCH

Since much of the pre-planning that has been undertaken in relation to a National Institute of Education and much of the testimony you have already heard has been addressed to the topic of educational research I would like to insert into the record some cautionary notes and to frame some questions which I believe should be answered before a table of specifics is imprinted on the structure that is proposed either in the enabling legislation or in the blueprint that is attached to the resulting Institute by virtue of the testimony submitted to you thus far.

In the study entitled The National Institute of Education: Preliminary Plan for the Proposed Institute, prepared by the Rand Corporation for HEW, copy of which you graciously sent to me, Mr. Chairman, and in subsequent testimony before the Subcommittee much has been said about the relative lack of research in education. Statistical data on expenditures for educational research have been compared to research disbursements compiled for major industrial areas and federal activities outside the field of education. The discrepancy is alarming.

I submit, though, that the parallels indicated are something less than scientifically accurate. Within the vast structure of American education there are countless items that are in the category of basic research or that are research-oriented which are not budgeted or identified as research in any fiscal report. Think, for example, of the research efforts and expenditures involved in thousands of individual graduate studies which are not identified as research in the budget of the university, governmental agency, or in any personal financial report filed by the individual pursuing the study. What research-dollar factor is attached to the temporary loss of earnings experienced by the graduate student? Or to the man hours contributed to that student's efforts by graduate advisers and members of doctoral committees? How many school systems budget under research the salary of a classroom teacher assigned to a pilot study of new curriculum materials or new instructional techniques? In private industry, on the other hand, salaries of research personnel and all attendant expenses are correctly included and reported as research expenditure. These are but two instances, Mr. Chairman, of how lineitem budgeting, alone, might explain some of the disparity in the allocation or non-allocation of research funds.

Of and by itself, the foregoing analysis may be of little significance. I introduce the item as a cautionary note and I am forced to wonder how the figures introduced in the data were arrived at. I might add that I have heard the identical figures repeated in subsequent and non-related educational meetings and in no case has a verifiable source been cited. Too often, gentlemen, have we seen loose estimates transformed into hard-rock, uncontested facts.

THE ERIC SYSTEM

A direct examination of the status of current educational research might be more revealing and convincing. A convenient vehicle for such examination exists in the apparatus we know under the acronym of ERIC. Without going into the history of the ERIC system I simply note that there is a cataloging of a vast reservoir of existing research-one that, in the main I think it safe to saylies serenely untouched and unexplored. The complexity of the system itself is such that one now needs a corpus of tools to probe for the hidden treasure. None of these remarks is to be interpreted as derogatory of the ERIC system. The question is, rather, is that system being used? Is existing research known, utilized, tested?

A partial listing of the major tools, that is, the tools universal to the system as distinct from the additional tools unique to the individual clearinghouses within the system is useful in achieving a perspective of the research that does, in fact, exist.

The following list is taken from the ERIC publication How To Use Eric (HEW-1968, OE-12037-A)

CONTINUING PUBLICATIONS

Research in Education—a monthly abstract journal reporting research projects funded by USOE, and other research items judged to be of educational significance by the several clearinghouses.

Research in Education: Annual Inder.

Thesaurus of ERIC Descriptors—a structured compilation of approximately 3,200 (in 1968) terms used to index and enter documents into the ERIC system.

HISTORICAL COLLECTION

Office of Education Research Reports, 1956-1965-Research reports received before the publication of Research in Education . . . compiled in two volumes: Resumes and Indexes of reports by author, institution, subject, and report numbers.

Quantitatively, what are we talking about here? I have no official figures for the current volume of research items listed in the ERIC storehouse but the following indicators will serve as a gauge:

(a) from November 1966 thru September, 1968, 8675 titles were listed in the cumulative issues of Research in Education. (cf. How To Use ERIC, p. 10)

(b) the Thesaurus described above was revised and a second edition published in 1970. In the preface to the 1970 edition it is described as still being in a stage of "adolescence." (cf. Thesaurus of ERIC Descriptors p. 10). As of July, 1971 at least ninety new descriptors had been added by the twenty clearinghouses. (cf. ERIC/TM NEWS, Vol. 1, No. 1 ERIC Clearinghouse on Tests, Measurements, and Evaluations-ETC, Princeton) The same source states "unfortunately, because new terms are added each month the Thesaurus is out-of-date at the moment it is printed."

THE ERIC/CRIER CLEARINGHOUSE

A closer look at the complexity of the ERIC System can be gained by focusing on just one of the twenty clearinghouses. Since the area of reading is the universally recognized priority problem area, a scanning of that clearinghouse, ERIC/CRIER, is in order. In the interests of brevity I will simply list some of the special resources available from that source in the area of research on reading. Please note that here, again, the quantitative data is several years old. The source is the Portfolio of Information on Reading Available from ERIC, ERIC/CRIER, and IRA published by ERIC/CRIER in 1968.

BASIC REFERENCES ON READING

Published Research Literature in Reading, 1900-1949-presents 2,883 citations and annotations . . compiled on a yearly basis by the Reading Research Center of the University of Chicago.

Published Research Literature in Reading, 1950-1963-presents 1,913 citations and annotations prepared by the Reading Research Center of the

University of Chicago.

Published Research Literature in Reading, 1964-1966-presents 849 citations and annotations . . . prepared by the Reading Research Center of the University of Chicago.

USOE Sponsored Research in Reading-items already listed in RIE, etc. Recent Doctoral Dissertation Research in Reading-Lists dissertations completed in colleges and universities since 1960 in the areas of pre-school, elementary, secondary, college and adult reading. A comprehensive analytical abstract. reported for each dissertation. Three hundred seventynine theses are listed alphabetically by author.

.

.

International Reading Association Conference Proceedings Reports on Elcmentary Reading-Lists... 345 papers... (in 16 categories).

The list of research collections goes on for several pages, Mr. Chairman, and for the sake of completeness I list only the headings for general collections on a level with the heading above, BASIC REFERENCES, etc.

BROAD SUBJECT BIBLIOGRAPHIES; BIBLIOGRAPHIES RELATED TO ERIC SPECIAL COLLECTION PUBLICATIONS; SPECIAL BIBLIOGRAPHIES AND REVIEWS; INFORMATION ANALYSIS PRODUCTS

Qualitatively, Mr. Chairman, the ERIC System is a magnificent accomplishment. All those who have functioned in it and those who continue to function within it deserve the highest praise. I fully believe that it is a research apparatus that is unrivaled.

The point is that the treasury of research it catalogs is almost unbelievable in extent and beyond the assimilative capabilities of many research/lifetimes. This is not a unique, personal conclusion.

Private conversations with some of the principals of the ERIC system corroborate the judgment that the system is woefully underutilized. Some of these men and women are making concerted personal efforts to convince educators to dig into the system and start retrieving the ore-bearing lodes beneath the surface. By way of underscoring the recognized non-utilization of ERIC, I quote from the Preface to the Thesaurus written by Dr. Frederick Goodman of the Univer sity of Michigan :

"Thus, very few people have ever been in a position to use the depth of indexing that indexers have provided from the beginning. In this sense the relatively deep indexing has been 'wasted' to date, and, by the same reasoning, it might be argued that the relatively complex structuring of terms within the Thesaurus has been 'wasted' to date. As ERIC begins to expand its capacity for computer searching, as it is now doing, the deep indexing and the Thesaurus are likely to become much more important." (cf. Thesaurus of ERIC Descriptors, p. 10). And again,

"Very little is known about the people who have been using the ERIC Thesaurus, why and how they have been using it, and what they think of it.” (ibid.. p. 23).

Finally, gentlemen, please remember that ERIC does not claim to list all known research.

In summary, Mr. Chairman, it seems to me to be dangerously misleading to speak glibly about the paucity of educational research when we face the herculean task of utilizing the research we do have available.

It is not easy to assume the unpopular role of questioning the impressive array of testimony submitted to you thus far, Mr. Chairman, but someone must pose the question before we rush into yet another cycle of needless replication of existing data.

Nor, again, is this a singularly personal position. More than a few educators, at the university level and at the administrative level have privately expressed the fear that many more millions of dollars will be spent in a wave of duplicate research while practical problems continue to beg solution. At the conclusion of the hearings in Chicago, Mr. Chairman, one educator remarked, "Well, here we go again, some guys are going to be paid to discover the wheel."

I am convinced that such is not the intent of the legislative proposals nor the goal of your Select Subcommittee, Mr. Chairman, therefore I presume to urge upon you the following:

RECOMMENDATION

That the Select Subcommittee recommend that it is the explicit intent of the Congress that the National Institute of Education consider it mandatory as a first order of business to devise ways and means of searching out, checking, and refining existing research currently cataloged in the ERIC System.

CURRENT EDUCATIONAL PROGRAMS

Not totally distinct from the matter of current educational research is the question of current educational programs. When sweeping generalizations about the poverty or inefficiency of existing programs are removed from the arena of debate one is left with the simple proposition that there are many good and viable instructional programs in our schools today along with many poor and inefficient ones.

These programs are generated from a variety of sources. Some are developed by individual school systems, some have their origin in one or another curriculum research group, and still others, Mr. Chairman and distinguished members-and I am sure you will pardon the sarcasm-still others, including textbooks, are produced as commercial products by private firms in the education industry. But the great majority of modern programs are based in research regardless of

source.

It is to the latter category of programs-specifically, the modern textbook, that I would like to turn the attention of the Subcommittee at this time. For some time we, in the textbook industry, have let go unchallenged many statements that are derogatory or, at best, half truths. Usually these statements are based on misapprehension or ignorance of the modern textbook and most of us in the industry have adopted a "truth-will-out" stance. My sense of self-respect impels me to speak out at this time.

Let me state very clearly, however, that despite a commonality of interests the textbook industry is composed of people who prefer to speak for themselves. In no way do I claim to speak for anybody but myself though I am confident that many of my colleagues would echo what I have to say.

According to the latest figures available the elementary and secondary schools of the United States spent nearly one-half billion dollars for textbooks in 1970. Absolutely considered, even in this inflated economy, that represents an awesome sum of money. In perspective of total expenditures, however, that sum represents but a minute percentage of the secondary/elementary dollar. Either way one chooses to regard the matter the role of the textbook and other instructional tools should be a proper object of study for a National Institute of Education. Yet, in the Preliminary Plan referred to earlier, the textbook is barely mentioned. If the textbook is even half as bad as its most vociferous critics claim it is, then it is an obstacle to good education. On the other hand, if it is only half as good as its most ardent proponents claim, then it is still a valuable and indispensable tool that should be strengthened and perfected.

In passing, Mr. Chairman, I would like to note an incident that is still very fresh in my mind. Not long ago I participated in the deliberations of one of the regional seminars on reading held under the auspices of the Office of Education. During the course of the proceedings I was involved in a small group discussion of teacher-college faculty members. At one point a member of the group began an attack on textbooks that made me wonder when he had last seriously examined a modern textbook. I thereupon asked the group as a whole if any one of them had, within the last few years, really looked at a modern reading program. All of them were honest enough to admit that they had not done so.

I mention this incident because I am personally convinced that much of the criticism levelled against textbooks is in the 'beating-a-dead-horse' or 'strawman' category. Textbooks, in these situations, are spoken of as if they were produced and published "in vacuo," so to speak, and are contrasted to multimedia techniques, etc. I frankly do not know a major publisher who offers such restricted programs to the schools. Modern reading programs, for example, in the K through 6 range usually include several hundred single items and come complete with a full range of tactile, manipulative, audiovisual devices and aids. The textbooks of twenty or thirty years ago are just that-they are obsolete, they have passed away, just as the classroom of the 1930s and 1940s has passed away-unless local lethargy has allowed them to continue.

VERIFICATION AND REVISION

But you have heard a charge made before this Subcommittee that, along with other instructional materials, textbooks are still produced, sold, and used in the schools virtually without any effort to test the validity of the instructional content in them.

I am referring, specifically, to the statement of Mr. P. Kenneth Komoski before this Subcommittee on May 11, 1971. Mr. Komoski's statement covered many points and I agree with most of what he stated. But the overall tone of his statement, together with significant omissions of relevant facts, seems to have created the impression that textbooks are produced by whim and distributed to helpless school administrators with arrogant disregard for pedagogical validly. The fact that this impression was distilled and so reported in national news media makes it imperative that a balance be inserted in the record of these hearings.

Let me say a word to the point of the telephone survey Mr. Komoski refers to on page 23 of his statement. This survey did not include Harper & Row's School Department. I cannot fault Mr. Komoski or EPIE for that, however, for I realize that a telephone survey sampling cannot be considered a complete investigation. But there are several factors I would like to have included in such a survey.

In the first place, we definitely have had some experience in field testing of the type Mr. Komoski urges. A program under development incorporated indicated revisions based on learner verification procedures. But the cost was very high-not so much in the mechanics as in time. If the results in the sense of indicated revisions had been significant then we could have said the time delay was worthwhile in terms of what we learned. The changes were really insignificant and I think the explanation is twofold. The cumulative experience of our authors and editors produced a good program to begin with, and the cooperation of the schools left something to be desired. It is exceedingly difficult for most schools to properly monitor an experimental program. You may have noticed that Dr. Barak Rosenshine of the Bureau of Educational Research at the University of Illinois touched upon this problem in the recent Washington, D.C._conference on "How Teachers Make A Difference" (cf. Education Daily, 7/16/71, p. 4).

Secondly, the term "field-test" is elusive and means just about whatever the user wants it to mean. I noted above (3.4.2) that a modern reading program usually consists of several hundred single items, takes anywhere from three to eight years to produce, depending on the base from which the developer starts and demands an investment of many millions of dollars. How does one go about field-testing such a program? In fragmented units? The results would be little more than interesting-this would not give valid information on the total program. What then? After all the components are finished? If so, for how long should the test go on? For one year? What would that tell the educator or developer about the retention factor of what is learned? A really valid field test is one which would require several years and discount Hawthorne effect and other variables before anyone could say, on the basis of field testing that program X or program Y really works. It would require scientific follow-up through several grades to determine how well children retained learned skills. A consideration of the population mobility factor alone leads one to doubt the feasibility of reporting certifiable results in such a procedure. The problem of field-testing is massive but does this mean the situation is hopeless and completely unsolvable?

No, because a third factor has been overlooked. Major textbook programs. especially at the elementary school level, are seldom produced "de novo," without benefit of author/editor experience or company history. Most major textbook publishers have their material in a constant state of verification and revision, and when and if a company decides to venture into a "new" curriculum area it does not proceed blindly or haphazardly. Such a decision invariably involves securing of competent authors who have taught, tested and retested the materials they put into their programs and recruitment of editorial personnel wellgrounded and experienced in the "new" subject area.

Classroom teachers, indeed, are heavily involved in textbook preparation and revision, far more, I know, than they realize or believe. Their letters and suggestions are carefully noted (and balanced, incidentally, with diametrically opposed letters and suggestions). Their verbal statements to salesmen are important, too, even if this is a mechanism that does not have the trappings of