trust"-results in either rerunning experiments to establish the data, overdesign of components, faulty and wasted products, or abandonment of the effort.

The collection and evaluation of measurement data from the literature is so important that many of our large industrial concerns and our Government laboratories have for years maintained their own groups of highly skilled scientists and engineers to produce compilations of measurement data for the use of the laboratory or firms on research and development effort. In many cases these compilations find their way into the general public use, but in many other cases they remain of proprietary interest.

As you will hear in subsequent testimony, this is not a new subject. The establishment of critical tables is a long-standing national and indeed continuing international activity. I have some examples with me to show the committee the kind of thing that is put out. This is the last edition of the International Critical Tables, compiled in 1929. And this is as you see compilations of numbers. The original numbers appeared in scientific papers, because they were measured one at a time in the course of elaborate research.

To find the measurements in the original literature would take a lot of work on the part of an engineer or scientist, but by using these tables, he finds the numbers all compiled where he can have ready access to them.

Mr. DADDARIO. What does it cost to buy that book?

Dr. HORNIG. I am afraid I don't know the cost of the International Critical Tables volumes. I don't think it is in print any longer. I know libraries have it. I am not sure you can buy it. But perhaps Dr. Astin can answer that for you.

Mr. CONABLE. Is the information still accurate?

Dr. HORNIG. Oh, no. That is-some of it is, and some of it isn't. I can tell you from firsthand experience that this is always one of the problems. You look things up under International Critical Tables and the last reference is in the 1920's and yet almost all the researching in the world has been done since 1920.

Mr. DADDARIO. That gets back to the point "data that the user can trust."

Dr. HORNIG. Precisely.

Another example I have here. This is a compilation of data on the structure of crystals, which is published by the American Crystallographic Association.

Mr. CONABLE. The deterioration of this information as further research continues is likely to be a matter for the future as well as for the past, is it not?

Dr. HORNIG. Yes; this is why it is important to have a continuing activity, and the reason the Critical Tables ran out of gas is that the amount to be published and collected got to be too big for their staff and finances and they simply couldn't continue it any more.

And we are not the only ones. I have here a Russian compilation of thermodynamic data. They have a Russian publication program in the same area. It is not much use for people like me who don't read Russian. And this is a Department of Commerce publication on atomic transition probabilities. It is typical of the kind of thing that comes out of the Department of Commerce.

Now these things have to be updated constantly. It is important. Mr. DADDARIO. You refer to the Russian data book, Dr. Hornig. Do the Russians do a better job keeping the data current, considering the cost and the difficulties involved in compiling the data?

Dr. HORNIG. I am not acquainted in great detail with their effort, but my impression is "No." "I am not specifically aware, but through their large information dissemination system which you know aboutI know they have a very considerable effort. But I am afraid I have not investigated the exact relationships of their standard data program to our own.

I should mention that besides these examples of what you might call primary data compilations every scientist and engineer has on his bookshelf various secondary compilations, such as handbooks. For instance, I have a thing called the "Handbook of Chemistry and Physics." Now the publishers of this book don't go back to the primary sources, and do critical evaluations. They use these primary sources to put together these secondary data compilations which contain extracted data and which is presumed to be useful to a particular class of scientist or engineer. This is because if you try to get the whole data collection together you have many volumes, even when it is compiled.

Now you might ask the question: "With all the indicated effort, both within and outside of the Government, why do we need a Standard Reference Data Program under the management of the National Bureau of Standards?"

The primary reason is that no one group has undertaken to assess the total standard reference data needs of the national research and development community and to insure that these needs are being met. In many areas of science and technology, there is an adequate effort, and there is considerable suspicion that there is some duplicative effort, especially in the private sector.

In other areas of science and technology, we know that the compilation of standard reference data is woefully inadequate, and yet no group feels a responsibility or obligation to remedy this lack.

The second important reason is that the methods of making such information readily available have been overtaken by the advance of technology itself. Three factors have caused serious difficulties: the increased complexity of the field of science and technology; the increased volume of scientific and technological information generated; the increase in volume and complexity of the use patterns of such data. I should add one other factor, that is, just the need to update constantly whatever has been done. Yet these same factors result in an intensified need for such data.

We are dealing, then, not with a brand new subject or activity, but rather with an aggravated situation of fragmented, dispersed effort and antiquation of methods. The task is to organize disparate efforts and apply modern techniques to an existing activity which is not being done well. Two questions naturally arise: How should it be done, or by whom, and at what level?

The preparation of standard reference data is no simple job. An exhaustive search of the published literature must be made, and where the evaluators know of unpublished information they must endeavor

to obtain it also. The various numbers for a given measurement must be weighed against each other, and a judgment rendered as to the probable best number and the probable accuracy of that value. This effort calls for scientific and engineering competence of the highest degree. Yet, because this type of effort only rarely leads to the discovery of new knowledge, there is a shortage of able scientists and engineers working in this field.

The problems of making standard reference data readily available have been accentuated by the increased complexity of science and technology itself. For example, the cost of making a measurement of the effective cross-section of a nucleus is in the order of $100,000. Yet this information is essential for the efficient design of nuclear power

reactors.

Measurements of nuclear cross-sections are being made in atomic energy installations around the world. The volume of information is steadily increasing, and is being published in more different languages. Another complicating factor is the more diverse and more frequent use of standard reference data. The multidisciplinary teams working on many of our research and development projects today call for standard reference data from the most unexpected sources. Yet we have learned that this interdisciplinary approach is an exceedingly fruitful one.

This kind of a situation cries out for a closely coordinated program of governmental and private data collection, evaluation, compilation and dissemination efforts. This is what is being proposed as the Standard Reference Data System.

Because it was a matter which jointly concerned all of the Federal agencies involved in scientific and technical advance, the Federal Council for Science and Technology, as the subcommittee chairman noted, recommended in 1963 that the National Bureau of Standards be given the responsibility for a Government-wide program to develop critically evaluated or standard reference data in the physical sciences and technology and to insure its ready access for the benefit of the entire scientific and technical community in the United States.

This program was named the Standard Reference Data System, and was to be the primary means by which the Nation's scientists and engineers obtained the critically evaluated data which undergird modern science, technology, industry, and commerce. To meet the needs, new activities are required of the Department of Commerce and these are provided for in the Standard Reference Data Act.

The level of activity should be one which will result in a viable program; that is, one where the solutions or rate of improvement at least exceed the rate of growth of the problem or activity. The projections that the National Bureau of Standards has made, which you will hear about from them, to attain a fully operational level are consonant with my own experience in dealing with this kind of data and my feelings as to a viable level of effort.

It would take time to build to such a level due to organizing and staffing requirements and the limited number and availability of qualified people. The cost of such a program would be small compared to the research and development effort itself. In my judgment, it is still premature to decide that the Federal Government will operate at

least all of the information distribution part of the standard reference data system in all its aspects and forms. Perhaps the private publishing industry would be willing to undertake a large share of the information dissemination responsibility. Professional scientific and engineering societies may also wish to participate.

Mr. DADDARIO. Dr. Hornig, this question will come up from time to time and there will be different judgments. So we might as well ask you: You refer to the cost of such a program and the return as a result of it. Do you have any estimate as to this cost, and what its value would be in the final analysis?

Dr. HORNIG. Well, I think

Mr. DADDARIO. The reason I ask is that we have heard a very small figure as to the cost and tremendous figures as to what the return will be. So it would be helpful to us if we can get some judgment from you on it.

Dr. HORNIG. I think you will have to get this in detail from Dr. Hollomon and Dr. Astin, but at the moment we are spending on the order of a couple of million dollars a year in this area. Since we are talking about the improvement in the national research and development effort which in the public and private sectors together is in excess of $20 billion, I would think that a program of the order of $10 to $20 million at equilibrium would be a reasonable one, but this has to be based on detailed plans, so this is just to indicate a general order of magnitude.

The estimate of the return is a very, very hard one. What I know from personal experience, though, is that you simply can't carry on a research program without this data available and, therefore, our $20 billion program can't proceed efficiently without this other effort. I believe the estimates that have been made were ranging from 10 to 1, to 100 to 1 and more, for the return per dollar invested. The reason it is so hard to estimate return, of course, is knowing "what would have been" is a problem. If I have the wrong number and I design a bad machine, this costs us money. But since I never have the machine I would have designed to compare with the machine I do design, this gets to be a judgment problem-there is a bit of guessing involved. But I think that the judgment that the returns are very great, since one is talking about an investment number which is a fraction of 1 percent of the total R. & D. effort, is certain to be true.

Mr. DADDARIO. You refer to the staffing requirement and the limited number of qualified people available. Do you believe that if this program is emphasized it will attract such people, or will they need to be trained and recruited specifically for this purpose?

Dr. HORNIG. Well, in general, the people who do this job will have to be practicing scientists at the same time. I don't think you can simply train a class of person to look at other people's numbers and work over their probable reliability and accuracy unless he is actively working himself in these fields.

I believe that the people are available within the scientific and technical community, but I think that it will take time to get the people and assemble them-they will have to be taken from, as I said, the practicing scientific field, and it will take time to get them acquainted with the special jobs of critically evaluating and compiling the data.

Mr. DADDARIO. We are not talking about new people, but about an extension of the use of their knowledge in a more efficient way?

Dr. HORNIG. I am talking about new people in the Bureau of Standards. I mean they will have to add to their staff. But it is not a new class of persons.

Mr. VIVIAN. Will the chairman yield?

Mr. DADDARIO. Yes, Mr. Vivian.

Mr. VIVIAN. Dr. Hornig, it seems to me that adding additional people to the Bureau of Standards to do this work might be one of the staler ways of doing the job. A much better approach would be to add a limited number of personnel to the Bureau of Standards, and carry on most of the specialized research in other institutions, whether it be nonprofit institutions, such as Battelle, or a university such as Cal Tech, which has done excellent work. In this way, the quality of the work is likely to be better because you change the people as the scope of the job begins to be completed.

Dr. HORNIG. I think there is no question that you are right. As I said, this work has to be done by practicing scientists and engineers, and the present pattern in the Bureau of Standards is to do it by contract in universities and in the nonprofit organizations to get data when it is available and it has been generated in industry, and I think every one of these sources should be used to the maximum practicable

extent.

Mr. VIVIAN. So we would not interpret this $20 million as all being spent internally in the Bureau of Standards?

Dr. HORNIG. I don't think it should be internal in NBS. The NBS has, of course, a key role in the United States in that it has been historically the center for very accurate determinations, so it has a tradition in this area which is very important in bringing together all the various elements you mentioned.

Mr. DADDARIO. Will you proceed, please?

Dr. HORNIG. Now, I understand you will be hearing from Drs. Hollomon and Astin later, on the more detailed aspects of the plans and progress of the program at the National Bureau of Standards, so I would like to spend a few minutes on more related general problems. The problem of standard reference data is part of the general question of the network of information systems in the whole of science and technology. The overall information problem has been a subject of considerable importance in the Office of Science and Technology for several years. It has resulted from the three factors mentioned above; namely, the increasing quantities of information generated out of the increased research and development efforts, the growing numbers of scientists and engineers, and the increasingly complex use of information for purposes far beyond those in the mind of the original investigator. Added to these should be the factor of mounting costs of information generation and processing.

The assurance of continued progress is dependent upon our ability to keep informed of past progress in terms of both successful and unsuccessful results and the ability to transfer results from one area of effort to other areas of effort. So we build information systems. National information systems constitutes not a new activity so much as an effort to get better organized and do a more effective job of information transfer in both the public and private sectors.