cent deflection. Specifications are being upgraded to overcome this deficiency.

THE AIR FORCE and the tire industry, working side by side, have made substantial progress in improving the quality of aircraft tires and in the management of this high-dollar-investment program. Many additional improvements are foreseen, and, particularly on the technical side of the program, they are essential to keep pace with technological advances in the aircraft programs to be supported. Most of all,

a concentrated research and development effort is needed to produce the breakthrough that will create a tire material which provides a completely safe, longer-wear tire for use on both the current and future aircraft fleet. As Major General Don Coupland, OOAMA Commander, recently put it to an Air Force/industry meeting: "While I'm not in the development business, I would like to urge the entire industry to go 'way out.' My idea of the tire we need is one that is vastly different from the one we are all accustomed to seeing in the warehouse today.”

Ogden Air Materiel Area

OB-KNOWLEDGE tests have long been used

tory is documented either in the literature on testing or in that edited for the professional military person. Although tests of the aeromechanical skill of enlisted personnel have been used since the Twenties, aeronautical job-knowledge testing has received its chief impetus and development during the past fifteen years. Through research, experimentation, and validation these job-knowledge tests constitute a major factor in the evaluation of USAF enlisted personnel.

The first extensive use in the United States of examinations for men engaged in military aeronautical activities came after the passage of a bill by the Congress elevating the air arm of the Army

to a corps, the Army Air Corps, in 1926. At this particular time many of the better qualified men were leaving the Army Air Service. The Congress therefore allowed the Army Air Corps to establish procedures for identifying preeminently qualified personnel who would be eligible to receive supplementary pay for duties essential to flying. To that end, Air Mechanic (AM) ratings were established. Eligibility for these ratings became, in part, a matter of a candidate's being able to perform relatively well on a paper-and-pencil test covering his job. A practical grade, however, based upon performance, was added to written test scores to arrive at an overall evaluation for each examinee. By mid1928 some 900 men from a total of 9500 enlisted persons in the Army Air Corps held AM ratings. In

This article is based on research sponsored by the 6570th Personnel Research Laboratory, Aerospace Medical Division, under AFSC Project 7734, and in somewhat different form was delivered as a speech at a Technical Training Measurement Conference at Keesler AFB on 6-7 February 1963.

1930 the corps enlisted strength had climbed to 12,000 men, and 1500 of them were rated AM'S.

The examinations used in AM testing were assembled from the item files maintained at the service training centers, where the subject matter of part of the curriculum was the same as that of the required tests. Questions in the modern paperand-pencil format were chosen to sample information assumed to be important for those individuals who were, or were soon to be, responsible for maintaining aircraft, aircraft engines, aircraft electrical systems, aircraft hydraulic systems, or other such equipment or subassemblies. In each examination an attempt was made to duplicate the relative emphasis given the various topics in particular courses. In an engine examination, for example, 15 questions might touch on the topic of carburetion, 25 might touch on the topic of ignition, 10 on lubrication, and so on.

In the years immediately before World War II an extensive series of job-knowledge examinations was developed for the AM testing program, and most enlisted persons working in aircraft maintenance activities had opportunity to take one or more of these tests. However, with the expansion of the armed forces at the beginning of World War II and organization of the Army Air Forces, the AM testing program and AM ratings were abandoned. Instead, those holding these ratings were given advanced rank to compensate for the loss of AM pay. It was evident at the time that many of the AM duties required highly intelligent and able men who either were not to be found in the lower grades or should not have been kept there. Yet under the AM system it had been possible for a person in the grade of private to earn as much as a technical sergeant, excluding rations and quarters allowances. In 1941, the last year the system was used, 8500 men from a total enlisted strength of 134,000 held AM ratings.

During World War II no explicit attention was given to the further development and use of job-knowledge tests. But with the establishment of an independent Air Force and the initiation of a Career Guidance Program after the war, the making and giving of job-knowledge tests again became a large activity. Three separate programs were started, each with a slightly different orientation and with slightly different overall purposes.

Two of the programs were established within Air Force research organizations where job-knowledge tests were developed as necessary or preliminary parcels of the total research effort. The tests developed in one of the research units evolved into operational tools of continuing value. The third program, which extended in time beyond the experimental efforts of the research organizations, was an outgrowth of the career guidance concept and is still in existence. To enable the interested reader to learn more than the basic history and underlying educational philosophies of these first three Air Force test-development organizations, a few key references will present information concerning the characteristics of the examinations these organizations devised.

Technical Training Research Laboratory,
Chanute AFB (1949–52)

Procedural aspects of research on job proficiency conducted within the Human Resources Research Center (HRRC) of the Air Training Command required the development of measures on a number of hypothesized kinds of job knowledge such as "basic" knowledge, "functional" knowledge, or "troubleshooting" knowledge. Altogether, about 50 tests were constructed by or under the contract sponsorship of personnel of the Chanute laboratory. These tests covered a number of the more important technical specialties of the time. and were used extensively in the research program of the laboratory. All were paper-and-pencil tests, and most were multiple-choice in form. Demaree et al. present a good description of tests used in one study on the proficiency of Q-24 radar mechanics.1 The "Tab” test used in that study demonstrated an early, if not the first, application of "branch programing," which has come to be associated with the name of Norman Crowder in the field of automated instruction. Other members of the laboratory have reported research in which prominent use was made of the tests constructed at the laboratory. 2, 3, 4

The Chanute tests are now of historical interest, principally. Acceptable levels of accuracy (reliability) and relevance (validity) were attained, but the extended use of permanently assigned technical experts and the additional use

of highly qualified technical representatives from manufacturing concerns made the test-development operation an expensive one. So far as the writer is aware, no subsequent attempt has been made to duplicate the procedures or any of the special kinds of tests pioneered by the Chanute group.

Human Resources Research Laboratories,
Bolling AFB (1949–52)

As compared with the Chanute enterprise, a somewhat greater amount of attention was concurrently and later given to the matter of jobknowledge test construction by elements of the Human Resources Research Laboratories (HRRL), Headquarters Command, USAF. There, also, a critical need existed for good measurements of job knowledge that could be used in researching onthe-job proficiency. The initial emphasis was on oral examination techniques, in which specially trained evaluators were needed.5 Later, paper-andpencil tests, constructed with the help of experienced airmen, were substituted for the oral examinations.6

After HRRL became the Human Factors Operations Research Laboratories (HFORL) under the Air Research and Development Command in 1952, a larger program was started in which an attempt was made to provide examinations covering the jobs of airmen responsible for the maintenance of all major weapon systems then used by the Strategic Air Command and the Air Defense Command. When HFORL became a part of the Air Force Personnel and Training Research Center (AFPTRC) in 1954, additional tests on the pattern established by HRRL were prepared, published, and used by the Strategic Air Command and the Air Defense Command. A report by Morsh gives detailed information on one of the tests constructed under the sponsorship of HFORL.7 One of the last examinations constructed under the sponsorship of AFPTRC (which was dissolved in 1958) covered the knowledges necessary on the part of personnel responsible for the maintenance of B-52 aircraft. The development of this test has been described by Buckner.8

Unlike the Chanute tests, written examinations of the variety which derived from the early

research at HRRL are of more than historical significance. They continued in use after 1958 within the Strategic Air Command, and tests covering assigned equipment (or assigned jobs) seem to be a continuing requirement for the evaluation of command job training programs.

2200th Test Squadron, Mitchel AFB (1953–58)

In 1949 two new units were established, one in the Air Training Command at Scott AFB and the other in the Air Materiel Command at WrightPatterson AFB, for the purpose of constructing joboriented tests for the selection of Air Force warrant officers. The two units were expanded to include the development of other job-knowledge tests required in the new airman career program. and the total workload eventually became so great that a third unit was established in the Continental Air Command at Mitchel AFB. In 1953 the three units were consolidated and designated as the 2200th Test Squadron, which on 1 January 1958 was reorganized as the Airman Proficiency Test Branch, Personnel Laboratory, Lackland AFB, then a part of the Wright Air Development Center. As of May 1963, approximately 225 tests are being developed or revised annually, so that a goal of a revision every two years for each test may be achieved. At the present time tests are available to cover the jobs being performed by about 85 per cent of the airmen in the Air Force. Each year these tests are taken by approximately 200,000 airmen stationed throughout the world.

A good description of test development accomplished by the 2200th Test Squadron, including types of tests, construction procedures, administration and control, processing and analysis of the tests, has been provided by Gilhooly.9 Most of what Gilhooly said is still true of the program now in existence, though the job-knowledge tests and airman proficiency tests are now designated as specialty knowledge tests (SKT). The SKT program is designed to measure the job knowledge required for an individual to progress in his career field.

Useful information about the limitations and potential contribution of job-knowledge tests in Air Force settings is contained in the numerous reports that have been prepared as a part of the research and evaluation effort associated with the

on-going test development programs. (Sixty-six of the reports prepared at the 2200th Test Squadron, all unpublished, are in the library of the 6570th Personnel Research Laboratory, Lackland AFB, Texas.)

Where do we stand today in job-knowledge testing?

Because job-knowledge tests enter into important decisions, it is proper to inquire into their adequacy from the standpoint of their relevance to those decisions. Among people responsible for building tests, this matter is covered under the concept of "validity,” and most test psychologists would say that it is the most important characteristic of an examination. This accounts for the extreme care generally taken to ensure that a test really does cover the important aspects of a job and that the information called for is unquestionably needed, not merely something "nice to know." One aspect of that carefulness is the close attention given to the selection of the most knowledgeable of the available persons to serve as subjectmatter consultants. The judgments of these persons are crucial when it comes to the identification of the important things to cover in a test. Fortunately, the procedures used by the Air Force can be, and usually are, specified in much detail in manuals, regulations, and other technical publications. These materials are unquestionably the appropriate point of departure for test construction.

Test validity is essentially a matter of judgment, and many test psychologists who have been closely associated with job-knowledge testing programs in the Air Force have maintained that careful judgments obtained from subject-matter specialists concerning test content are more meaningful than judgments of another kind used in the typical "validation" study. In such a study, judgment or opinion of peers, supervisors, or others concerning the rating or ranking of a sample of airmen on some scale of “proficiency" is compared with their scores on a job-knowledge test. In the Air Force situations these relationships have been found to be rather low, generally. 10, 11, 12 McQuitty, Wrigley, and Gaier have demonstrated further that Air Force supervisors tend to describe trained mechanics (those selected on the basis of

varying proficiency) much more in terms of interest and motivation than in terms of the amount of job knowledge possessed.13 Some of the work of Humphreys and Schmid supports the notion that the best measure is a consensus obtained from several different ways of defining proficiency, specifically including the use of printed tests and the ratings of supervisors. 14 These findings have important implications for the use of tests. These implications will be mentioned later.

One traditional concern with respect to the matter of validity, i.e., whether or not job-knowledge tests measure what they are supposed to measure, has to do with the circumstance that they are written tests and may to a large extent be measuring ability to handle verbal material. Of course the ability to read and understand technical publications may be one valid aspect of "proficiency," but test psychologists have given some attention to the larger problem. One facet of the general matter was investigated by Polin, whe found a mean reading level (using the ForbesCottle Readability Formula) of "11th grade" for tests of four specialties in the intelligence field.15 Chajet, using the Flesch formula with 24 tests, found a mean reading level corresponding to the "12th grade."16 His syllabic count was equivalent to that of the second year of college, but the average sentence length corresponded to the eighth grade. These levels are not above the range of reading levels of airman groups now entering the Air Force.

Another point of view is that written jobknowledge tests tend to be "book tests,” which greatly favor those with academic interests and skills. Most of the research involving test scores and measures of formal education have shown the relationships between them to be low and nonsignificant. As a result, one writer observed that "irrelevant academic factors are not playing an important role in the measurement of job knowledge."17 More recently, however, with the use of information on the completion or noncompletion of certain high school courses rather than a gross measure such as "years of education," it appears that the relationships may be higher. These findings are being explored further, but a time may soon be coming when more information on the formal education of airmen will be used in selection