phrase is repeated extensively in GAO documents and discussions. While nongovernmental groups and executive agencies are important users of, and are affected by, GAO products and processes, Congress is the source of:

basic legislative authorization under which GAO conducts self-initiated studies; and

appropriations that determine the resources available to GAO.

Congress is the primary user of GAO's work; GAO devotes a large share of its staff time and resources to responding to committees' and members' needs for information and analysis. Many members view GAO as an institution created to serve them and as an extension of available staff resources. GAO's capacity to gather facts, look independently at information, and evaluate programs -- and the trust and credibility arising from that independence -- have formed the basis of GAO's usefulness to Congress.

While Congress largely defines GAO's mission, the phrase "Congress as customer" does not clarify how GAO should carry out its mission. Congressional staff who call upon GAO for research and reports generally do not view themselves as "customers." In fact, according to interviews conducted for this study and documented by GAO from its 1992 congressional survey, the word "customer" makes many congressional staff members uncomfortable.

Normally, a customer is a source of demand for a product or service, with a price system to manage demand and monitor quality. Congress, however, consists of multiple potential sources of demand for GAO work none of whom pay directly for the work or even know its cost -- and they have nearly unlimited and highly diverse demands for information and analysis. And the interests of those requesters are often in conflict with each other.

The major drawback of the phrase "Congress as customer" is that it does not help to identify GAO's primary roles and substantive functions. The phrase has created a perception among many interviewed for this study -- even frequent and supportive requesters of GAO work -- that GAO is overly willing to please the requesters, raising questions as to whether GAO conducts studies with an independent eye or whether, by contrast, it shapes the research with an eye toward satisfying the requester's policy or partisan interests. The panel found no evidence that GAO has been steering its research toward satisfying particular policy or partisan interests. The panel recommends, however, that GAO take additional steps to respond to the concerns that have been raised. While GAO has extensive standards and guidelines for procedures and outputs, including standards on supporting Congress, the materials refer to timeliness and frequent communications but do not provide substantive guidance to GAO staff on what it means to be "responsive" to congressional requesters in terms of the substance of the work." That term can have multiple meanings, with serious implications for

Policies/Procedures Manual, November 1992, Chapter 3, "Supporting the Congress."

GAO's reputation for objectivity and credibility.

Chapter VI takes a closer look at GAO's relations with Congress, including congressional members' and staffs' views of GAO, relations between congressional committees and GAO, Congress' use of GAO detailees, and congressional requests for GAO's work and assistance.

RECOMMENDATION:

As part of its overall quality management initiative, GAO should clarify the meaning of, and criteria for, being "responsive" to Congress, in order to give practical guidance to staff and provide assurance to Congress that GAO work will be objective, impartial, and independently derived.

IV. GAO WORK PROCESSES

Despite the diversity of tasks GAO performs, it tends to work in uniform patterns that produce reports in similar formats, writing styles, and approaches to expressing conclusions. While GAO leadership has made a commitment to quality management -- including continuous improvement in work products, reduced rework, and employee involvement -- GAO's quality control process remains hierarchical, expensive, time consuming, homogenizing, and uneven in its effectiveness. GAO's approach to audit and evaluation work focuses on formal processes, administrative procedures, and bureaucratic management techniques. The following sections describe how GAO generally conducts studies, as revealed in case studies and interviews made by the panel and project staff.

Internal Design and Review of GAO Work

GAO work originates in several ways, including congressional requests, statutory mandates, agency requests, and self-initiated proposals to carry out basic legislative responsibilities (BLRs). In fiscal year 1993, according to GAO's statistics, work requested by congressional committees or members or required in law took 80 percent of GAO staff-years of work; the remaining 20 percent of GAO staff-years went to self-initiated projects (BLRs). For the first quarter of fiscal year 1994, congressionally requested work dropped to 73 percent while the share accounted for by self-initiated work rose to 27 percent (see Table 4).

TABLE 4

PERCENT OF GAO STAFF-YEARS SPENT ON CONGRESSIONALLY-
REQUESTED AND SELF-INITIATED WORK

In recent years, the work GAO classifies as self-initiated has largely consisted of:

General management reviews and specific management reports directed at executive users;

The 1992 "high-risk series" -- 17 reports on areas identified as at high risk of waste, fraud, and abuse;"

Election-year transition reports, which summarize previous GAO work in major organizations and program areas; and

Special one-time studies.

After GAO issue area managers and staff and congressional requesters, if any, agree on a job request, the GAO staff estimates the budget expense and staff-years for the job and puts it on the jobstarts list, which the comptroller general and his high-level "job starts" group review. After GAO decides that it will undertake a particular study, GAO's internal work process generally fits the following pattern. The GAO Office of Congressional Relations sends any request letter to the appropriate issue area, whose director or assistant directors (or both) may already have discussed the item with the requester's staff. Issue area leaders discuss how to design the job and what staff to assign to it. Often, as noted in the previous chapter, GAO discusses with congressional staffs how to match the requester's objectives with GAO's priorities and resources.

48

The issue area director and assistant director(s) select an "evaluator in charge" (EIC) and identify needed participants from other GAO offices (e.g., economists, lawyers, auditors, accountants, investigators or other staff from other program-oriented issue areas), as well as the GAO field offices and staff members who will participate. Most GAO work teams come from within an issue area and associated core staff in regional offices. The EIC and assistant issue area director are responsible for assembling the complete job team, which may work out of Washington headquarters or out of a GAO field office. When field staff is involved in a project, headquarters-field interactions involve a considerable exchange of paper and travel. GAO is developing an enhanced computer network and improving electronic communications, partly to reduce that problem; however, GAO does not expect the agency-wide computer network to be operating fully until the end of 1995.

For blue-cover reports, GAO often spends 8 to 18 months on research, audit, data collection, and drafting the text, all of which the EIC supervises. GAO's internal master job reports track, month to month, time and staff-year targets. In the early phases of project planning and initial research, however, high-level GAO leadership and supervisors devote relatively little attention to the design and conduct of work in progress.

47GAO/HR-93-1 through GAO/HR-93-17.

48 Discussions between GAO and congressional requesters sometimes last for the life of the job, particularly when the research questions, objectives, and scope of work are not clearly articulated and mutually agreed upon. The panel's recommendation that GAO and requesters negotiate clear "terms of reference" for a project will alleviate much of that difficulty.

Once research has been completed and a first draft prepared, an iterative review and "rework" process begins. GAO's personnel widely view this process as overly time consuming and say it has inconsistent effects on final quality. The process often consumes three to six months, and sometimes takes a year or more on a complicated or sensitive study.

In the division that produces the draft, an independent reviewer (a GAO evaluator not on the job team) reads the report, and in the regional or issue area office doing the work, a "referencer" checks factual assertions against work papers. (Work papers include calculations, interview notes, data, documents and other sources. GAO files them and holds them closely.) Generally, GAO reports do not provide footnotes and lists of sources to the extent normally expected of similar work under contract or in an academic context. GAO's "referencing process" involves carefully checking every fact included in a draft report to ensure that the statements in the text accurately reflect what is in the background work papers, but does not usually go back to the source or provide extensive footnotes.

After the issue area director approves an initial draft, he or she may see it again four or five times, with comments or mark-ups coming back from the division's planning and reporting office, its technical group, and its economics unit, the Office of General Counsel, Office of the Chief Economist, and assistant comptroller general in charge of the division. One EIC said that, within GAO, he had to defend his work 27 times before GAO printed it.49

-

What does GAO's current review process accomplish? It definitely adds cost and time to GAO work. While individual projects differ substantially, the entire research, writing, and review process leading to a blue-cover "chapter" report costs on average $575,000 -- primarily for staff time according to GAO's internal estimates. The average cost for all types of GAO work products in fiscal year 1993 was $265,000. Large multi-year studies often cost several million dollars. The cases examined by the panel show that GAO's internal review process can account for one-fourth to onehalf of the total time required to complete a study. The review process puts some restraints on recommendations and adds qualifiers to findings. In and out of GAO, some participants believe that caution and a tendency to take out specifics in favor of generalized ("homogenized") statements dominate the review process. Nevertheless, reports do not escape controversy, nor necessarily should they. The panel has reviewed cases in which:

Good audit and research work was mildly toned down in the review process;

Studies survived that should have been terminated (or never started) due to the lack of solid evidence or sound methodology;

Work with a clear and strong empirical base and relevance to policy debates became the subject of criticism; and

Studies based on unbalanced sources received praise from congressional requesters and

"In this example, the study became controversial even though it was grounded solidly in fact, the controversy arose largely because the underlying program was strongly supported by some congressional members and sharply criticized by others.