Quality Management and Customer Focus

GAO has taken an example from the private sector and TQM to "focus on the customer," which GAO generally defines as Congress. Interviews for this study and GAO's own 1992 survey of congressional staffs indicate some discomfort in Congress with GAO's "customer focus" language, and some suspicions that GAO is becoming too willing to tailor its work to the interests of requesters and satisfy the committee and subcommittee staffs who are the major requesters and users of GAO work.

1.

2.

3.

Approaching Congress as "customer" has several built-in problems:

Congress consists of hundreds of individuals with diverse and often conflicting interests. Therefore, work that might please one committee or member is unlikely to please all the others.

Unlike the typical private sector "customer," congressional requesters do not pay or even perceive a cost for GAO work, and most are not aware of the costs of the work GAO does for them.

GAO's credibility depends on its reputation for impartiality, which could be damaged by an overemphasis on pleasing any individual requester.

The panel recommends that GAO clarify what it means to be "responsive" to Congress while preserving objectivity and professional standards for audit and evaluation.

GAO WORK PROCESSES

Internal Design and Review of GAO Work

GAO work processes tend to proceed in uniform, hierarchical patterns with inadequate definition at the outset of the objectives, methods, and type of work, and cumbersome review processes at the end.

-

GAO work products include blue cover reports -- ranging from substantial "chapter reports" to shorter "letter reports," written briefings, fact sheets, and correspondence issued in blue covers as well as oral briefings, shorter and less formal correspondence, and historical or other factual series of studies or guidance. Despite the diversity of its tasks, GAO tends to work in uniform patterns that produce reports in similar formats. In the panel's view, controversies over GAO studies generally reflect misunderstandings of the basic research objectives and different views on sources and methods. Some congressional staffs do not feel that GAO did the work they requested. In some cases, GAO's methods do not seem suitable to its conclusions.

The panel has several major recommendations on GAO's work processes: 1) GAO should develop clear "terms of reference" (TORs) for each job before it is started, outlining the objectives, general methods and skills to be applied, timing and estimated cost; 2) at the beginning of each job, GAO should form work teams that represent the full range of skills and experience needed across all organizational units and levels at GAO, working collegially throughout the project; and 3) GAO should replace sequential, hierarchical reviews of reports with concurrent, interactive reviews, as well as frequent consultation and participation of supervisors and managers throughout the planning, research, and drafting process.

The panel sees developing terms of reference as the most important change from the way GAO now works. GAO should negotiate these terms with congressional requesters of a project, if any, and share them with the agency subject to study. Terms of reference for studies requested by committees should be shared with all committee members, majority and minority. GAO should retain flexibility to design the specific methodology and develop the research, but developments during the course of study that require major adjustments in the terms of reference should be discussed with requesters and the agencies in question.

Agency Comments

A major cause of concern about the objectivity of GAO's work is the way in which the results are shared with agencies and interested members of Congress. Contrary to GAO's published standards for government auditors (the "yellow book") -- which call for comments from the agency subject to audit and evaluation -- congressional requesters have increasingly instructed GAO not to get agency comments on its draft reports and sometimes not even to brief agencies orally about the results of a study. In some cases, this sets up "ambush hearings" for which neither executive agencies nor other parties have substantial notice.

Executive comments on GAO reports can serve several purposes:

Reduce the potential for factual errors or misunderstandings;

Reveal language that might raise sensitivities that GAO did not realize; and

Contribute to greater cooperation and receptivity to the GAO reports on the part of the agencies being studied, thus significantly increasing the possibilities of effective response in the organization.

The panel recommends that GAO consistently seek written comments from subject agencies on all reports that audit or evaluate executive activities, with a 30-day limit for responses, and potential extension to a total of 60 days in special circumstances.

Performance Indicators

GAO emphasizes process not only in its recommendations but also in its tracking systems and its indicators of its own performance and accomplishments. The panel recommends that GAO revise its performance indicators, to reduce emphasis on process and inputs and focus on measures of the outcomes and impacts of its work. GAO's performance indicators should include not only estimates of net savings but also associated improvements in the efficiency and effectiveness of programs. GAO should also increase its internal resources and capacity to analyze the costs and budget impacts of its recommendations, as well as to assess agency cost accounting and performance measurement systems.

External Peer Review

GAO should establish and budget for external peer review of its completed work on a continuing basis. The volume and diversity of GAO work are not compatible with formal review before jobs start or as part of the report review process. GAO, however, should work with professional associations and other experts to organize a process for having specialized panels of objective peers from outside GAO assess and report on samples of completed GAO work, covering several selected categories each year.

GAO FUNCTIONS AND WORK PRODUCTS

The nature of GAO work has evolved over the past 30 years, resulting in less emphasis on audits, particularly financial audits, and increased resources directed to program- and policyrelated work. The quality, usefulness, and importance of GAO's work products are uneven. The panel also found lack of clarity in the objectives of individual GAO reports and the standards or criteria for making findings and recommendations. Overemphasis on procedural controls still prevails in many studies.

Categories of GAO Work

GAO currently undertakes six principal categories of work: 1) financial audits; 2) economy and efficiency audits (including non-financial compliance audits); 3) program evaluation; 4) policy analysis and policy development; 5) management studies; and 6) special investigations. These categories are not sharply defined and elements of several categories are frequently mixed in a single GAO study. Partly as a consequence, a job may lack clear objectives at the outset, in conflict with the "yellow book" standards that GAO prepares as a guide for federal, state and local government auditors.

The panel recommends that GAO develop clearer definitions, objectives, and standards for all the categories of work it performs; strengthen results-oriented assessment of government activities (particularly in the category of program evaluation); continue to strengthen its capacity to audit and advise agencies on systems and standards for financial and information management; include substantially greater cost analysis in its work in audit, evaluation, and policy analysis; base its policy analysis and policy development on fact-based audit and evaluation; and restrict recommendations for added oversight and control that are not based on a demonstration of expected net benefits.

GAO's special investigations work (which seeks to identify criminal behavior in government) should be kept separate and distinct from audit, evaluation, and management analysis done by issue area staffs. The panel recommends that congressional oversight committees work with GAO to develop clear standards for GAO investigative work and guidelines for referring requests for investigations to federal law enforcement agencies.

General Management Reviews and Related Work

GAO is at a crossroads in its management work. It has developed a cooperative, consultative approach to conducting general management reviews and providing technical assistance to agency leadership on systems improvements, which many agencies have found constructive. The panel finds inconsistencies between the attitudes and approaches involved in process-oriented, adversarial audits and the skills and relationships needed for GAO to play a potentially growing role in providing general management analysis and technical assistance to executive agencies, with the support of Congress.

The research done for this study shows that adversarial relationships between auditor/evaluator and agency or program management seldom lead to productive general management improvements. Aggressive oversight can identify, expose, and punish mismanagement that violates law -- a legitimate function, within limits -- but it historically does not improve management in the long term. Providing effective review and technical assistance to agency management usually involves a cooperative relationship with executives responsible for management change.

The panel recommends that GAO launch a major internal training process to convey the lessons learned from its general management studies throughout its own issue area staffs to bring those lessons to bear on all audit and evaluation activities. It should continue this area of work and build the lessons into the whole range of GAO work. To fulfill its statutory role under the Government Performance and Results Act of 1993, GAO should work cooperatively with OMB, executive agencies, and oversight committees, not to audit but to record the experiences in pilot projects under the act, in a non-adversarial context, in order to make a methodologically sound evaluation of the implementation of the law as required in 1997.

CONGRESS AND GAO

While the panel found no evidence of deliberate partisan bias in GAO work, Congress has increasingly embroiled GAO in political and policy controversy, threatening its impartial role and institutional standing. Increasingly adversarial relationships between congressional and executive entities in the last 20 years and intra-Congress rivalries make it hard for GAO to produce objective work – and to be perceived as doing so. Any use of GAO that erodes perceptions of its fairness and objectivity reduces its usefulness.

The way Congress uses GAO largely determines GAO's continued value as an institution. Congressional views of GAO's mission vary widely: some see it as an independent audit agency; others treat it as a servant of Congress and an infinitely flexible addition to congressional staff resources. The demand for GAO work always exceeds the capacity and resources of GAO, particularly if GAO is to produce timely, high quality results. The result is a considerable variation in the time GAO can devote to, and the quality of work GAO can produce for, different jobs.

Congressional Requests

Requests from congressional committees and subcommittees and individual members for GAO jobs have increased dramatically in number and scope, while GAO budgeted staff remained stable for a decade and declined 10 percent from 1992 to 1994. In fiscal year 1993, 80 percent of GAO staffyears of work were devoted to fulfilling congressional requests or mandates and the other 20 percent was for self-initiated work, which falls under the name "basic legislative responsibilities" at GAO. This represents a sizable decline in self-initiated work from a decade earlier, when self-initiated work took 53 percent of GAO staff time.

The degree of actual change is not clear, however, because a large proportion of congressional requests involve continuing streams of work in the GAO issue areas or reflect GAO priorities raised in its internal strategic planning process. Many committee staff members are satisfied with the way GAO projects are developed, although some are uneasy with having GAO spend so much of its time fulfilling congressional requests and others would prefer that GAO not shape so much of its own work.

As much as possible, congressional requesters of GAO work should create a non-partisan environment in which GAO can function. They should not assign to GAO report topics and research questions that would place GAO in the midst of value-based debates and political controversies. Also, congressional requesters should expand their focus from immediate shortcomings of programs and agencies (the "gotcha" approach) to efforts to develop systems that will generate better management, timely and accurate information, and comprehensive program evaluation on a sustainable basis.