ing GAO's work processes more open, visible, and efficient. One category is the group of recommendations that would open up scrutiny of GAO work priorities to public and congressional discussion. Another group of recommendations would alter internal GAO work processes in ways that would shorten the time needed to complete work while simultaneously improving the quality of the product.

Before summarizing these recommendations, it is important to point out that some of the panel's recommendations on GAO priorities may actually require, in the short term, added expenditures for certain types of resources in the short term, for example, increased capability in computer hardware and software for analysis and communications. While GAO is making progress in these areas, it is not up to what would be called "state of the art." Another example is the panel's recommendation that GAO strengthen its skills in cost analysis and financial management systems. At present, GAO does not have all the capability needed for sophisticated cost/benefit analysis. In the long run, additions of these resources and capabilities will save money, but not in the short run.

4. In order to open up GAO's internal processes and contribute to its efficiency and effectiveness, the panel recommends that GAO establish clear "terms of reference" (TORs) for all GAO projects.

What do we mean by TORS? TORS for a GAO project would outline clearly, in writing, objectives and research questions; the scope of the work; general methods and sources to be applied; staff skills, costs, and time needed to complete the work. GAO staff, managers, and requesters, if any, would agree on the TORS before the job is approved and work begins.

TORS are not a new invention. Studies contracted by federal, State and local governments from the private and non-profit sectors have long required specific terms of reference in the bidding or procurement process. Business-to-business contracting for research and development and other consulting services also requires clear benchmarks, cost and schedule estimates. The TORS identify the product that the organization is providing at an agreed upon price.

The panel concluded that effective use of TORS would have three major impacts: (1) improving the efficiency and effectiveness of GAO operations; (2) linking its work agenda more effectively to congressional priorities; and (3) giving higher visibility to GAO undertakings.

On the first point, better definition of jobs and the teams assigned to them up front will improve quality, substantially reduce review and rework after drafts are prepared, decrease time and costs of jobs, and give job teams clear benchmarks to meet. They will also better define sources and methods, thereby avoiding later criticisms of weak or biased research.

On the second point-linking GAO's work agenda more effectively to congressional agendas-the use of TORS as recommended by the panel would produce substantial changes. To elaborate on one, each GAO job would have a cost tag attached before it was authorized. In the past, any congressional committee or member could ask for a GAO job (in effect generating demand for GAO services) without any consideration of cost. As any of us who has taken elementary economics knows, if a good or service is perceived as free, demand for it is nearly unlimited.

The panel considered the proposal frequently made that congressional requesters should be allocated vouchers to purchase GAO services. This proposal seemed to the panel to be likely to lead to complex, bureaucratic procedures and logrolling tradeoffs, dragging GAO into political processes in which it does not belong. The panel concluded that if job objectives and cost estimates for GAO jobs are spelled out in writing, shared with Majority and Minority Committee Members, and open to Committee discussion, demand will be appropriately limited without use of an elaborate voucher system.

The panel also considered and rejected the idea of a bipartisan congressional board to provide prior review for requests for GAO work. The volume of required work is simply too great for prior review by a special board. Members appointed to such a board would not have time to screen all GAO job starts in advance. In all likelihood, a congressional staff bureaucracy would perform this task and politicization of the process would be a danger. The panel concluded that the process of establishing TORS could provide incentives for internal review before work began and also provide a basis for external peer review after work is completed that would be more effective.

In conclusion, the Academy panel, which began from quite different perspectives on GAO, produced a consensus report approved by all the panel members. The panel concluded that GAO, an institution created almost 75 years ago, is essential to functioning democratic government. While the panel does not see a need for any legislative changes, it found opportunities and needs for improvements in processes and perspectives in both GAO and Congress. Now it is up to Congress to ensure that

such improvements are made and that GAO's capacity to perform its basic mission is preserved and even enhanced.

Mr. Chairman, this concludes my prepared statement. We would be pleased to respond to any questions.

ATTACHMENT I

EXECUTIVE SUMMARY

The Senate Committee on Governmental Affairs asked the National Academy of Public Administration to convene an expert panel to examine the roles, mission and operation of the U.S. General Accounting Office (GAO) in May 1993. This report presents the findings and recommendations of the panel, focusing on GAO's mission and roles, its relationships with Congress and with federal executive agencies, the quality of its work, and its work processes.

GAO has been a valuable part of the federal government for more than 70 years, providing auditing, research and evaluation to government generally and to Congress in particular, which could not be easily and readily replaced. GAO's most important assets are its objectivity and impartial fact-finding and analysis that have given its reports and testimony a special standing in discussions and debates on government performance.

In recent years, there has been considerable criticism about GAO's work, including concerns about its objectivity and impartiality. Interviews conducted for this study revealed these concerns even among congressional staffs who most strongly support GAO. While the panel found no evidence of deliberate partisan bias in GAO's work, the panel did find that GAO's credibility and authoritativeness will be eroded by its involvement in policy areas without a solid base in research and evaluation.

The panel's major conclusion is that GAO and Congress should give greater attention to the scope of work GAO does, how it uses its resources, the quality of its work products, the objectivity of its findings and conclusions, and the fairness of their presentation and release.

GAO's status as the government's central agency for audit and evaluation requires bipartisan trust, particularly in a time of partisan conflict. The panel concludes that two major approaches are necessary to serve that objective: (1) restraint on the part of GAO and congressional requesters of its work, to avoid reports that do not have a firm factual base-within GAO's competence; and (2) continuing congressional oversight of GAO, with increased sharing of information, including GAO's strategic plans, lists of jobs started, terms of reference for individual studies, periodic peer review reports, and improved GAO performance indicators, as well as agency comments routinely and consistently obtained on individual GAO reports.

GAO MISSION AND ROLES

GAO's Vision, Mission, and Guiding Principles

GAO's mission and roles are becoming broader and more diverse than appropriate for the government's central audit and evaluation agency, going beyond its core purpose, skills, and resources.

GAO's statutory authorization, dating from the 1921 legislation that established it as an agency independent of the executive branch, gives the comptroller general a broad mandate to make recommendations on the economy and efficiency of public expenditures, to prescribe systems and procedures for appropriation and accounting, and to undertake investigations and reports ordered by any congressional committee. This mission has been broadened over the years by additions to the original statute and specific provisions in laws requiring numerous periodic or one-time reports on specific programs and expanding GAO's general management-oriented responsibilities.

Internal GAO processes for defining mission and roles-most notably the processes of strategic planning and total quality management (TQM)-have defined objectives for GAO that are very ambitious and reach beyond GAO's core mission. As part of the TQM process, GAO has defined its organizational vision as follows: "We aspire to be the world's leading organization engaged in audit, evaluation, and public policy analysis." This formulation encourages GAO to become more involved in policy questions, in a way that may extend beyond the appropriate role of the government's central audit agency and beyond its resources. To be the world leader in audit, evaluation, and policy analysis is an unrealistic aim, in light of the diversity of management, program and policy issues and the degree of specialization required to perform sound policy analyses and policy development across that full range.

GAO has become increasingly involved in policy analysis and policy development. Elected officials have the responsibility to set public policies and priorities. GAO's appropriate role is not to formulate policy but to contribute information and analysis that decisionmakers can use in evaluating options and making policy choices. Congressional staffs, majority and minority, overwhelmingly suggest that GAO's most effective contribution to decision making is to provide accurate, reliable information and fact-based audit and evaluation.

The panel recommends that GAO revise its vision and mission statements to reflect more focused and realistic objectives, building on its own standards and guiding principles, along the following lines:

GAO's principal mission is to produce high quality research that is objective and independently derived; fact based, accurate, and timely; and presented in a way that will be meaningful and useful to responsible officials performing oversight and formulating legislation and policies to guide the management and accomplishment of public purposes.

The panel recommends a shift in GAO perspectives and methods, from promoting process-oriented controls to examining the root causes of problems in order to help improve the effectiveness of program outcomes. GAO should also ensure that it has the staff skills and resources to match its work priorities, particularly in the areas of program evaluation, cost analysis, and financial and other management systems. GAO should take a cautious approach to policy analysis and policy development, building from a foundation in research, audit, and evaluation. Congressional requesters of GAO work should not put GAO's role and reputation as impartial, objective auditor and evaluator in jeopardy by posing research questions that inevitably place GAO in areas of conflict over policy priorities and values, without a solid factual base or objective standards for review.

GAO Strategic Planning

GAO's internal strategic planning and its relationship to congressionally-requested work is not known to most congressional members and staff. A more open strategic planning process at GAO would help create greater understanding of GAO's work priorities and address concerns about GAO's perceived reliance on congressional requesters to define its work.

GAO maps its work priorities through an internal strategic planning process that identifies research questions and projects likely to make a useful contribution to meeting vital congressional interests and needs. The GAO strategic planning process is largely confidential, as is the nature of the jobs GAO is starting. Few outside GAO are aware of the strategic planning process, the resulting work plans in each GAO issue area, and the important role they play not only in GAO's choice of selfinitiated work but also in the requests that Congress makes for work. GAO and congressional staffs hold frequent discussions of issues and interests and negotiate requests for specific jobs, often based on issues that GAO has identified as the potential focus for useful future work or related to continuing streams of GAO work.

In the panel's view, the comptroller general's discretion to define important issues and to advise Congress on areas for audit and evaluation is important and valuable for GAO. But a higher level of openness, accountability, and exchange of information is needed; more sharing of information on GAO's strategic planning and choice of work can help to resolve misunderstandings and congressional uneasiness with the work GAO does and its use of resources. The panel recommends that GAO systematically consult with congressional committees, executive agencies, and experts in each issue area, as part of its planning process. GAO should provide its summary strategic plan to GAO's oversight committees and distribute the plan in its final form to other Members of Congress, as well as executive branch agencies and officials.

Quality Management and Customer Focus

GAO has taken an example from the private sector and TQM to "focus on the customer," which GAO generally defines as Congress. Interviews for this study and GAO's own 1992 survey of congressional staffs indicate some discomfort in Congress with GAO's "customer focus" language, and some suspicions that GAO is becoming too willing to tailor its work to the interests of requesters and satisfy the Committee and Subcommittee staffs who are the major requesters and users of GAO work. Approaching Congress as "customer" has several built-in problems:

1. Congress consists of hundreds of individuals with diverse and often conflicting interests. Therefore, work that might please one committee or member is unlikely to please all the others.

2. Unlike the typical private sector "customer," congressional requesters do not pay or even perceive a cost for GAO work, and most are not aware of the costs of the work GAO does for them.

3. GAO's credibility depends on its reputation for impartiality, which could be damaged by an overemphasis on pleasing any individual requester.

The panel recommends that GAO clarify what it means to be "responsive" to Congress while preserving objectivity and professional standards for audit and evaluation.

GAO WORK PROCESSES

Internal Design and Review of GAO Work

GAO work processes tend to proceed in uniform, hierarchical patterns with inadequate definition at the outset of the objectives, methods, and type of work, and cumbersome review processes at the end.

GAO work products include blue cover reports—ranging from substantial "chapter reports" to shorter "letter reports," written briefings, fact sheets, and correspondence issued in blue covers-as well as oral briefings, shorter and less formal correspondence, and historical or other factual series of studies or guidance. Despite the diversity of its tasks, GAO tends to work in uniform patterns that produce reports in similar formats. In the panel's view, controversies over GAO studies generally reflect misunderstandings of the basic research objectives and different views on sources and methods. Some congressional staffs do not feel that GAO did the work they requested. In some cases, GAO's methods do not seem suitable to its conclusions.

The panel has several major recommendations on GAO's work processes: (1) GAO should develop clear "terms of reference" (TORS) for each job before it is started, outlining the objectives, general methods and skills to be applied, timing and estimated cost; (2) at the beginning of each job, GAO should form work teams that represent the full range of skills and experience needed across all organizational units and levels at GAO, working collegially throughout the project; and (3) GAO should replace sequential, hierarchical reviews of reports with concurrent, interactive reviews, as well as frequent consultation and participation of supervisors and managers throughout the planning, research, and drafting process.

The panel sees developing terms of reference as the most important change from the way GAO now works. GAO should negotiate these terms with congressional requesters of a project, if any, and share them with the agency subject to study. Terms of reference for studies requested by committees should be shared with all committee members, majority and minority. GAO should retain flexibility to design the specific methodology and develop the research, but developments during the course of study that require major adjustments in the terms of reference should be discussed with requesters and the agencies in question.

Agency Comments

A major cause of concern about the objectivity of GAO's work is the way in which the results are shared with agencies and interested Members of Congress. Contrary to GAO's published standards for government auditors (the "yellow book”)—which call for comments from the agency subject to audit and evaluation-congressional requesters have increasingly instructed GAO not to get agency comments on its draft reports and sometimes not even to brief agencies orally about the results of a study. In some cases, this sets up "ambush hearings" for which neither executive agencies nor other parties have substantial notice.

Executive comments on GAO reports can serve several purposes:

• Reduce the potential for factual errors or misunderstandings;

• Reveal language that might raise sensitivities that GAO did not realize; and

• Contribute to greater cooperation and receptivity to the GAO reports on the part of the agencies being studied, thus significantly increasing the possibilities of effective response in the organization.

The panel recommends that GAO consistently seek written comments from subject agencies on all reports that audit or evaluate executive activities, with a 30day limit for responses, and potential extension to a total of 60 days in special cir

cumstances.

Performance Indicators

GAO emphasizes process not only in its recommendations but also in its tracking systems and its indicators of its own performance and accomplishments. The panel recommends that GAO revise its performance indicators, to reduce emphasis on

process and inputs and focus on measures of the outcomes and impacts of its work. GAO's performance indicators should include not only estimates of net savings but also associated improvements in the efficiency and effectiveness of programs. GAO should also increase its internal resources and capacity to analyze the costs and budget impacts of its recommendations, as well as to assess agency cost accounting and performance measurement systems.

External Peer Review

GAO should establish and budget for external peer review of its completed work on a continuing basis. The volume and diversity of GAO work are not compatible with formal review before jobs start or as part of the report review process. GAO, however, should work with professional associations and other experts to organize a process for having specialized panels of objective peers from outside GAO assess and report on samples of completed GAO work, covering several selected categories each year.

GAO FUNCTIONS AND WORK PRODUCTS

The nature of GAO work has evolved over the past 30 years, resulting in less emphasis on audits, particularly financial audits, and increased resources directed to program- and policy-related work. The quality, usefulness, and importance of GAO's work products are uneven. The panel also found lack of clarity in the objectives of individual GAO reports and the standards or criteria for making findings and recommendations. Overemphasis on procedural controls still prevails in many studies. Categories of GAO Work

GAO currently undertakes six principal categories of work: (1) financial audits; (2) economy and efficiency audits (including non-financial compliance audits); (3) program evaluation; (4) policy analysis and policy development; (5) management studies; and (6) special investigations. These categories are not sharply defined and elements of several categories are frequently mixed in a single GAO study. Partly as a consequence, a job may lack clear objectives at the outset in conflict with the "yellow book" standards that GAO prepares as a guide for federal, State and local government auditors.

The panel recommends that GAO develop clearer definitions, objectives, and standards for all the categories of work it performs; strengthen results-oriented assessment of government activities (particularly in the category of program evaluation); continue to strengthen its capacity to audit and advise agencies on systems and standards for financial and information management; include substantially greater cost analysis in its work in audit, evaluation, and policy analysis; base its policy analysis and policy development on fact-based audit and evaluation; and restrict recommendations for added oversight and control that are not based on a demonstration of expected net benefits.

GAO's special investigations work (which seeks to identify criminal behavior in government) should be kept separate and distinct from audit, evaluation, and management analysis done by issue area staffs. The panel recommends that congressional oversight committees work with GAO to develop clear standards for GAŎ investigative work and guidelines for referring requests for investigations to federal law enforcement agencies.

General Management Reviews and Related Work

GAO is at a crossroads in its management work. It has developed a cooperative, consultative approach to conducting general management reviews and providing technical assistance to agency leadership on systems improvements, which many agencies have found constructive. The panel finds inconsistencies between the attitudes and approaches involved in process-oriented, adversarial audits and the skills and relationships needed for GAO to play a potentially growing role in providing general management analysis and technical assistance to executive agencies, with the support of Congress.

The research done for this study shows that adversarial relationships between auditor/evaluator and agency or program management seldom lead to productive general management improvements. Aggressive oversight can identify, expose, and punish mismanagement that violates law-a legitimate function, within limits-but it historically does not improve management in the long term. Providing effective review and technical assistance to agency management usually involves a cooperative relationship with executives responsible for management change.

The panel recommends that GAO launch a major internal training process to convey the lessons learned from its general management studies throughout its own issue area staffs to bring those lessons to bear on all audit and evaluation activities.