RECOMMENDATION: GAO should give particular emphasis to instituting a regular process of objective, external peer review involving periodic review of a sampling of recent GAO work in the category of program evaluations (see pp. 35-36). Policy Analysis and Policy Development Policy analysis and policy development work is the most difficult category of GAO work to isolate and assess. Many GAO reports and testimony, including most program evaluations, have clear policy implications -- often both appropriate and unavoidable. In recent years, Congress has asked for work with a primary focus on evaluating a policy initiative, advising on the effects of various legislative formulations, or performing analyses that can provide a foundation for developing policy. Examples of areas in which Congress has requested GAO work include: agricultural support programs, immigration reform, acquisition reform, military base closing strategies, and deficit reduction. Any audit or study of program areas like these touches on policy and generally affects political interests. GAO performs little formal policy analysis (i.e., evaluating and comparing multiple policy or program options to achieve a specified objective);" however, many GAO reports are directly related to gathering information about a particular program or option. GAO's internal strategic plans also emphasize policy-relevant work. - Based on the panel's review, GAO's policy studies range from solid, compelling, fact-based work to summaries of prior work to views on legislation, policy, or program proposals, building on limited interviews and literature review. Some in Congress have expressed concerns which the panel shares that GAO has on occasion moved too far into advocating policy, pushing into policy formulation more appropriate to elected officials and then urging congressional acceptance of GAO recommendations. GAO's appropriate role related to substantive federal policies is in analyzing and presenting policy options, building on its program expertise, previous evaluations, and fact-based research. The panel found no evidence of deliberate partisan, political, or other bias in GAO's policyrelated work. However, the closer GAO comes to value-laden policy issues on which consensus is lacking, the more cautiously it should proceed and the more emphasis it must give to developing a solid, reliable factual basis for its analysis. In heavily policy-related work, GAO also must explicitly delineate its methodology and sources. GAO should carefully review the nature of each proposed project in this category, the questions to be addressed, and the approach to be followed before accepting and initiating the work. "While GAO offers courses in policy analysis in its Training Institute, it does not include policy analysis as a category of work in its formal listings of work products. GAO, however, highlights what it calls "options analysis reviews," targeted to "improving the information base on which policy decisions are made." Policies/Procedures Manual, p. 2.0-2. Congressional requesters can be expected to turn to GAO for policy analysis -- particularly for factual analysis and informed perspectives that seem to be missing in the political debate, and also for assessment of issues on which they want an outside view. A considerable number of committee staffs interviewed for this study expressed interest in having GAO evaluate and present options for addressing identified policy goals -- a more classic "policy analysis" function -- rather than having GAO simply extrapolate from a program review to present a single policy recommendation. The current strategic planning process at GAO and work its priorities seem to be moving in a different direction, encouraging issue area leadership to become more involved in and to make their work more relevant to policy formulation. GAO can obtain congressional requests for a considerable amount of policy-related work, including reports on many of the significant and controversial emerging policy challenges for the federal government. However, both GAO's 1992 congressional survey and interviews for this research indicate substantial uneasiness among congressional members and staffs with this approach. In policy areas in which GAO is simply weighing in with its views among many other sources including interest groups, executive agencies, think tanks, technical and academic experts, and congressional staff -- without any special expertise or background support, the authoritativeness of GAO's contribution to congressional understanding and deliberations is weakened. Moreover, many congressional comments note that effective audits and program evaluations can speak louder than brief summaries of opinion and policy recommendations. Staffs and members appear to want GAO to provide them with facts from which they can draw their own conclusions. RECOMMENDATION: In its contribution to policy debates and decisions, GAO should focus on providing professional quality audits and evaluations based on sound, objective information and analysis. GAO should give renewed emphasis to developing and assessing policy options and their potential costs and benefits, rather than recommending and defending a single policy choice (although GAO leaders should continue expressing their recommendations in areas they have researched when their opinions are requested). RECOMMENDATION: - GAO policy analysis and other policy-related work should build as much as possible on fact-based analysis and program evaluations. GAO should establish a special process to review proposed policy analysis and policy-related work, both as part of its strategic planning process and in reviewing individual job starts. RECOMMENDATION: GAO should use the framework of existing laws that created federal agencies and programs as its guide to public purposes and the starting point for its studies. RECOMMENDATION: GAO should be especially careful in formulating terms of reference for studies with heavy policy implications, particularly if GAO has no past base of empirical evidence from its earlier audits and program evaluations. It should spell out subjective and objective factors to the requester. In the terms of reference for jobs that involve policy analysis, GAO should include: identification of the problem and questions to be examined; specification of data, methods and sources to be used; criteria for assessing existing legislation and programs that supply the evaluation framework; potential options; and explication of the likely limitations of the GAO analysis. An increasing volume of GAO work involves summarizing past work to emphasize findings or bring to public attention recommendations that have not yet been implemented. GAO also has prepared a number of policy-oriented reports based largely on secondary sources. The summarizing work includes GAO's Transition Series of reports addressing "important policy and management issues facing the Congress and the new administration." This series -- which GAO initiated on its own in 1988 and received congressional requests to produce again in 1992 -- reviews and synthesizes existing GAO work related to particular issue areas, programs, and organizations. Some congressional staff readers described the transition reports, issued just after the presidential and congressional elections, as useful summary and background information. While these reports did not reflect new work or newly-developed data, in most cases they pulled together facts and perspectives from a series of reports and provided the results of substantial numbers of reports in brief, highly readable, easily accessible, and up-to-date form for busy staff members and officials. Several committee staffs, appointees, and others said they used the transition reports heavily as background primers on issues and challenges facing an organization or program, and to identify patterns of problems or threats. To others, however, the transition reports seemed shallow and self-evident, or overly self-referential and self-congratulatory. Some congressional critics focused on only one or two points in the transition reports -- for example, statements in the 1988 Budget Issues transition report on the need to address the federal budget deficit, which they saw as overtly political and partisan in their message and effects. Addressing the Deficit: Budgeting Implications of Selected GAO Work" is an example of a one-time report based on prior GAO work. Compiled by AIMD, it draws together GAO findings on 57 different programs or federal activities, summarizes the options/recommendations GAO has presented in past work, and provides estimates (obtained from CBO) of the potential federal savings of following GAO's recommendations. The report is not as comprehensive as CBO's March 1994 report, Reducing the Deficit: Spending and Revenue Options. However, it brings together the results of a wide range of GAO studies and presents them in brief, readable form, with associated dollar benefits for each. The material is clearly based on past research, and reflects rapid work on GAO's part to meet tight timetables in Congress' federal budget cycle. 79GAO/OCG-94-3, March 1994. The report groups past GAO work according to three major themes and objectives, which it presents to Congress as possible decision rules: The purposes of a program: whether the purposes are still appropriate for conditions today, and its cost-effectiveness at meeting stated purposes; Beneficiaries of a program: how well the current beneficiaries match the program's intended purposes, and the effectiveness of formulas or eligibility standards for targeting intended beneficiaries; and Operational efficiencies and potential improvements in organization, processes, information, and performance measurement, to improve accountability and effectiveness. Several committees have found the budget report useful, particularly for those who have not tracked GAO work on particular programs, though some described it as "nothing new," "superficial," or arbitrary and adversarial in its selection of programs and spending cuts to feature. Programs not subject to prior GAO studies escaped consideration. However, this type of report may prove a valuable test of how GAO can present its wide range of work and knowledge in a way that will be valuable to its primary audiences, in Congress and in the executive branch. RECOMMENDATION: GAO should change its Transition Series to a periodic summary of issues, not tied to presidential election years or transitions in the executive and legislative branches. GAO's issue summaries should be targeted to compiling previous GAO analysis and research findings that would be useful to top legislative and executive branch officials. GAO should avoid commenting on highly political policy positions except in the context of solid, fact-based research and analysis. RECOMMENDATION: GAO should keep up to date on program and policy developments and emerging issues in the executive departments and programs, and be prepared to provide summary briefings on current federal management and spending issues and concerns to new congressional members and other elected officials and their staffs, appointees, and other managers in the executive branch, if requested. RECOMMENDATION: GAO's contribution to federal budget decision making should be focused on carrying out its core mission: providing first-rate audits and evaluations of programs and activities that use public resources, building on a foundation in fact-based research, data, and analysis. General Management Reviews and Related Work As GAO conducted more agency audits and evaluations through the 1970s and early 1980s, it began to recognize that many deficiencies in program performance and financial and other systems were actually general management problems that required changes in management perspectives and processes. In the early 1980s, GAO began a series of self-initiated studies which it called general management reviews (GMRS) to deal with agency-wide or department-wide management issues. It centered responsibility for this work in the General Government Division (GGD), and brought in staff from other issue areas on special assignments to the study teams. GAO cites 48 completed GMRS, which were reviewed for this study. The products ranged from thick volumes that cover all of a department's major management systems to relatively slim summary reports that highlight points for top-level management, building on a series of briefings and other less formal communications. The GMRS had several objectives: To offer constructive recommendations to executive agencies and their managers for organizational and systems changes to improve performance; To provide information to Congress to stimulate interest in management issues and suggestions for change conducive to better management in the federal government; and To increase GAO's understanding and experience relating to agency operations and management improvement options. In recent GMRS, GAO has emphasized close constructive relationships with top organizational leadership in the agencies. In the course of these studies, GAO actively sought cooperation of the top leaders of each agency or department. A constructive relationship is particularly critical in this type of work because most recommendations relate to management change and require direct action and initiative by agency executives. As with many initiatives, good management cannot be dictated or audited in; ultimately, it depends on the understanding and commitment of executives themselves. GAO staff interviewed about their work on recent general management reviews believe the work has had a positive impact on the federal agencies, built improved relationships with agency top management, and developed GAO's understanding of management and agency-wide dimensions of program performance. The actual results in terms of recommendations implemented and results stemming from them are not clear, and they vary from agency to agency. While the perspectives of agencies involved in GMRs differ, many appreciated the opportunity to work with GAO in a positive way and agreed with most of the GMRs' conclusions and recommendations. Agency representatives, however, often noted in interviews about the GMRS that the GAO teams initially were not familiar enough with, or did not appreciate, general management considerations and practical day-to-day operational issues; a common comment was that agency staff had to spend considerable time and effort to "educate" GAO teams about how the agency and management structure worked before they could begin their review. (In interviews at GAO, staff |