CHAPTER FIVE

LESSONS LEARNED FROM THE SUMMER OF SERVICE

This chapter examines some of the basic conclusions that can be drawn, and the lessons learned, from the Summer of Service experience. These insights may be of use in guiding future funding and technical assistance decisions by the Corporation. The chapter begins by reviewing the overall performance of Summer of Service sites, then looks at ways to improve local program management, and finally suggests mechanisms to facilitate on-going tracking and evaluation of similar programs in the future.

Overall Assessment

It is important to keep in mind that the basis for this report's analysis of program characteristics was data collected through forms completed by grantees and participants. Because the scope of the study did not include either an implementation process analysis or an impacts analysis, there are limitations on the conclusions that may be drawn. Nonetheless, it seems clear that:

1.

The 16 Summer of Service grantees represented an extremely diverse set of programs in terms of organizational structure and setting, activity focus, and local program philosophy.

Despite the fact that all were operating under and satisfying the same set of program regulations, it is important to recognize that the 16 Summer of Service programs represented a remarkably diverse set of local programs along a variety of dimensions.

Participant levels varied from 40 to 250 among the sites. Some sites had all participants operating under the direct supervision of a single lead agency (for example, ACORN), whereas others out-stationed the participants with almost two dozen independent organizations (East Bay Conservation Corps). Some programs were located in inner-cities (Harlem Freedom Schools), others in rural areas (Red Lake).

In terms of program complexity, the 16 sites ranged from grantees focusing on a single one of the four Service Areas, to programs addressing all four (see Exhibit 5.1). For example, some emphasized health issues (such as ICARE in Philadelphia), while others undertook a much broader range of issues (Ohio Wesleyan). The number of distinct service activities reported by

1

Chapter Five: Lessons Learned from the Summer of Service

individual grantees ranged from 3 (at Clark University in Atlanta) to 52 (at Building-Up in L.A.).

There also were interesting variations in the local program philosophy regarding the emphasis on beneficiaries versus participants. Some sites (such as Summerbridge) espoused the view that the fundamental focus of the local program was service delivery, and the principal role of the Summer of Service participant was to "give it all up" during the nine and a half weeks to provide optimum service to beneficiaries. Other sites gave more equal emphasis in their allocation of program time and resources between service to beneficiaries and the service learning experience for participants (Ohio Wesleyan). In yet other sites (such as Red Lake), the participants themselves could be seen as the primary beneficiaries.

The sites also varied in terms of the labor-intensiveness of the services and the range of beneficiaries. For example, some grantees focused the bulk of their program resources on the same group of children continuously over the course of the summer. Other grantees had projects that lasted a single weekend and were designed to benefit the entire community.

2.

Despite their differing approaches, the Summer of Service sites were all largely successful in terms of the delivery of tangible service benefits to their communities, and most reported a high level of satisfaction for the majority of participants. The diversity fostered under the Summer of Service program guidelines appears to be

a positive feature of the program. The flexibility of the program guidelines allowed creativity and local initiative. Programs were allowed to develop their own special features to reflect local conditions and relationships, which resulted in a wide array of benefits.

All the sites contributed valuable services to beneficiaries as well as providing leadership skills and an enhanced sense of civic responsibility to their participants. At 14 of the 16 (87 percent) sites, a sizable majority of participants indicated that they were satisfied or very satisfied with their summer experiences, and at the remaining two sites no more than 31 percent of participants expressed outright dissatisfaction. We have learned, therefore, that the overall community service model embodied in the Summer of Service initiative appears to work effectively in a number of different settings.

Chapter Five: Lessons Learned from the Summer of Service

Management Issues

Despite this overall success, however, there were still a number of features of the summer experience that suggest ways in which similar future community service efforts might be improved.

3.

At some sites, negotiations with project partners and specification of objectives were still in an rudimentary stage when the program began.

The initial training visits conducted by Abt research staff found more than a few grantees, in the second or third week of the service delivery phase, still in the process of identifying project locations and negotiating partners' roles. This situation stemmed in part from the last-minute start-up of the summer initiative due the uncertainty over federal funding sources and delayed award announcements. We also found that at the beginning of the summer many local programs had made little progress in specifying concrete objectives and performance measures for their activities, despite the requirement in the Summer of Service application guidelines that applicants specify the quantifiable benefits that they hoped to achieve for their community and participants, as well as the criteria upon which their program should be evaluated. In some cases, the lack of specificity about project focus and objectives left participants uncertain about the functions that they should be performing in their local placements. The process that the Abt staff followed of assisting the grantees in completing the Program/Service Area Description forms both encouraged those grantees that had not yet specified performance objectives to do so, and provided them with technical assistance in this task.

In the interests of improved program planning and focusing of resources at the local level, prior to the execution of future grant agreements the Corporation may want to be more systematic about confirming the formal status of inter-agency agreements, project objectives and workplans, and the measures of progress to be employed. If any of these elements in determined to be inadequately specified, it can be addressed through a special condition in the grant agreement.

4.

Chapter Five: Lessons Learned from the Summer of Service

Many sites were not fully prepared for the extent of the administrative demands of the Summer of Service.

Many sites seemed to be overwhelmed by the administrative workload associated with the Summer of Service activities, including both the tasks associated with the evaluation and the other requirements regarding records and reporting, setting up payroll, transportation, insurance, mechanisms for disbursing post-program benefits, etc. This occurred in both novice and experienced organizations. In part, it was the result of the last-minute start-up of the Summer of Service and the fact that the local programs were trying to undertake an extremely complex venture over a very short timeframe, with little in the way of administrative resources. Those entities that were new to federal grants were taken by surprise by the extent of administrative provisions and assurances that had to be satisfied. As a result of these factors, some Summer of Service grantees found themselves trapped in a reactive, crisis-oriented management mode for the entire summer, which was extremely stressful. Most program sites probably could not function effectively on a long-term basis under such strain.

To help reduce such stresses on program sites in the future, the Corporation should conduct an explicit examination of what these administrative demands will be, and what level of administrative staffing and capabilities need to be available at the local site to comply with them. The Corporation should also take steps through its funding process to ensure that the necessary resources are in place as a condition of grant award.

Implementation of these recommendations should improve the level of program management over what was seen at some of the Summer of Service sites. However, the point should be made that, given the short lead time and limited administrative resources available to the Commission and the 16 program sites for the start-up of the new initiative, the varied and extensive accomplishments of the Summer of Service are even more impressive.

Evaluation Issues

In addition to the lessons that can be derived from the Summer of Service experience regarding ways to improve program management capacity, the summer initiative also provides some insights into steps to facilitate future evaluations of similar efforts.