Volume 20 Number 1
January 1981

Linda Tigges and Leona M. Zastrow

IN THE Spring of 1979, the Tribal Council of the Santa Clara Pueblo requested that an evaluation be made of its Headstart program. The purpose was to determine whether the program related to the needs of the Santa Clara children, how successful it was, what problems the teachers encountered, and whether the Pueblo should administer its own Headstart program, rather than having it administered by an outside agency.

Santa Clara Pueblo, one of 19 in the north central part of New Mexico, has existed since the 1300s. According to Francis Tafoya, Santa Clara educational planner, the Pueblo consists of 45,744 acres and has a population of 1,041 residing on or adjacent to the reservation. The labor force includes 436 people of whom 250 are employed. The educational programs within the Pueblo are the Bureau of Indian Affairs (BIA) Day School, the Headstart program directed by the regional tribal administration, and a bilingual program supported by federal funds.

Traditionally the Pueblo people have had little involvement in their schools. One of the ways they have preserved their culture through centuries of domination by outside groups is to keep much of their society closed to non-Indians. The Pueblo culture—the religious beliefs, arts, ceremonials and social behaviors—has been carefully separated from the education taught by outsiders in the schools. In the past, one of the purposes of the "white man’s schools" was to wean Indians away from their native culture.

Since the early 1970s, the policy of the federal government toward Indian education has changed. An effort is now being made to involve the parents of students in the educational system and to allow greater local autonomy. Federal programs such as Titles IV and VII are providing funds to Native American parents for developing cultural and linguistic programs for their children. In addition, the Indian Self Determination and Education Assistance Act of 1971 allows Native American communities to operate the federal schools in their communities and determine the curricula.,

In response to these programs, many tribes have taken steps intended to lead to the development of their own schools. One of the first steps is an assessment of the existing "Anglo" educational system to determine what they wish to delete or maintain in their own schools. In this way, the tribes can benefit from their contact with the dominant society while maintaining strong cultural bases.

It is in this context that the request by the Santa Clara tribal officials for an evaluation of the Headstart program was made. The tribal officials needed information to decide whether to administer the program themselves, and what changes to make if they did so. They needed to know whether the program was achieving its stated goals, how well it was being run, and what changes should be made. The evaluator needed to design and implement an evaluation to suit Santa Clara’s needs, fitting it into the context of the Pueblo culture yet maintaining certain standards of objectivity and respect for the cultural sensitivity of the people.

After discussions with tribal officials and a review of the literature, an evaluation design was developed in four steps:

1. Indentification of the items the tribal officials wished to have evaluated.

2. The restraints on research the governing board wished to impose.

3. Resolution of contradictions between the constraints imposed by the governing board and their requests for information.

4. Identification of methods by which the information could be gathered. A visual model for this methodology is shown below.

1. Does the program do what it is intended to do; that is, does it meet its own stated goals which include:

2. What administrative problems exist in the Headstart program?

Step Two: Responses from the group interviewed showed that the governing board wished the evaluators to operate under the following constraints:

1. Interviews were to be conducted within two days. A one-time evaluation was requested rather than any testing with follow-up.

2. Parent interviews were to be with those parents available and to take place during the two-day period, rather than with a random sample or with all parents.

3. Verbal interviews were to be used rather than written questionnaires because of time constraints and language barriers.

4. Individual names were not to be used.

5. Contact was not to be made with administrative or educational units outside the Pueblo.

6. Individual Headstart classes were not to be compared with each other.

The people of the Pueblos such as Santa Clara have been subjected to many studies some of which were initiated by the BIA. One result has been an expressed suspicion and hostility to further research. This operated as a constraint, requiring the research instruments to be designed to respect this sensitivity.

Step Three: The above information shows that the constraints did not allow for pre- or post-testing, random sampling, follow-up questionnaires or much comparison. Because of this and because of time constraints, little information on behavioral changes of students on other measurable program outcomes could be provided. Tribal officials accepted this limitation on the evaluation. The kind of evaluation design that seemed most appropriate and could be carried out within the constraints was a management or process audit of the program goals.

Step Four: The methods chosen were interviews and observations designed to gather information about whether program activities were being carried out as set forth in the program goals. A more extensive discussion of methods is provided in the next section.

Items referring to pre-kindergarten education were designed to gather information about the respondents’ perception of:

Items referring to health and nutrition were designed to gather information about the respondents’ perception of:

In regard to the social services goals, respondents were asked to describe:

To gather information about perceptions of parent involvement, respondents were asked about:

To gather information about perceptions of career involvement, respondents were asked to describe:

Subjects. Headstart students were taught in three classes for a total of 33 students. Each classroom had one teacher and one teacher’s aide. All Headstart teachers (3) and two teachers’ aides were interviewed, one teacher’s aide being absent. All teachers were members of the Santa Clara Pueblo. Of 32 possible sets of parents (57 persons), 16 persons were interviewed. Of these parents, at least five were from each classroom, three were men (one from each classroom) and eight of the parents worked for the Santa Clara Pueblo in some capacity. Two persons associated with the Headstart administration were also interviewed as well as the BIA kindergarten teacher, teacher’s aide, and bilingual teacher.

Procedure. The five Headstart teachers and teachers’ aides were interviewed for approximately one-half hour each. Parents were interviewed for approximately 15 minutes each; and BIA kindergarten staff and administrators were interviewed for approximately one-half hour each. Classroom activities were observed for an average of 20 minutes each. The procedures were carried out by two interviewers on two consecutive days.

Following the completion of the evaluation, the following discussion and recommendations were made to the Tribal Council.

Pre-Kindergarten Education. The results of the evaluation showed that the educational goals teachers emphasized were communication, physical development and readiness for kindergarten. Psychological social development was the area in which kindergarten teachers said Headstart children were best prepared. The methods of teaching most frequently used involved a great deal of teacher-student and student-student interaction. Children were encouraged to become involved in projects and to learn from each other. The evaluation did not identify any area of pre-kindergarten education that was neglected. However, there did appear to be some coordination problems among classes and with the kindergarten. Because of this the following recommendations were made:

Meeting Health and Nutrition Needs. Evaluation results showed that teachers were providing health and nutrition education in several areas such as teeth brushing, exercise, safety, and cleanliness. In addition, the Headstart program provided nutritious meals and health tests for students and teachers. Benefits derived by the students appeared to be better general health, better awareness of health needs and improvement in the speech of the children and of their eating habits. Because the evaluation showed that teachers were interested in learning more about health education and because parents apparently had a tendency to disregard reports made about students’ health needs, the following recommendations were made:

Social Services. While the evaluation showed that some social services contacts and referrals had been made by teachers, particularly with health representatives and food stamp programs, it appeared that this goal had not been made clear to the teachers. Because of this, the following recommendations were made:

Parent Involvement. The evaluation showed that, according to teachers and parents, the level of parent involvement in the Headstart program was "moderate" (3 on a 5 point scale). The level of influence by parents also appeared to be "moderate." Parents were involved in a variety of activities including substitute teaching, providing teaching demonstrations, helping with the playground or attending special student programs. Parents appeared to become most easily involved in substitute teaching or special projects and involved with the most difficulty in parent committee meetings and parent-teacher visits. Interviews with teachers suggested that parent demonstrations and parent-teacher meetings were the most useful part of parent participation.

Parents and teachers made a variety of suggestions about improving the participation program. These suggestions have been incorporated into the following list of recommendations:

Career Development for Teachers. This evaluation showed that career development opportunities were being provided to the teachers including in-service training, courses offered by the University of New Mexico (UNM) for completion of a baccalaureate degree, and a program offered by administrative staff for completion of the Child Development Association (CDA) credential.

Most of the teachers found these programs useful, though many commented that some of the in-service training programs were repetitious, provided no classroom follow-up and did not allow them to become involved in the program. Teachers also stated that the credentialing program was proceeding very slowly and irregularly, and the courses offered by UNM were not always appropriate for Headstart teachers.

Because of these comments, the following recommendations were made:

The evaluation showed the program to be relatively well-run, though there were areas of friction. Much of the friction seemed related to coordination problems such as conflicts in the Headstart and the tribal calendars, and lack of synchronization in Headstart and kindergarten entry dates. In addition, problems seemed to exist in program development and federal program evaluations. Because of this, the following recommendations were made:

The information gathered by the evaluation was provided to the tribal officials who considered the results and made a decision to continue the association with the regional agency. Responses from Pueblo officials suggested satisfaction with evaluation procedures. Based on these responses, the evaluators concluded that the evaluation design was successful in that it satisfied the needs of the users and was carried out within the constraints imposed by them.

A second conclusion of the evaluators was that many of the results of the evaluation reflected the culture and needs of the tribe and the inherent difficulties in administration of a tribal program by a regional agency. For example, the emphasis on teaching communication in the classroom and on using teacher-student, student-student teaching methods reflects the child-centered culture of the Pueblos as well as the need they see for teaching their children to speak English. The care in teaching about health and nutrition also appears to reflect the need teachers see to move away from the poor health practices of the past. It was suggested by members of the tribe themselves that the "moderate" involvement of the parents in the program may show a real interest by parents, given the large number of tribal committees and duties that make demand on their time. It was also pointed out that many of the parents are potters with workshops in their homes and cannot always leave easily.

Some of the regional scheduling and coordination problems may be unavoidable when dealing with several different tribes (of which Santa Clara is only one), all of which have different ceremonial calendars and strong feelings of autonomy. Some of the problems of coordination between the Santa Clara Headstart and kindergarten may occur because the kindergarten is operated by the federal BIA Day School, a different agency than that administering the Headstart program.

The evaluators also came to a third conclusion which, in terms of educational research, they felt to be the most important product of the study. This was that the constraint-needs model, which served as a basis for the study, was successful in determining the kind of evaluation that could be carried out given the social context in which the evaluators found themselves. It appeared that the model was successful because evaluators frequently do find themselves in social context different than the one in which they were trained to be evaluators, and occasionally, very different from the social context in which the concepts of social science research were developed. However, this difference is rarely accounted for in the evaluation they are asked to do. That is, evaluators are asked to do acceptable social science evaluations of programs whose participants are from a culture with basic assumptions different than those of Western science. At the very least, this poses problems in communication. More seriously, it may lead the evaluators to inadvertently violate certain tribal customs, which results in the failure of the study and a loss of trust from the tribal for them and other outsiders.

In the case of this evaluation, the constraints needs model helped sort out some of the contradictions between the basic assumptions of the social context and a social science evaluation and provided a vehicle for resolving the contradictions. The model served to protect both the client group and evaluators from unrealistic expectations and provided a more useful outcome.

Blackwell, Peggy J. Goals of Secondary Education as Perceived by Education Consumers. Albuquerque: University of New Mexico Press, 1977.

Chadwin, Mark L. "Introduction - The Nature of Legislative Program Evaluation." Legislature Program Evaluation in the States: Four Case Studies, edited by Mark L. Chadwin. New Brunswick, New Jersey: Rutgers University Press, 1974.

Gotts, Edward E. Marketable Preschool Education (MPE) Program. Charleston, West Virginia: Appalachia Educational Laboratory, 1977.

Jannusch, Marlene R., and Dolli H. Big John. "Hochungra Headstart Model". Journal of American Indian Education. October, 1976, Vol. 16, No. 1, pp. 1-9.

Kaines, Merle B., et al. Immediate Short-term, and Long-Range Effects of Five Preschool Programs for Disadvantaged Children. ED 152-043. March, 1978.

Mead, Margaret. Growing Up in New Guinea. New York: William Morrow and Company, 1975.

Steiner, Vera John. Learning Styles Among Pueblo Children. Albuquerque: University of New Mexico Press, 1975.

Dr. Linda Tigges may be reached at 322 Magdalena, Santa Fe, New Mexico 87501. Leona M. Zastrow authored an article for JAIE in October, 1978: "American Indian Women as Art Educators."