M. Bryce Fifield, Navajo Community College:
Marvin G. Fifield, Utah State University

If the investment in individual assessment is to be cost-effective, school personnel responsible for making placement and programming decisions must not only understand the reports, but also have confidence that the conclusions and recommendations are valid and appropriate. This study developed and employed a combination of questionnaires and interviews for gathering data from educators to determine the effectiveness of psychoeducational assessment reports in communicating testing results and programming recommendations. A sample of 19 reports were selected from the 90 students tested in a BIA boarding school for junior high and high school students. Measures of relevance and the appropriateness of diagnostician's conclusions and recommendations concerning the referred students was obtained. Conclusions suggest methods of improving psychological testing reports and recommendations are provided.

STATE, FEDERAL, and BIA regulations implementing PL 94-142 (Education for ALL Handicapped Act of 1975) outline specific guidelines to be followed in determining a student's eligibility for special education services. These guidelines make provisions for the development and implementation of an Individual Education Plan (IEP) which outlines both long term goals and short term instructional objectives for the student. One of the most frequently used methods of determining eligibility and a student's academic strengths and weaknesses is through a battery of individually administered psychoeducational tests (Bureau of Indian Affairs Special Education Handbook, 1980). The special educator often relies on this detailed assessment of the skills the student has mastered in developing the specific instructional objectives (Hofffieister, 1977).

The data obtained through individual psychoeducational testing is a valuable source of information for the development of IEPs. Critical components of special educational planning such as formulating instructional objectives, identifying entry points in skill hierarchies and evaluating the effectiveness of a student's educational program can be made easier if the results of individual psycho-educational testing are understood and are used appropriately by school personnel (Schenck & Levy, 1979; Hofmeister, 1977).

In many situations, the written report of psychoeducational findings is the primary means by which the psychologist communicates testing scores, observations, and recommendations to the teacher (Bagnato, 1980; Erwin & Cannon, 1973; Hartlage & Merck, 1971; Rucker, 1967; Shively & Smith, 1969). The meaningful communication of testing results is as important as the validity of the tests or the skill of the psychologist who administers the instruments (Harflage, et al., 1968; Talent, 1976; Hammond & Allen, 1953).

Several researchers have explored the effectiveness of psychoeducational testing reports as a means of communicating testing results. Talent (1976) reported that the problems professionals encounter with psychological reports can be categorized into five areas: (a) problems of content, (b) problems of interpretation, (c) problems of the psychologist's orientation, (d) problems of communication, and (e) problems of the science or profession.

The frequent use of technical terms and confusing jargon is often one of the primary factors inhibiting effective communication between the parents of handicapped children and professional staff members (Dembinski & Mauser, 1979). Rucker (1967) found that teachers and psychologist could only agree on ten of thirty-one terms commonly found in school psychologists' reports. In a similar study, Shively & Smith (1969) found that teachers, counselors, and college students on the average only knew sixteen of thirty commonly used technical terms and phrases. Conversely, Baker (1965) reported that public school personnel did not find psychological reports too technical or theoretical in nature, but still felt that the communication between the school psychologist and the teacher was poor.

Another measure of the quality of a psychoeducational testing report is the value of the information in determining the most appropriate educational placement and program for the student. As far back as 1953, Daily reported that if the psychological evaluation was the only source of information available to independent clinicians, they could only agree on clinical decisions about the client 53% of the time. More recently, Hartlage & Merck (1971) found little relationship between what supervisors at a rehabilitation facility felt was useful information and what was included in psychologists' reports.

Rucker (1967) found that reports which were rated as "poor" by school teachers frequently had vague recommendations, unrealistic goals, and did not adequately address the referral questions. Brandt & Giebink (1968) also found that teachers tended to prefer psychological reports which included specific and appropriate recommendations.

Although the use of confusing technical terms may not be typical of most psychological reports, ambiguity and jargon are often found in a specific report for a given reader (Caudra & Albaugh, 1956). This unique interaction between the psychologist writing the report, the report, and the teacher or parent reading the report is likely to result in ineffective communication when the psychologist has poor training in the techniques of report writing. To eliminate or decrease some of the technical difficulties encountered in report writing, Carr (1968) advocated the use of a standardized test report outline in which specific information was reported and discussed for all clients tested.

Although the use of the written report as the primary means of communicating assessment results may have many limitations and shortcomings, the fact remains that no matter how well a testing report may be written, how clear and free from jargon it may be, how specific and realistic the recommendations may be, if the testing report is not read and used by the student's teachers, the expensive and time consuming process of assessment will have little effect on the student's education. In some cases, the psychological report is completed to meet the letter of the law, but ignored in preparing the IEP (Holland, 1980). Practitioners report that psychological assessment reports are often either accepted without question, or are totally ignored by the placement and IEP teams (Duffey & Fedner, 1978; Fifield, 1982; Fifield, 1983). Methods are needed to gather objective data to determine the effectiveness of written psychoeducational testing reports and communication assessment results to teachers, special educators, and support personnel.

It was the purpose of this study to determine how well psychoeducational testing reports communicate evaluation results and programming consideration to teachers employed in a Bureau of Indian Affairs secondary boarding school. From the above data, recommendations designed to improve the effectiveness and relevance of psychoeducational testing reports were prepared.

This study was conducted at Intermountain Intertribal School (Brigham City, Utah), a Bureau of Indian Affairs secondary boarding school with an enrollment of approximately 600 high school and junior high school Native American students. The Exceptional Child Center at Utah State University contracted to conduct psychoeducational evaluations to students referred for special education services. Table 1 summarizes the domains tested and the instruments used.

The student assessment procedures utilized the following techniques to control for potential culture and language bias by:

1. Use of non-biased standardized tests.

2. Use of criterion referenced instruments.

3. Use of culture specific measures of adaptive behavior.

4. Use of a dispositional assessment model which focuses testing procedures on the solution of the referral problem (Cole & Magnussen, 1966).

5. Assessment of the student in his or her dominant language when necessary.

6. External validation of the psychometric findings by securing a second opinion.

7. Long term follow-up and evaluation of the assessment process and procedures.

The records of students referred to the school's special education program were first reviewed by the school's interdisciplinary committee. Those students the committee felt needed additional evaluation data were then referred for psycho-educational testing. Ninety junior high and high school aged Native Americans boarding at the school were referred by the interdisciplinary committee for further individual assessment. Individual evaluations were then scheduled following the procedures outlined above. Following the testing, a psycho-educational report was prepared for each student which addressed the following specific information.

Three certified school psychologists each with over three years' experience in school settings administered the evaluations and prepared the psychoeducational reports addressing the information listed above. The testing reports together with testing data were then presented to a second set of certified psychologists to review, critique, and provide a second opinion on the findings and recommendations. When there were differences in interpretation or recommendations between the two psychologists, such differences were negotiated and corrections were made. The reports along with testing protocols were then presented to the school.

Population. To determine the relevance and effectiveness of the psychoeducational reports, the findings and recommendations of the reports had to be correlated with other performance data about the student tested. Thus, for the purposes of this study, only those records of students tested who qualified for special education placement were analyzed. Forty-four of the ninety students tested met this criterion. A preliminary examination of student records indicated that the majority of the staff who signed out for the student records and reports were also members of the student's IEP committee. The school records showed that 34 staff members served on IEP committees, 16 of whom served on two or more committees.

Considering the above factors, the population for this study was defined as school staff members who had participated on two or more IEP committees or who had accessed the IEP files of two or more students. Twenty school staff members representing six instructional departments (special education, teaching supervisors, counseling, instructional media, public health, and consumer education) met the above criteria.

Selection of sample reports. Because of the duplication of staff assignments many students had identical or similar IEP teams. The names of the 44 students and the names of school staff members on the student's IEP committee were cross-referenced on a matrix. The cross-reference matrix made it possible to identify a small number of students who had the majority of staff members serving as IEP team members.

By examining the matrix, it was discovered that seven students evaluated as described above had been assigned IEP teams which included 19 of the 20 targeted staff members. It was determined that these 19 persons would be the respondents in the study and the psychoeducational testing reports of the seven students would be used as samples for the school staff members to review. Each of the seven sample reports were reviewed by the assessment team to verify that the sample reports were typical and representative of the assessment reports produced during the year.

This procedure ensured that school staff members would be required to review and critique reports which they had already used in the preparation and implementation of the student's IEP. The school staff members were asked to review and critique only the psychoeducational testing reports of students with whom they were acquainted. Staff members were therefore able to accurately assess the validity of the psychologist's conclusions because they had firsthand knowledge about the student.

Psychoeducational Testing Report Critique Form. A four-part critique form was developed to aid staff members in reviewing the sample reports. In Part One, the respondents were asked to rate general attributes of a sample report in five areas: (a) how clearly the findings were reported, (b) the frequency of technical terms or professional jargon, (c) how the sample report compared to other reports the respondent may have used, (d) how well the recommendations addressed the referral question, and (e) how well the report addressed the special issues of cultural and language bias.

In Part Two of the critique form, respondents were asked to evaluate how well each of the seven sections of the report met its stated objectives and the extent the information in the given section was useful in preparing the student's educational program. These ratings provided a measure of the effectiveness of the structured report format in providing useful and relevant testing information.

In Part Three, specific conclusions the psychologist drew from the student's performance during the evaluation were taken verbatim from the sample report. Respondents were asked to rate each specific conclusion and evaluate the clarity, whether the statement provided new information about the student, whether the statement was correct, and how useful each statement was in preparing the student's educational program.

In Part Four, the verbatim recommendations provided by the diagnostician for the student's educational program were rated. This rating determined how specific the respondent felt the recommendation was, whether the recommendation provided new information, whether the mcommendation was realistic given the existing resources at the school, and how useful the recommendation was in developing the student's IEP.

Structured interview. After completing the four parts of the critique form, each respondent was interviewed by the authors to identify specific examples of strengths and weaknesses in the sample reports. The use of the structured interview allowed respondents to clarify their statements and to provide additional information not elicited through the critique form.

Teacher evaluations of the reports, particularly parts one and two of the questionnaire, were quite favorable. The teachers felt the reports were clear, appropriate, and that the reported results and information provided in the reports contributed to the preparation of the student's educational program. Among the problems noted by respondents during the interviews were that summary reports showed poor composition, confusing sentences and poor paragraph structure, repetition and inconsistent report format, and typographical errors. Few of these problems were indicated by more than two respondents. While the frequency of these problems was relatively low, it should be noted that such factors significantly limit the clarity with which a testing report communicates the results, conclusions, and recommendations of the diagnostician.

Fifty percent of the respondents felt that either some or substantial jargon was used in their sample reports. However, only one respondent (6%) felt that the use of the jargon interfered with her understanding of the report content. This finding suggests that the editing of draft reports by and with input from a second professional who provided a second opinion on the findings and recommendations is a useful means of keeping technical words or jargon from obscuring the information in the report.

One of the primary methods used by examiners to correct for potential bias was the focus placed on functional assessment aimed at answering the referral question. Seventy-eight percent of the respondents rated their sample reports as addressing the referral question very well. This finding, in combination with the high ratings given to the sample reports with respect to how well they gave consideration to "special" circumstances, suggests that the use of the dispositional assessment model in combination with a sound diagnostic approach is effective at correcting for bias when testing minority students with bilingual/ bicultural backgrounds.

One hundred and two individual ratings were made on the 41 verbatim conclusions in the seven sample reports. In general, the respondents rated the conclusions as clear, correct, and generally useful. Table 2 provides a cross tabulation of how useful the conclusions were in providing new knowledge to the teachers. Table 2 suggests that the conclusions the psychologists made about the students which were viewed as most useful to the teachers were those which either confirmed what a teacher suspected or those which provided new information about the student.

Ninety-nine evaluative ratings were made of the 35 verbatim recommendations provided in the 7 sample reports. In general, the recommendations were viewed as appropriately specific, realistic, and useful. Sixty-one percent of the responses indicated that the recommendations confirmed what the teacher previously thought. Only 33% of the recommendations contributed new information into the preparation of the student's IEP.

The responses to the conclusions and recommendations made by the psychologist in the sample reports suggest that teachers view the usefulness of psychoeducational testing reports within a narrow range. Conclusions or recommendations made by the diagnostician which confirm what the teacher already suspected about the student were viewed as most useful. Statements which make conclusions or recommendations which the teacher didn't know about (i.e., add new knowledge about the student) are also perceived as helpful, but to a lesser degree. This finding suggests that efforts to improve the effectiveness of the psychoeducational testing reports must be directed to the teachers' understanding of what information can be derived from testing scores and train the diagnostician in how to communicate those findings to the teacher via the written report.

This study demonstrated a useful method for gathering data to determine the effectiveness of testing reports and how psychoeducational information is used in a special education setting. Findings suggest that the structured psychoeducational testing report format was useful in maintaining consistency and standard reporting data.

TABLE 2
Cross-Tabulation of Conclusions Rated as Useful and as Providing
New Information about the Student

The findings of this study suggest that the dispositional assessment model in combination with a functional educational assessment focused on the referral question provides teachers with useful information relevant to planning the student's IEP. These assessment procedures in further combination with language and culture free assessment instruments provide an effective means of controlling for culture and language bias, as perceived by the teachers using the reports. However, considering the limited sample size used in this study, the above conclusions do need to be explored further.

Bagnato, S.J. The efficiency of diagnostic reports as an individual guide to prescriptive goal planning. Exceptional Children, 1980 49 (8), 554-557.

Baker, H.L. Psychological services: From the school staff's point of view. Journal of School Psychology, 1965, 3 (4), 36-42.

Brandt, H.M., & Giebink, J.W. Concreteness and congruence in psychologists' reports to teachers. Psychology in the Schools, 1968, 5 (1), 87-89.

Bureau of Indian Affairs, Division of Exceptional Programs, Navajo Area. Special Education Handbook. Window Rock, Arizona: June, 1980.

Carr, A. C. Psychological testing and reporting. Journal of Projective Techniques and Personality Assessment, 1968, 32 (6), 513-521.

Caudra, C.A., & Albaugh, W.B. Sources of ambiguity in psychological reports. Journal of Clinical Psychology, 1956, 12, 109-115.

Dailey, C.A. The practical utility of the clinical report. Journal of Consulting Psychology, 1953, 17 (4), 297-302.

Dembinski, R.J., & Mauser, A.J. What parents of the learning disabled really want from professionals. Journal of Learning Disabilities, 1977, 10 (9), 578-584.

Duffey, J.B., & Fedner, M.L. Educational diagnosis with instructional use Exceptional Children, 1978, 44 (4), 246-251.

Erwin, H.M., & Cannon, T.M. Psychological reports: An alternative to traditional methodology. Psychology in the Schools, 1973, 10 (4), 404; 406.

Fifield, M.B. Evaluation of the Perceived Usefulness and Effectiveness of Psychoeducational Testing Reports at Intermountain Intertribal School. Unpublished Thesis, Utah State University, Logan, Utah: March, 1982.

Fifield, M.G. & Casto, G. Psycho-educational evaluation of approximately 90 students at Intermountain School. A contract proposal submitted to Bureau of Indian Affairs, Phoenix Area Office. Logan, Utah: Utah State University, 1979.

Hammond, K.R., & Allen, J.M. Writing Clinical Reports. New York: Prentice Hall, Inc., 1953.

Hartlage, L.C., Freeman, W., Horine, L., & Walton, C. Decisional utility of psychological reports. Journal of Clinical Psychology, 1968, 24 (4), 481-483.

Hartlage, L.C., & Merck, K.H. Increasing the relevance of psychological reports. Journal of Clinical Psychology, 1971, 27 (4), 459-460.

Hoftneister, A.M. Educating the mildly handicapped child, field test edition. Logan, Utah: Department of Special Education, Utah State University, 1977.

Holland, R.P. Analysis of the decision-making process in special education. Exceptional Children, 1980, 49 (7), 551-554.

Rucker, C.N. Report writing in school psychology: A critical investigation. Journal of School Psychology, 1967, 5 (2), 101-108.

Schenck, S.J., & Levy, W.K. IEP's: The state of the art -- 1978. A paper presented at the Annual Convention of the American Educational Research Association, San Francisco, April, 1979. (ERIC Document Reproduction Service No. ED 175 201).

Shively, J.J., & Smith, A.E. Understanding the psychological report. Psychology in the Schools, 1969, 6 (3), 272-273.

Talent, N. Psychological report writing. Englewood Cliffs, New Jersey; Prentice Hall, Inc., 1976.