---

The Assessment System at the SOE is a dynamic process rather than a static product. The faculty decided on the information needed to make decisions, create assessment instruments and collect and analyze the data. The system is a combination of well-established practices with new ideas designed to facilitate reflective practices of the candidates, programs, and unit.

The use of technology to assist with the management of data was initiated incrementally, beginning in the Spring of 2005 in the initial education courses (ED110/ ED192) with the adoption of an on-line portfolio system. This database grows each year, as more and more courses and instructors utilize the e-portfolios. This will allow for more timely analysis of data as well as simplifying the process of disaggregating the data for individual candidate and program improvement. The process of refinement of this system is ongoing.

---

The university community was involved in the development of the Unit and Program assessment systems. University regulations require that every academic major program be reviewed on a regularly scheduled basis every five years. For this review each major program prepares a self study of the curriculum, student outcomes, and supporting areas such as the library and registrar’s office. These self studies conform to a common format and utilize data for program planning and evaluation supplied by the University Planning and Assessment Committee (Academic Degree Program Self-Study Outline) The self-study is reviewed at two levels, the College level (AAC) and the University level (faculty senate) before being forwarded to the Office of the Senior Vice President for Academic and Student Affairs.

The candidate assessment system involves multiple data sources, many of which were designed in conjunction with K-12 teachers and administrators. Conversations with our K-12 cooperating teachers who supervise student teachers led to a revision in 2004 of long established instruments to more accurately define student achievement. Meetings are conducted each semester with these classroom supervisors to share any concerns about the assessment system. The GPSS district administrators have also provided input. The major change made to the SOE assessment system was the addition of the PRAXIS I exam at entry point to align with the new GPSS standards.

The assessment system (Table 1) allows for data to be collected throughout the candidates’ college career. Every course in the SOE has aligned its assessments with the elements of the Conceptual Framework. Generally speaking, as the various programs review their required course-work against the elements of the Conceptual Framework, they are able to comprehensively assess each candidate. This is an on-going process.

Table 1: Candidate Assessment System

Assessment Instrument	Used At
	Entry	Midpoint	Exit
	Admissions Interview	Application for Student Teaching	Completion of Student Teaching
Initial Programs: Elementary, Elementary: Early Childhood, Elementary: Chamorro/Culture, Elementary: TESOL, Secondary (all programs), Special Education
Praxis I	X
Writing Sample	X
Interview	X
Letters of Recommendation	X
Grade Point Average	X	X	X
Portfolio (professional and pedagogical knowledge)	X	X	X
Course Perform. Assessments	X	X	X
CE Course Evaluation	X		X
Exit Survey- Self Reflection			X
Classroom & University Supervisor Evaluation			X
Employer Surveys			X
Advanced Programs: Administration and Supervision, Language and Literacy, Secondary Education
	Application to Graduate School	Completion of Coursework	Completion of Thesis
Graduate Record Exam	X
Personal Statement	X
Grade Point Average	X		X
Comprehensive Written Exam		X
Thesis/Special Project			X
Portfolio			X

In order to gain entry into the SOE, candidates are evaluated using rubrics based on the Conceptual Framework for each data source (General rubrics). Data is obtained from student grades in their general education coursework (Transcript rubric), letters of recommendation (Letters of recommendation rubric), writing samples (Writing rubric), and personal interviews (Interview rubric).

In addition, more structure is being added in the form of a developmental portfolio. The Midpoint assessment (application for student teaching) is an area that is being strengthened at this time. The candidate meets with his/her advisor, where coursework, course evaluation from faculty, and GPA are reviewed. The content and rubrics of the Developmental Portfolio are determined by each program. Exit from the program is successful completion of student teaching. During this experience, data is gathered from Classroom Supervisors (ER-Evaluation form), University Supervisors (Evaluation form, and School Principals (where applicable). Student self-reflection is continued with an Exit Survey (Exit survey). In addition, data from recent graduates is gathered from employer surveys (Employer survey). This integrated approach allows for a complete picture of candidates’ knowledge, skills and dispositions.

Beyond the candidate level, the operations and programs of the SOE are also comprehensively assessed. Instructor evaluations (Course evaluation form) by candidates are carried out each semester to provide feedback to instructors and administrators concerning curriculum and instruction. As mentioned earlier, all programs regularly undergo university review. Through an Exit Survey (Exit survey), graduates provide feedback on program and unit operations. The Unit utilizes data relevant to the operations and programs gathered from the Classroom Supervisors of the Student Teachers. The Faculty promotion and tenure system is another quality control method for the faculty and administration (CFES manual, and faculty binders/vitae).

Yes. The assessment system begins with faculty assessment of candidates in courses. Every course has been approved by the Faculty Senate of the University to ensure institutional standards are met. Every course has aligned its outcomes and assessments with professional standards (SPAs or INTASC; see Table 2). The INTASC standards were adopted, as this is the direction the GPSS will be taking. Additional state standards (e.g. PRAXIS) have been incorporated into the assessment system.

Table 2: Alignment of Assessment System with State, Professional and Institutional Standards

Candidate Assessments
Evidence Source	Evaluated by	Frequency	State Standard	SPA Standard	UOG Standard	Status
Praxis I	Admissions Committee	Entry	X			Anticipated
Writing Sample	Admissions Committee	Entry	X			In Progress
Interview	Admissions Committee	Entry	X			In Progress
Letters of Recommendation	Admissions Committee	Entry, Exit	X			In Progress
GPA	Admissions Committee	Entry, Midpoint, Exit	X			In Progress
Portfolio	Admissions Committee, Advisor	Entry, Midpoint, Exit	X	X		Anticipated
Course Perform. Assessments	Instructor	Semesterly	X	X	X	Completed
CE Course Assessments	Instructor	Semesterly	X			Pilot
Exit Survey- Self Assessment	Candidates	Exit	X			Completed
Master Teacher Evaluation	Practioners	Exit	X			Completed
Dispositions	Instructor	Semesterly	X	X		Completed
Program Assessment
Program Review Process	UOG Peers	Every 4-5 years		X	X	In Progress
Student Evaluation	Students	Semesterly			X	Completed
Exit Survey	Students	Semesterly		X	X	Completed
Master Teacher Survey	Practioners	Semesterly		X	X	Completed
Employer Survey	Principals	Semesterly		X	X	Completed
Unit Assessment
WASC Accreditation	National Organization	Every 4 years			X	Completed
P&T of Faculty	UOG Peers/Admin	4-5 years			X	Completed
Student Evaluation of Faculty & Courses	Students	Semesterly			X	Completed
Annual Goal Setting	Administrator	Annually			X	Completed

As candidates pass through the first of three gates that determine entry into programs, multiple measures (writing samples, GPA, Interviews, performance in the courses and in the field) are examined to help faculty make decisions about acceptance. At midpoint (application to student teaching), evaluations by instructors, examination of class performance assessments and evaluations by advisors are examined. Before exiting the program, data from the classroom supervisor, UOG supervisor and the student teacher are triangulated to make a decision about successful completion of the program. All assessments of candidate performance have been aligned with the Conceptual Framework, which identifies the knowledge, skills and dispositions of successful teacher candidates.

The performance indicators (General rubric) for all elements of the Conceptual Framework are published and available for all students. The use of rubrics in measuring each element helps eliminate sources of bias in assessments. The rubrics for performance assessments in courses are shared and reviewed each semester within programs with standardization as much as possible without infringing on faculty academic freedom. Some rubrics, such as those for the writing sample at entry, have been developed collaboratively with faculty from the College of Liberal Arts and Social Sciences to ensure accuracy. Generally, the use of the rubrics provides additional feedback as to the fairness, accuracy and consistency of the assessment process.

---

Of the major components of the Unit assessment system is the Program Review process. This activity begins with the gathering of materials and additional information, including the comments of external members. A report describing both the qualitative and quantitative aspects of the program under review is written which includes an evaluation of the program’s advising process.

A draft program review report containing recommendations is forwarded to the Dean and all program faculty. The writing team allows the faculty and Dean two weeks to respond to the report, correct inaccuracies in fact or data, and take reasoned exception to judgments or conclusions drawn. All such input is appended to the self-study. After endorsement of the Faculty Senate, the report, all responses, and final recommendations are forwarded to the Senior Vice President for action.

Data about candidate performance is analyzed at each decision point: entry, mid and exit (see Table 2). The SOE faculty examines candidate artifacts that are maintained in an electronic portfolio. Some artifacts are common to all programs (transcripts, writing samples) while others are program specific (modified lesson plans for the Special Education program portfolio). Each artifact is measured against the specific elements of the Conceptual Framework. Program advisors meet regularly with their advisees to review candidate portfolios. This data not only helps in the monitoring of student progress, but also helps the Unit and Programs identify strengths and areas for further growth.

The formation of a Curriculum Committee, made up of representatives of each program within the SOE, has been a venue for reviewing aggregated data of all SOE programs. Any changes proposed by programs are discussed at this Unit level to examine its effects on all of SOE. This committee makes recommendations for assessment policies and procedures for consistency of data across programs.

The university has a student grievance system that the Unit responds to when required. This system has established procedures for complaints from students regarding courses, faculty, policies and procedures (Handbook). The SOE administration maintains records of all student complaints. They are handled on a case-by-case basis. In cases where a candidate chooses to not use the university grievance system, the School has a mechanism for establishing an ad hoc committee of the Academic Affairs Committee that may address the concerns of students. (UOG Student Handbook)

Basic demographic information of teacher candidates is managed utilizing the University “Colleague” system, a database of all university students. The registrar’s office provides demographic reports to various faculty and programs to assist in compiling of data. In the Spring of 2005, the SOE began the use of LiveText, an online portfolio software program to assist in the compiling of data related to student performance. Presentations were made by eStar, a company designing data bases to suit individual needs of Universities and companies. It was decided that Microsoft Access would be the database for use to keep demographic information on students while LiveText will be used to maintain candidate performance data.

Data from internal and external sources are vital to the assessment and evaluation of candidates, Programs and Unit. Internally, candidates are constantly assessed in their courses by faculty. Externally, at entry, letters of recommendation for candidates are required. Many times these come from community leaders or civic organizations. In addition, Cooperating Teachers and Classroom supervisors provide input on candidate performance. The recent addition of employer satisfaction surveys help round out the variety of external data sources.

As a Unit, the SOE program faculty conducts self-studies as part of the Program Evaluation process. Once completed, a committee of University peers outside the SOE examines the program review. Comments, concerns and recommendations are sent back the program to ensure program quality.

---

Since the SOE began its NCATE quest, faculty have become active consumers and users of data. When needs are identified, faculty members share this information at regular meetings to discuss the issues and recommend solutions. Changes are then initiated that address the identified needs.

An important source of exit information on School of Education graduates comes from the Yamashita Teacher Education Scholarship Program The Yamashita Program offers comprehensive financial assistance to students at UOG who are teacher candidates; the majority of our students participate in this program. Each university graduate who receives funds from the Yamashita scholarship is given an extensive exit survey. A survey of Secondary Education graduates indicated a need for more coursework on handling disciplinary problems and classroom management skills. In response to this survey, documented in the 1998 Secondary Education Program Review, a course in Teaching Strategies was development and added to the course of study for secondary teacher candidates.

Additionally, data from the Yamashita Scholarship exit survey has been an effective tool in a number of important decisions affecting teacher candidates (survey instrument). In 1997 and 1998, exit surveys cited inadequacies in the School of Education's technology equipment. This was one of the factors in the decision to upgrade, replace, and generally overhaul the school's technology lab. Graduating students also expressed a desire for more field experiences sooner in their academic careers. In response to this, the practicum handbooks have been rewritten and the total number of observation and participation opportunities were increased (Student Handbook.)

In piloting the Employer Survey on UOG graduates currently working in Guam's public and private schools, two principals indicated a desire for greater teacher candidate exposure to Direct Instruction as implemented in Guam’s public elementary schools. Likewise, students surveyed in Reading and Language Arts Methods courses have expressed a desire for more DI exposure. In response to these needs, the focus of the practicum component of the Reading and Language Arts Methods courses has been revised to ensure that candidates receive greater exposure to Direct Instruction in the public elementary schools.

In its 2000 Program Review, UOG's Second Language Program documented a need for greater visibility and stronger relationships with community colleges and departments of education throughout Micronesia. In response to this need, there has been a concerted effort by the SOE faculty to address these needs, particularly via distance education. In spring of 2000 they piloted a distance education course available at all Micronesian college campuses and Guam; in Fall of 2000 they added a second course. (course syllabi)

The GPSS has amended its requirement for teacher certification to include satisfactory achievement on the Praxis I examination of candidate's basic skills and knowledge. Early reports for the GPSS indicate a lower than expected passing rate of recent graduates. In response, UOG's School of Education has adopted a similar achievement standard on the Praxis I as one criteria for admission to its teacher education program.

Changes are made to SOE courses not only in response to data collected from student evaluations, but also in response to the collective professional knowledge of the program faculty who desire to keep their courses relevant and current, including the newest research findings as well as established methods and content. Particularly since the onset of our efforts toward NCATE accreditation, there have been a number of courses added, deleted, changed, or combined. Each change is documented in meeting minutes where they were discussed by faculty and in the justifications for substantive or non-substantive changes submitted for course approval and kept on file in the office of the Senior Vice President for Academic Affairs.

Graduate courses in Special Education have been modified and upgraded in response to a program review (2001). The program review committee recommended that the Special Education course of study needed to be redesigned to reflect inclusive practices for students with disabilities. The redesigned program is now in place and the faculty is pursuing further grant opportunities to enhance the quality of education and services that Special Education can offer.

Another area of need that, in the past, has been cited by numerous program reviews, student evaluations and student exit surveys as requiring immediate attention was directed toward upgrading the School of Education's technology lab. In response, an assessment was made as to the most cost effective and productive method of upgrading critical hardware and software needs. Reports of the assessment are noted in the Foundations and Educational Research Division file. The report provided a plan whereby the technology lab was thoroughly revamped and upgraded and is now a much used work station for all students in the School of Education.

Teacher Candidates are kept well informed of their progress as they move through the university system. They are made cognizant of progress, or lack thereof, within each course by means of grades and scores on exams and assignments, performance rubrics, comments from instructors, and activities as outlined in the course syllabus.

As they progress toward graduation, teacher candidates receive entry, midpoint and exit evaluations. The artifacts examined at each of these points are identified in the Candidate Assessment Manual (CA Manual) which is used by all programs. There are standards and rubrics in place for each step and students are given a copy of these standards, the rubrics with which they will be assessed, and the criteria for attainment of target, acceptable, and unacceptable performance in every category assessed. Students work with advisors to ensure that all evaluations are completed and that they are making steady, appropriate progress in their programs of study.

In addition, course performance assessments are completed by the instructors of specified courses. These evaluations focus on the three elements of the School of Education's Conceptual Framework: knowledgeable scholar, effective communicator, reflective decision maker, and disposition. They are accompanied by rubrics that indicate progress, and are added to the teacher candidate's file to show progress (assessment instruments).

Advisors review all data with teacher candidates at each decision point (entry, midpoint, exit). (SOE student advisement forms). Areas of strength and need for improvement are noted. When candidates do not move past a decision point, they, in collaboration with their advisor, develop a plan to address areas of concern.

Even as the quality of the graduates that the School of Education produces is evaluated, so, too, are faculty themselves regularly and comprehensively evaluated. This is accomplished using UOG’s Comprehensive Faculty Evaluation System (CFES), peer review and via the Promotion and Tenure Committee. An evaluation plan is completed yearly and requires the faculty member to reflect upon his/her own performance in teaching, research, and service, to determine if the current year's goals have been achieved, and to set appropriate goals for the next academic year (ER-UOG CFES manual). The faculty member compiles an extensive portfolio of evidence to substantiate accomplishments of goals established for the year. In a CFES conference with the Dean, or his appointed representative, the faculty member shares the evidence of achievement of the previous year's goals and discusses establishment of new ones for the next academic year. The effects of this ongoing assessment is evident in the achievements of, and work loads successfully carried by the faculty in UOG's School of Education.

At the end of every semester, each course and each instructor is evaluated by students in 23 areas that include instructor competence, effectiveness, responsiveness to student needs, adherence to high standards in course content, and overall quality of performance (UOG course evaluation form). In addition, many instructors conduct their own informal surveys to look for ways to improve course content and meet student needs.

Each semester, data gathered from student course evaluations of faculty performance are collated, summarized and returned to the instructor along with student comments. Instructors use this information to improve the overall quality of their teaching, to enhance or shift focus of teaching activities, and even as a means of expanding their thoughts on research possibilities (finding more information to share with students); service opportunities (application of learning theories in real life situations as an example to students) and participation in professional organizations (professional collaborations that expand reasoning and awareness that can lead to a more global approach to information acquisition and teaching practices).

The aforementioned response to students' suggestions for more exposure to Direct Instruction before student teaching would be one example of how data from student comments was used to initiate a shift in focus in one aspect of a course's content. Another example of data utilization would be changes made to activity assessment rubrics as a result of student comments. As mentioned earlier, student course evaluations were a major factor in the decision to upgrade the School of Education's technology lab.

---

The Assessment System described addresses the candidate, programs and unit. In collaboration with University and K-12 partners, the system is continuously evolving and changing to better measure the knowledge, skills and dispositions as outlined in the Conceptual Framework. Input is sought via surveys, program reviews, general meetings and one-to-one contact to refine the system and ensure it aligns with professional, state and institutional standards.

Data collection, analysis and evaluation have been a continuous process for the SOE. The education faculty have researched and attended training seminars on viable options to bring our students more rapidly into the computer age in providing evidence of their meeting standards as they progress through the SOE. State standards and NCATE accreditation have encouraged movement toward a greater reliance on electronic solutions to data collection, assessment, and documentation of students and programs. The faculty has identified LiveText software as the primary tool for the assessment process.

The timely availability of data through the use of information technology allows the faculty, programs, and the School to use the data for improvement. Faculty have adapted courses and teaching methods in response to regular evaluations by students. Programs review exit and employer surveys to identify strengths and weaknesses of programs and make necessary changes. In addition, data from external sources such as the GPSS are examined to help guide actions needed from program improvement. As assessment is an ongoing process, the SOE will continue to have an Assessment Committee to monitor the system and ensure valid and reliable data are collected and used.