Online Surveys: Completed Reports |
|
|
| |||
| Surveys and Research Home | ||||
TUSD Success+ Summer School ProgramSummary of FindingsA census of teachers and coordinators were surveyed for their opinions and suggestions regarding the 2006 Success+ Summer Program in which they participated. The main goal of the survey was to solicit feedback for improvements to the 2007 summer program. A total of 20 scaled questions, and one open ended comment question were requested of participants. Overall, a vast majority of those participating in the current survey were satisfied with the summer program. What is interesting about the survey results is the overwhelming agreement among participants summary statistics compared with coded comments. The coded comments provide more in-depth descriptions about a particular concern as well as the opportunity for respondents to offer suggestions and recommendations that were not addressed in the scaled questions. By analyzing the summary statistics alone, one could conclude that very little in the program needs improvement next year. However, when participants are solicited for further information, different results are revealed. The following is a list of positive outcomes of the survey and recommendations that might improve and/or enhance the 2007 Success+ Summer Program:
An overwhelming amount of participants either strongly agreed or agreed to the scaled questions asked of them. Comments, however, were very detailed, and provided a wealth of suggestions and recommendations for the 2007 program. While many teachers and coordinators participating in Success+ walked away from the 2006 program feeling like it had positive outcomes on student achievement, the recommendations mentioned above, might strengthen the program even further. IntroductionThe Success+ Summer Program was conducted for its second year during the summer of 2006 for Kindergarten to 8th grade students at 53 schools. Students who participated in the program were selected based on academic measures and teacher recommendations. A total of 5,272 students were enrolled and scheduled in 300 classes numbering no more than 15 students per class. The 20 day intensive program focused on mathematics, language arts, and technology skills. Survey & Data Analysis ProceduresIn an effort to improve the 2007 Success+ Summer Program, organizers of the curriculum requested that a census of teachers and coordinators participate in an anonymous voluntary survey. A total of 20 scaled questions, and one open ended question were requested of participants. Department of Accountability & Research staff conducted the survey for the program organizers between June 26th and July 5th through an automated online survey application [View Online Summary Procedures]. The list of survey participants was provided by the program organizers. Accountability & Research sent an initial invitation email to 392 teachers and 54 coordinators on the morning of June 26th. A reminder email was then sent on the morning of June 29th to those teachers and coordinators who had not yet completed the survey. Once the survey closed Accountability & Research emailed the summary statistics and coded comments to the program organizers. Both quantitative and qualitative results are described below. Data Findings (Statistics and Coded Comments)Out of the total number of teachers (392) invited to participate in the survey, 64% (N=250) responded. Additionally, 91% (N=49/54) of the coordinators participated. The combined response rate of teachers and coordinators was 68%. Because the survey was anonymous, results described below combine responses from both teachers and coordinators. While this combination of responses skews results, it is unavoidable due to how the survey was administered (one set of survey questions was asked of both teachers and coordinators). Questions included on the survey fall into six main categories: training, materials, curriculum, technology, Middle School, and the use of materials in the upcoming year. The following is a summary of findings for each group of questions. TrainingFour questions related to the training teachers received prior to the start of Success+ were included on the survey. Overall, teachers were satisfied with the training. However, the literacy component of the training scored a slightly lower rating than the math. This statistic is supported by the comments provided during the survey [View coded comments]. A total of 36 (17%) individuals commented that the training they received was inadequate. Some reasons mentioned include: disorganization, lack of in-depth information, and missing information (i.e. Language Arts instructor was not available at the scheduled training session). Additionally, teachers felt they could have used a little more time practicing the strategies they were expected to utilize during the program. When elementary teachers were asked to respond to the question: The training sessions provided me adequate information and guidance on how to implement the English Language Development component, over 60% strongly agree and agree to this statement. 
MaterialsA few questions included on the survey solicited feedback on instructional materials provided to teachers for use during the summer program. Overall, the majority of participants thought they had received their teaching materials in a timely manner. In addition, most felt that the instructional literacy and math materials were complete for each day’s lesson (Table 2). However, 17% of the participants who provided a comment on the final question mentioned that they were missing instructional materials. Additionally, a small group of these participants went on to say that they monetarily supplemented the missing materials and/or utilized supplies from their personal classrooms. From the comments, it appears that most of the missing supplies were literacy, and English Language Development instructional materials, teacher and research guides, as well as basic equipment such as scissors, drawing paper, calculators, and materials for specific exercises (i.e. recipes and dream catchers). 
CurriculumWhen survey participants were asked if the level of reading and math materials were appropriate for the majority of students, over 55% agreed (Table 3). The math results, however, varied slightly. The reason for this was revealed in some of the comments. A small percentage of respondents argued that the 6th grade math pacing calendar and curriculum was inappropriate for students. Furthermore, several compared the 6th grade instructional materials with the 7th and 8th and mentioned a disparity between them. Those that compared the two agreed that the 7th and 8th grade math curriculums were not as challenging as they should be, while the 6th grade syllabus was too challenging. In response to the question posed for Elementary teachers only: The level of English Language Development materials was appropriate for the majority of my students, over 55% agreed. Information not revealed in the summary statistics however, surfaced in participants comments to the last survey question. For the 15% (N=31/214) who mentioned English Language Development in their comments, they also stated that the Success+ program needs to be flexible enough to accommodate students who require more time on the literacy portion of the program before moving onto other curriculum items such as math. In addition, this population also mentioned that the literacy portion of Success+ needs improvement. However, this comment is focused once again on the curriculum’s ability to take into account the needs of English Language Learners. 
Two additional curriculum questions were included on the evaluation. These focused on the strategies that made the lessons comprehensible in both literacy and math. Results revealed that an overwhelming amount either strongly agreed or agreed to this statement. Approximately 78% of the population felt that the literacy strategies supported student learning, while 85% believed the strategies in math supported student learning. 
TechnologyThe technology portion of the summer school incorporated a math Study Island, the use of Harcourt literacy software, assessment tests, and group learning into the program. In response to the functionality of computers, over 80% strongly agreed or agreed (Table 5). However, comments revealed that in some cases computers had to be rebooted on a daily basis due to malfunctions. In addition, some participants mentioned that several of the Internet links students were asked to visit did not work. Students would also get distracted with the online advertising on the websites and “pop-ups”. Because of this, it was easy to migrate to inappropriate pages or get lost altogether. One suggestion recommended for next years program, was to assign a technology expert to each school to help with computer problems. When teachers and coordinators were asked to rate the literacy and math technology portions of the program, the majority agreed that the technology component supported student learning. Other comments participants raised were to increase the number of computers for student use, administer assessment tests on paper rather than on the computer (because during the regular school year, assessment tests are not completed online), decrease the number of usernames and passwords assigned to each student to ease the use of the technology, and include more technology training for teachers next year [View coded comments]. Overall, however, teachers seemed to agree that the technology portion of the Success+ program was well implemented, and helped to improve student learning. 
Middle SchoolTwo survey questions targeted Middle School teachers only. All other survey participants were instructed to answer not applicable (NA) [View Summary Statistics]. The survey questions asked if the literacy and math teacher edition books were easy to navigate and whether they provided enough information to effectively deliver the daily lessons. A slightly higher percentage of teachers and coordinators felt that the literacy teacher edition was more effective than the math edition (Table 6). While almost 3% of the population strongly disagreed that the literacy edition was less effective, teachers did not express this strong disagreement when referring to the math edition. 
Use of Materials in the Upcoming YearOne of the most positive outcomes of the survey was the finding that the majority of teachers (84%) that participated in the Success+ program, intend on utilizing the teaching skills and strategies they learned through the participation in this summer school program (Table 7) during the 2006-2007 school year. 
CommentsThe last survey question requested that survey participants provide any comments and/or concerns relevant to the Success+ Summer Program. A total of 214 (72%) individuals that participated in the survey provided a comment on the final question. The majority of these were very detailed and covered a variety of different themes. To account for the variability, 81 unique codes were utilized. These codes were then organized into nine major categories: Curriculum, Technical Suggestions, Positive, Programs and Assessments, Technology, Success+, Leadership and Coordinators, Miscellaneous, and Training (Figure 1). Table 8 below displays the codes attributed to all the comments provided in the evaluation. The most frequently occurring codes are in bold italic text. 
Table 8. Frequency of codes attributed to provided comments.
The most frequently (N=40, 19%) occurring code utilized for participant comments was literacy needs improvement. Some of the comments that were mentioned include: the need to adjust the difficulty of reading materials and pacing to better accommodate the various reading levels of students and the suggestion to alter the literacy curriculum content (i.e. topics read and discussed, linking the daily word list to readers, providing “direct instruction” rather than “centers”, supplying student friendly instructions, etc.). Other comments that fell within this category focused on the writing portion of the program. Several survey participants suggested modifying the existing curriculum to include additional topics for students to write about and refocus writing assignments on more instruction rather than on “busywork”. While only a small percent of survey respondents mentioned literacy needing improvement when compared with the summary statistics, several of those that provided comments, offered valid suggestions and insights into the program. The second most recurring code attributed to comments, raised the issue of allowing more time for lessons. Multiplication arrays, math rotations, expository paragraphs, computer activities/ lessons, sharing written stories with peers, overall processing of concepts, and reading, and writing activities were just some of the exercises teachers and coordinators felt the program should make more time for next year. Two codes, missing necessary materials and training needs improvement were both mentioned equally among survey participants (17%) in their comments. Teachers and coordinators recommend that next year all supplies be checked and verified prior to being delivered to schools. Many had to supplement lessons and activities with their own personal supplies because English Language Development materials, Into English Kits, literacy books (for the younger grades), and general supplies (i.e. drawing paper, scissors, calculators, food for recipes, and materials for art activities) did not make it to classrooms at all, or were two weeks late. In addition to comments related to missing materials, survey participants addressed the training they received prior to the start of the program. The majority of these comments focused on the need to include more technology and language arts training in the future, reduce the length of training sessions, and increase the depth of knowledge relayed to teachers. Some respondents suggested that the training that was provided was “monotonous” and a “waste of time”. To improve future training, some participants recommended making the training more informational or theory and application based. The last comment type of any significance was coded; Success+ needs flexibility in curriculum. A total of 38 individuals (18%) felt this issue should be addressed prior to next year’s program. This code attempted to group all comments related to making the curriculum flexible to accommodate English Language Learners and students requiring extra help in math and/or language arts that are below the level of the summer curriculum. In addition, comments suggested that curriculums need to accommodate the wide mix of student levels that can be found in one class. Participants mentioned that the current curriculums made it difficult to deal with these unique situations effectively. Comments provided during this survey varied widely, and offered a wealth of suggestions and recommendations to improve the Success+ Program next year. Because all the comments can’t be discussed in the present summary, it is suggested that more exploration into the comment portion of the survey be conducted. To view additional comments, visit [View coded comments] for further results. Discussion & RecommendationsOverall, a vast majority of those participating in the current survey were satisfied with the 2006 Success+ Summer Program. What is interesting about the survey results is the overwhelming agreement among participants summary statistics compared with coded comments. The coded comments provide more in-depth descriptions about a particular concern as well as the opportunity for respondents to offer suggestions and recommendations that were not addressed in the scaled questions. By analyzing the summary statistics alone, one could conclude that very little in the Success+ Program needs improvement next year. However, when participants are solicited for further (asked open-ended questions) information, and that information is organized into common themes (codes), different results are revealed. The following is a list of recommendations that might improve and/or enhance the 2007 Success+ Summer Program:
While many teachers and coordinators participating in Success+ have walked away from the 2006 Program feeling like it had positive outcomes on student achievement, the recommendations mentioned above, might strengthen the program even further. Qualitative analysis and Summary Report written by Department Contact InformationOffice of Accountabitity and Research |