Wednesday, March 13, 2013

Assignment #5 - Design and Test a Survey

ECUR 809
Assignment #5 – Design and Test a Short Survey
The survey that I created is intended for the teachers who teach in the Grade Nine Learning Community at my school. It was difficult to create because there will undeniably be bias in their responses; I considered creating one for grade nine students regarding the program, but came to the conclusion that that could be entering a sensitive situation. Most of the teachers in the program have helped to create it, so they would see the changes and the impact on the students more than any other teachers. The new teachers in the program had no knowledge of what the situation was like before the program, but would still have valuable input about it.
My first survey was 7 questions with a mix of rated responses and longer answers. I sent the survey to four people for feedback. I received feedback from all four people, and changed my survey accordingly. I still feel like it could be better – the bias concerns me, although the feedback can still be valuable.
This is the first survey (done and sent through Survey Monkey):
Use link, or see below: http://www.surveymonkey.com/s/KGBC9GJ

Grade Nine Learning Community Survey
 
1. How long have you been teaching in the Grade Nine Program?

2. Do you believe that the Grade Nine Learning Community has improved attendance compared to before the program was implemented?
No, not at all
Somewhat
Yes, there has been improvement
    Yes, significantly





3. Do you feel that the Grade Nine Learning Community helps to build relationships among students more so than without the program?
No, it does not
Somewhat
Yes, for the most part
Yes, significantly


4. Do you find that teaching a variety of subjects instead of one specialized subject is not utilizing your strengths?
Yes
Somewhat
I have learned to adapt
No, it is working successfully


5. Do you believe that the Grade Nine Learning Community has long term benefits?
No
Somewhat
Yes, for the most part
Yes, definitely


6. What do you believe are the greatest strengths of the Grade Nine Learning Community?


7. What do you believe are the disadvantages to Grade Nine Learning Community?



The feedback I received was valuable; I had a feeling that my questions were vague or too open ended but was not sure how to fix them. Here are my revisions:

I added a page at the beginning – the feedback given stated that a purpose for the survey would be beneficial, so I stated who and what the survey is for.

#1 – Stayed the same
#2 – My original question started off with do you believe, and all the feedback stated that belief should not be part of the question – they should rely on stats and evidence. So I changed the question to make it less about opinion.
#3 – I kept this the same, but then added a question after it about relationships with teachers – one person suggested that might be just as relevant in this program.
#4-Kept the same – no feedback was given about it.
#5-I received a number of suggestions on how to improve this question – it was much too general and vague, especially to answer in a rating scale format.  Originally, I asked if the program had long term benefits. The feedback was to be more specific; people were unsure how long term I meant. Benefits was also quite vague. My new question is #6 and asks whether the program benefits students in their later years of high school. I then added one more question about resiliency, as that is something that affects the students long term. I made this question a response based answer, and I felt that the information would be valuable to determine how much benefit the program actually has for students.
#8 and #9 were kept the same, as there was no feedback on these.



My revised survey:
Use link, or see below: http://www.surveymonkey.com/s/KWCCQ5X

The purpose of this survey is for grade nine teachers to reflect on the impact of the Grade Nine Learning Community, and establish if the benefits are worth keeping the program.

Grade Nine Learning Community Survey Revised
 
1. How long have you been teaching in the Grade Nine Program?

2. Has the Grade Nine Learning Community improved attendance compared to before the program was implemented?
No, not at all
Somewhat
Yes, there has been improvement
Yes, significantly


3. Do you feel that the Grade Nine Learning Community helps to build relationships among students more so than without the program?
No, it does not
Somewhat
Yes, for the most part
Yes, significantly


4. Do you feel the Grade Nine Learning Community helps to build lasting relationships between teachers and students more so than without the program?
No, not at all
Somewhat
Yes, for the most part
Yes, significantly


5. Do you find that teaching a variety of subjects instead of one specialized subject is not utilizing your strengths?
Yes
Somewhat
I have learned to adapt
No, it is working successfully


6. Does the Grade Nine Learning Community benefit students in their later years in high school?
No
Somewhat
Yes, for the most part
Yes, significantly


7. How has the Grade Nine Learning Community affected resiliency among students?

8. What do you believe are the greatest strengths of the Grade Nine Learning Community?

9. What do you believe are the disadvantages to Grade Nine Learning Community?












Thursday, February 7, 2013

Assignment #4 - Logic Model

GOAL

OBJECTIVES

ACTIVITIES
OUTPUTS
OUTCOMES

IMPACTS

Statement of the overall purpose of the project
Specific statements of what the project sets out to accomplish
Specific tasks to complete through implementation of the project
Immediate results (direct products of project activities)
Intermediate results (1 to 3 years after project starts)
Long-term results (3 to 10 years after project starts)
To increase achievement  rates in grade 9 students

To improve attendance in grade 9 students.

To build resiliency in grade 9 students by establishing relationships with one main teacher.
Students will attend more regularly because of one teacher tracking them.( 3 year pilot)

Students’ achievement will increase based on improved attendance (3 year pilot)

Students will understand the value of attendance and success and take these skills with them to grade 10 when they are not in specific groupings. (1 year)
Students will feel a sense of responsibility based on relationships established – between students, teachers, and parents.
Establish the program as 3 year pilot project

Identify 5 teachers who are willing to teach all subjects to one group of grade 9 students all day

Organize preliminary reading tests (Gray Silent Reading Test (GSRT)) – to administer in fall

Organize math assessments – to administer in fall

Retests in spring

Create portfolios and evaluation plans for each student

Train all teachers in Assessment for Learning techniques (AFL) with Anne Davies

Track attendance over the 3 year period and compare to attendance for grade 9 students in the past.




Attendance will improve in all classes.

Test scores will improve

Homework will be regularly completed.

Attitudes towards school will improve
Students will feel more of a sense of accountability because there is one teacher who is keeping track of them.

Students will stay enrolled in school.

Students will realize more success in grades 10, 11, and 12

Students will set a goal to graduate

Students will establish relationships with peers and teachers that will assist in their future success.
Students will graduate

Students will seek post-secondary education because they see the value of education

Students will be contributing members of society
Goals and Objectives represent the Strategic Directions of your Project
Activities comprise the Project Work Plan, which should include details for each activity (Who?, What?, When?, Where? How?)
Outputs link directly to project activities; activities are what is done…outputs are the expected results of what is done
Outcomes relate to your objectives; objectives are desired, outcomes are the expected results
Impacts relate to goals; a goal is desired, impacts are the expected end-results















































This logic model outlines information that I have gathered about the program so far. The information mainly came from talking to teachers who have been involved in the process since the initial start up. While this was originally set up as a three year pilot project by the Saskatoon Public School Division, the initial data proved that it was highly successful in retaining kids; consequently, the division cancelled the pilot status and allowed it to continue without parameters from the division. Data was tracked the first year, and the results were encouraging; before the program, failure rates (failing at least on class) were at 50% before the program. Once the program was implemented, this number decreased dramatically. Test scores were also compared from fall and spring and results were positive. There has been no data tracked since the first couple of years, and there is no data to indicate whether the program’s benefits carried on to the following years. There is a concern that while the program shelters the grade 9 students and is overall quite positive for them, the benefits are not carrying through to the remaining years in high school. There is data collected over a four year period, but nothing was done with it, so I do have resources to work with. Teachers in the program and past administration involved in the process are also a wealth of knowledge and are willing to provide any information needed.

Assignment #3 - Evaluation Assessment

Engage Stakeholders

Who should be involved?
 Grade 9 Program teachers, resource teacher, students, parents, school board
How might they be engaged?
By determining if the goals are being met and if the program is more beneficial than detrimental.

          Focus the Evaluation

What are you going to evaluate?  Describe program (logic model).
The Grade Nine Learning Community at Mount Royal Collegiate
What is the purpose of the evaluation?
To determine whether the program is allowing for a smooth transition from grade eight, and helping to retain students.
Who will use the evaluation? How will they use it?
Who/users
How will they use the information?
Program Teachers
They can make changes to the program depending on the results.
School Administrators
Determine whether the program should continue or not.


What questions will the evaluation seek to answer?
Does the program actually help to transition kids from grade eight to grade nine?
How does it prepare them for grade ten if the program is similar to grade eight groupings?
Does it help retain students more effectively than without the program?
Are the relationships authentic – do they continue past grade 9 and consequently lead to success in later years.
What information do you need to answer the questions?
What I wish to know
Indicators – How will I know it?
Attendance data before and after the program was in place
If attendance improved, then the program has some benefit.
Reading and math scores before and after program
If better after, benefit to the program.
Failure rates before and after program.
If students are not failing as many classes as before the program was implemented, then it would indicate merit.
Grade 9 attendance data compared to grade 10 attendance data
If attendance is good in grade 9 and changes significantly in grade 10 it would indicate several things about the program.
When is the evaluation needed?
By the end of the 2012-2013 school year to determine if changes need to be made for next year.

What evaluation design will you use?
A mix of Stake’s Countenance Model, and Scriven’s Summative Model
          Collect the information


What sources of information will you use?
Existing information:
Attendance reports, test results
People:
Teachers in the program, resource teacher, administration,
Pictorial records and observations:
None

What data collection method(s) will you use?
*
Survey

Document review
*
Interview
*
Testimonials

Observation

Expert panel

Group techniques

Simulated problems or situations

Case study

Journal, log, diary
*
Tests

Unobtrusive measures

Photos, videos
Attendance Reports
Other (list)

Instrumentation: What is needed to record the information?
A program to compile the attendance reports to determine if change has occurred.


When will you collect data for each method you’ve chosen?
Method
Before program
During program
Immediately after
Later
Interview/survey past students of the program

*


Interview/survey teachers in the program

*


Attendance reports

*
*


Will a sample be used? 

No


Yes
*
If yes, describe the procedure you will use.
Students in grades 9, 10 and 12 will be used to gain information about their take on the program.

          Analyze and Interpret


How will the data be analyzed?
Data analysis methods:
Analyze the surveys to determine if students find benefit to the program –interview grade 9, 10 and 12 students – students who are in the program, recently out of the program, and students who have had time to reflect on the program.
Teacher responses will also be analyzed to determine how they believe the program is running.
Analyze grade 9 attendance and test results and compare them to the same students in grade ten

Who responsible:
Me


How will the information be interpreted—by whom?
I will interpret the information by compiling answers and looking for consistency



What did you learn? What are the limitations?
Some of the responses could be subjective, from both the teachers in the program, and from the students. There are many variables that can determine a response; grade 9 students may not be the most reflective when it comes to their learning and motivations.
Teachers of the program may be biased because they started it and are so close to it.




               Use the Information

How will the evaluation be communicated and shared?
To whom
When/where/how to present
Teachers of the program
Present reports on data and how students perceive the program and what is happening in reality.
Administration
Present reports on data and how students perceive the program and what is happening in reality.





Next steps?
Show this to the teachers of the Grade 9 Learning Community to determine where the gaps are


          Manage the evaluation                 Standards
Reports will be anonymous
Human subject’s protection

Utility

Management chart

Feasibility

Timeline

Propriety
*
Responsibilities

Accuracy
None Needed
Budget