Program Details:
The Business Office Skills (B.O.S.) program is intended to help low-income students train to obtain an entry-level reception or administrative job by the end of the 12 week course. Students take classes three days a week in Microsoft Office programs and typing efficiency. Students also take part in a job placement twice a week from partner agencies in the Toronto area.
The program is free for students and they receive two college credits from Centennial College. New Circles provides a computer laboratory for students to practice their typing and Microsoft Office Skills. For the best opportunity to succeed, the class size is limited to 15 students.
Two programs run each year, one begins in September (12 weeks), and the other begins at the end of January (12 weeks). Students must have a moderate ability to comprehend and speak English as the course is very short and focuses on computer programming and typing skills.
For more details, click here: newcircles.ca/services/skills-training/office
The program is free for students and they receive two college credits from Centennial College. New Circles provides a computer laboratory for students to practice their typing and Microsoft Office Skills. For the best opportunity to succeed, the class size is limited to 15 students.
Two programs run each year, one begins in September (12 weeks), and the other begins at the end of January (12 weeks). Students must have a moderate ability to comprehend and speak English as the course is very short and focuses on computer programming and typing skills.
For more details, click here: newcircles.ca/services/skills-training/office
Program FocusThe main focus of the program is to offer low-income students a chance at social inclusion and economic stability.
New Circles, in partnership with Centennial College, is offering a program that will help students improve typing speed, Microsoft Office skills, and other work place skills. |
|
|
Program GoalsThere are a few goals of this program and the main one being to get students jobs in the office administration field by the end of the 12-week course. Another goal of the program besides learning computer skills, is to to increase students' confidence and self-worth. An unintended, yet welcome goal is to have students become life-long learners and want to continue their learning journey into more programs at Cetennial College or other colleges and universities.
|
Size of Staff
The team consists of one ongoing teacher that the students meet three times a week, for twelve weeks and two additional guest speakers/mentors. Two of these mentors are teachers from Centennial College. One is Terry Pruner who teaches in the Office Administration program and can help the students with Office procedures training and customer service skills. The other is Sana Sagar who can help students with their typing skills and grammar proficiency.
Resources
Computers:
Due to the computer lab at New Circles Community Center only having 15 desktop computers, the class size needs to be limited to 15 so that each student gets their own. Additionally, students also need to borrow laptops as they most likely do not have their own computers at home to complete their assignments.
Teachers:
One full time teacher and 2 guest speakers who will rotate bi-weekly to check in on student projects.
Funding:
To pay for resources, New Circles relies on local donations and government funding to pay for placement opportunities and to provide students with two, free college courses.
Due to the computer lab at New Circles Community Center only having 15 desktop computers, the class size needs to be limited to 15 so that each student gets their own. Additionally, students also need to borrow laptops as they most likely do not have their own computers at home to complete their assignments.
Teachers:
One full time teacher and 2 guest speakers who will rotate bi-weekly to check in on student projects.
Funding:
To pay for resources, New Circles relies on local donations and government funding to pay for placement opportunities and to provide students with two, free college courses.
Community Demographics
The program is run in a low-income area in Scarborough Ontario and the students vary from recent immigrants/refugees to students who have live in the area their whole lives. The program is open to anyone in the nearby community. All ages are welcome and the program receives applications from students who vary greatly in age.
Evaluation Questions:
- Are the skills taught in the program likely to help students get a job?
- What skills can be added to the program to align them to skills needed in the industry?
- Should the program run with no cost to the students?
- What other resources (besides donations and government funding) are available to help fund the program?
- How many students obtained an office administration job shortly after or during the course?
- If students were successful/unsuccessful in obtaining a job, how much of that can be attributed to what they learned in the program or to the state of the economy?
- Are employers satisfied with the students as new employees?
- Were the skills taught relevant to the job placement and how can the program closely align the class experience to the placement experience?
- How well did the Microsoft training prepare students for the workplace. Are there any specific skills the program should focus on (example: V-lookup tables in Excel)?
- What elements of the course could be improved to meet the goals of the program?
Program Theory
The purpose of this program is to have students develop skills that will help them gain employment in an office administration career.
SWOT Analysis:
Creating a SWOT analysis could help define the program theory and focus the evaluation questions of the B.O.S. program. In the video below I will demonstrate how that could work:
Theory of Change Logic Model
Students will be taught typing, Microsoft Office, and office procedures to increase their chances of obtaining a career in an administrative role. The program theory of change for this program evaluation contends that learning the skills that are taught throughout the course will lead to the desired outcome of students succeeding in their efforts to obtain a career in office administration.
Evaluation Approach:
Formative Evaluation: Theory-Driven Process Evaluation
Theory-driven process evaluation will be used in at mature implementation stage of the evaluation. As Chen (2005) identifies, this evaluation strategy will benefit the program by ensuring accountability and give stakeholders a clear idea on how to improve the program. Accountability is important at every step of the way through the program evaluation and could be another way to ensure evaluation use. The table below can serve as a theory driven process evaluation tracking sheet.
With different organizations who are not always on hand, everyone has to be accountable. If Centennial College, for example, fails to provide adequate guest speakers, the student's skills could fail to develop. If this was happening, it would be identified and changes could be made during the program to fix the problem.
Huey-Tsyh, C. (2005). Assessing Implementation in the Mature Implementation Stage . Thousand Oaks, CA: Sage Publications
Huey-Tsyh, C. (2005). Assessing Implementation in the Mature Implementation Stage . Thousand Oaks, CA: Sage Publications
Summative Evaluation: Outcome Monitoring
The stakeholders of this program at Centennial College and at New Circles are looking to save money as the program already has a few expenses (paying teachers, guest speakers, and keeping textbooks up to date) and offer the program free to students. To keep the program free to students, a cost-effective evaluation strategy such as Outcome Monitoring must be taken. This evaluation approach is less rigorous than other methods of outcome evaluation but is shorter and less expensive as well. The program can greatly benefit from an evaluator's feedback of the program and input on how to improve it to ensure the success of the program. This approach will monitor the students’ performance once before the program and once after to see if the program had an effect on students obtaining and maintaining a career job in reception or office administration. This means that data and information about the students will be collected once before and once after the program.
In the context of the program, students will be asked to fill out a survey that will give an evaluator information about themselves and how they view their chances of achieving a career in office administration. A similar survey will be given to the same students and will ask if they have had success in achieving a career in an office job, and whether they feel confident with the skills they have to maintain that job. Data on employer satisfaction will also been gathered through a survey as well.
This approach is a great fit for the program because it is cost-effective and is focused on the goals of the program-student success in obtaining a promising career.
Huey-Tsyh, C. (2005). Program Monitoring and Outcome Evaluation. Thousand Oaks, CA: Sage Publications
In the context of the program, students will be asked to fill out a survey that will give an evaluator information about themselves and how they view their chances of achieving a career in office administration. A similar survey will be given to the same students and will ask if they have had success in achieving a career in an office job, and whether they feel confident with the skills they have to maintain that job. Data on employer satisfaction will also been gathered through a survey as well.
This approach is a great fit for the program because it is cost-effective and is focused on the goals of the program-student success in obtaining a promising career.
Huey-Tsyh, C. (2005). Program Monitoring and Outcome Evaluation. Thousand Oaks, CA: Sage Publications
Due to the small number of students in this program (12-15), Qualitative data in the form of observations, journals, interviews, and focus groups will be used to evaluate the B.O.S. program. Below is an outline of the significant data collection points in the program evaluation.
Analysis of this data will consist of finding current themes and keywords in student, faculty, and placement mentor surveys. This will allow the evaluator to investigate the strengths and weaknesses from a range of stakeholders. The evaluator will examine responses and group similar themes to develop a narrative from multiple perspectives.
Analysis of this data will consist of finding current themes and keywords in student, faculty, and placement mentor surveys. This will allow the evaluator to investigate the strengths and weaknesses from a range of stakeholders. The evaluator will examine responses and group similar themes to develop a narrative from multiple perspectives.
Data Collection: Students will be interviewed before the program begins. At this point, they will be interviewed to create a student profile that will outline their income, background, computer literacy, and other unique experiences and qualities that the students possess. Students will also complete a survey to gather more personal information about themselves as a student and employee. Students will self assess for things like learning styles, assessment preferences, and current motivation levels.
Analysis: The evaluator will compile student responses on the interview and survey a few weeks before the program begins to create a unique profile for every student. These profiles will be discussed with, and given to a teacher before the program begins to see how the learning activities can be tailored to the group and individuals in the classroom.
Analysis: The evaluator will compile student responses on the interview and survey a few weeks before the program begins to create a unique profile for every student. These profiles will be discussed with, and given to a teacher before the program begins to see how the learning activities can be tailored to the group and individuals in the classroom.
.
Data-Collection: Half-way through the semester, students will complete a survey that will ask for their opinions in three categories that focus on the quality of the program, how the skills and lectures are being delivered, and how the student is coping with the class schedule and placement opportunity.
The teacher will also be asked to keep a running journal to note observations about each student. The teacher will consider if the students are developing computer skills, increasing their typing speed, collaborating with others, and completing assignments on time.
Analysis: Student responses will be assessed to identify whether the program is meeting its goals, and whether things can be implemented to improve the student experience and the likelihood for career success.
Weekly teacher observations and notes will be examined by the evaluator to see if improvements to the program need to be made to better assist each student and to see if students are meeting the requirements of the program.
The teacher will also be asked to keep a running journal to note observations about each student. The teacher will consider if the students are developing computer skills, increasing their typing speed, collaborating with others, and completing assignments on time.
Analysis: Student responses will be assessed to identify whether the program is meeting its goals, and whether things can be implemented to improve the student experience and the likelihood for career success.
Weekly teacher observations and notes will be examined by the evaluator to see if improvements to the program need to be made to better assist each student and to see if students are meeting the requirements of the program.
Data Collection: Placement mentors will be asked to fill out a survey to rate how satisfied they were with the student as an employee and send it to be analyzed be the teacher and program evaluator. Samples of student work in the job placement will also be collected if it is available.
Analysis: The evaluator will meet with the teacher to go over the feedback that they get from the placement mentors. At this point, the evaluator should examine whether the program is helping students prepare for the workforce and what can be done to get students ready to obtain and maintain a career position in an administrative role. Evidence of student work in the job placement will be asked for to determine whether students were able to apply skills they learned in class to the real word.
Analysis: The evaluator will meet with the teacher to go over the feedback that they get from the placement mentors. At this point, the evaluator should examine whether the program is helping students prepare for the workforce and what can be done to get students ready to obtain and maintain a career position in an administrative role. Evidence of student work in the job placement will be asked for to determine whether students were able to apply skills they learned in class to the real word.
Data Collection: Students who have graduated from the program will be asked to join a focus group and will be interviewed individually at six months after they have completed the program and also one year after they graduated from the program. Focus group interviews will also be conducted with placement mentors, program coordinators, and teachers at the end of the program
Analysis: The first focus group that includes students will examine whether the program is successful in helping students obtain a career. Examining student feedback should also reveal the relevancy of the skills taught in the program. The second focus group that involves teachers, program mentors and coordinators will determine if the skills taught in the program aligned with skills needed in the industry. This focus group will also address if certain aspects of the program need to be improved for the next group of students. Keywords and major themes will be gathered from the interviews and surveys.
Analysis: The first focus group that includes students will examine whether the program is successful in helping students obtain a career. Examining student feedback should also reveal the relevancy of the skills taught in the program. The second focus group that involves teachers, program mentors and coordinators will determine if the skills taught in the program aligned with skills needed in the industry. This focus group will also address if certain aspects of the program need to be improved for the next group of students. Keywords and major themes will be gathered from the interviews and surveys.
Infographic provided by:
Approach to Enhance Use
Process Use: The B.O.S. program is a new program and will involve process use to enhance evaluation use. Dr. Michael Patton (2008) described process use as "changes resulting from engagement in the evaluation process and learning to think evaluatively". Process use happens when those involved in the program learn more from the process of having an evaluation occur than from the findings after an evaluation is complete. Program stakeholders in the B.O.S. program will be encouraged to participate in the implementation and planning of the program. The teacher will also be involved in the evaluation process as he/she will be asked to keep a journal of findings and observations during the class that will be later discussed and examined with the program evaluator. This will help everyone involved in the program to decide how the program should be run now and in the future. By having the staff and teachers involved in the process of evaluation through focus group interviews, surveys, and meetings, the evaluator can enhance program use.
The evaluator will create what Baron (2013) describes as a "culture of assessment" among teachers, placement mentors, and program managers. The evaluators can facilitate lessons to the placement mentors, teachers and program managers on how to assess, take effective anecdotal notes, and examine if useful skills are being developed throughout the process. It is important to note, that the evaluator will examine the teacher's progress throughout the program to maintain a culture of assessment for the people who are delivering the program.
Using theory-driven process evaluation, mentioned above, can also ensure that everyone involved in the program is accountable to deliver what they were intended to deliver.
Baron, Michelle. AEA 365 A Tip-a-Day by and for Evaluators: Michelle Baron on Building a Culture of Assessment. 15 January, 2013. Date Accessed: 15 November, 2017.
Patton, M.Q. (2008). Utilization-focused evaluation, 4th edition. Thousand Oaks, CA: Sage.
The evaluator will create what Baron (2013) describes as a "culture of assessment" among teachers, placement mentors, and program managers. The evaluators can facilitate lessons to the placement mentors, teachers and program managers on how to assess, take effective anecdotal notes, and examine if useful skills are being developed throughout the process. It is important to note, that the evaluator will examine the teacher's progress throughout the program to maintain a culture of assessment for the people who are delivering the program.
Using theory-driven process evaluation, mentioned above, can also ensure that everyone involved in the program is accountable to deliver what they were intended to deliver.
Baron, Michelle. AEA 365 A Tip-a-Day by and for Evaluators: Michelle Baron on Building a Culture of Assessment. 15 January, 2013. Date Accessed: 15 November, 2017.
Patton, M.Q. (2008). Utilization-focused evaluation, 4th edition. Thousand Oaks, CA: Sage.
Standards for Program Evaluation
The evaluation plan for the B.O.S. program meets all of the following standards for program evaluation: Utility standards, feasibility standards, proprietary standards, and accuracy standards
Utility Standards: The utility standards are intended to ensure that an evaluation will serve the information needs of intended users.
The stakeholders in this program the students, teachers, and program coordinators are clearly identified and a needs assessment will be conducted to ensure their needs are being met throughout the program evaluation. An experienced professional in the field of evaluation will need to be hired as a credible source to ensure accuracy and validity of evaluation findings. Information about the program is described in this evaluation plan, including the program goals, context, and purpose so that this information is understood by stakeholders and the program evaluator. As stated above, the program design encourages participation from all stakeholders in an effort to ensure that the program evaluation is used.
Feasibility Standards: The feasibility standards are intended to ensure that an evaluation will be realistic, prudent, diplomatic, and frugal.
This program is cost-effective in the way that data is collected as most of the qualitative data is produced from surveys (online or on paper), focus groups, and observations from teachers and program coordinators. The program is practical as students attend a three-hour class, three times a week, and participate in voluntary job placement only twice a week from 9 a.m. to 2 p.m. which allows them to balance the program, work, and family obligations. The program evaluation can be completed without disrupting the students' schedules.
Propriety Standards: The propriety standards are intended to ensure that an evaluation will be conducted legally, ethically, and with due regard for the welfare of those involved in the evaluation, as well as those affected by its results.
The program evaluation is intended to serve students as methods and implementation will be examined to identify ways to ensure that every student is provided with skills that will need to obtain financial stability. The students will be protected as their identities will not be revealed when responding on surveys, focus group interviews, and observations in the classroom. Students and other stakeholders will be debriefed about information that was found to provide insights for those affected by the program evaluation.
Accuracy Standards: The accuracy standards are intended to ensure that an evaluation will reveal and convey technically adequate information about the features that determine worth or merit of the program being evaluated.
The evaluator will facilitate ongoing training to program coordinators and teachers to ensure that evaluations and observations about the program are accurate and detailed. Based on the program evaluation plan that is outline above, the evaluation questions, data collections methods, and program goals can illustrate the worth of the B.O.S. program evaluation.
