Paper ID #22675Design, Implementation, and Assessment of a Summer Pre-collegiate Program at Harvard School of Engineering and Applied Sciences (SEAS)Dr. Anas Chalah, Harvard UniversityDr. Anas Chalah Assistant Dean for Teaching and Learning, Lecturer on Engineering Sciences, Directorof Lab Safety Program at Harvard University - John A. Paulson School of Engineering and AppliedSciencePierce Hall G2A, 29 Oxford Street Cambridge, MA 02138 (617)-495-8991 [email protected] Fawwaz Habbal, Harvard UniversityFawwaz Habbal has served as the Executive Dean for the Harvard School of Engineering and AppliedSciences (SEAS) from 2007 to present. He is also a Senior Lecturer in Applied Physics at SEAS. Priorto Harvard, he held the position as Corporate Vice President, responsible for research and product designat Polaroid Corporation where he served as a Senior Research and Engineering Fellow as well. Afterleaving this position he initiated two start-ups related to imaging. Dr. Habbal’s research interests focus onsuperconductivity, magnetic materials, silicon nanowires for photon detection and nano-photonics morebroadly.Michael Raspuzzi, Harvard UniversityMichael Raspuzzi is a second year Master in Design Engineering student at Harvard John A. PaulsonSchool of Engineering and Applied Sciences and Graduate School of Design. His most recent teachingroles involve instructing in innovation and entrepreneurship summer classes at SEAS as well as in theCollaborative Design Engineering core studio. His former roles include managing director of Life Changing Labs at Cornell University, founder of LCL’s summer startup incubator, founder of LCS’s global highschool entrepreneurship and computer science program, and director of the Caldwell House.c American Society for Engineering Education, 2018

Work in Progress: Design, Implementation, and Assessment of aSummer Pre-Collegiate Program at Harvard John A. Paulson Schoolof Engineering and Applied SciencesAnas Chalah, Michael Raspuzzi, and Fawwaz HabbalPre-Collegiate Program Background and Direct ObjectiveAs new experiments and design-based projects are envisioned, they must be evaluated andassessed before they become part of the curriculum. Initially, we introduced such new items to asmall group of the Harvard College students, but as the demands for new experiments increased,it became difficult to have a thorough evaluation through the small sample of students. Wedecided to engage a different student body of diverse backgrounds by establishing a precollegiate program. This program attempts to prototype and develop multiple new activelearning initiatives before integrating them into the full curriculum. Indeed, the program becamea valuable platform to develop, to experiment, and to evaluate new exercises, which has beenshown to help increase interest in engineering professions [1]. The outcome of the yearlyprogram helped modify and enhance our formal offering for the college students.Program StructureThis program is structured as a pilot for curriculum development and is designed with flexibilityin mind to create a cohesive cohort through team-based learning. It aims to offer our teachingstaff the ability to select the topics they aim to pilot and test during the summer before they areimplemented in our school curriculum. While topics may change in different years, the generaloutcome continues to be a rich selection of multiple engineering and applied sciences topics thatbecome available for the summer pre-collegiate students. Written text and backgroundinformation are provided to the students prior to the start of the program on the selected topic inpreparation to have them immersed in a new pedagogy with different learning outcomes thantheir previous learning environments. The topics differ between years, and the programsemphasizes different developing fields of technology as well as emerging humanity challenges.Students from all different backgrounds are invited to participate (Figure 1) in these exercises.Figure 1: The students’ diverse interests collected in pre-program assessment and in round tablediscussion on the first day of class with corresponding number of responses for each subjectarea. Black represents the sciences and physics, dark gray represents computer science andengineering, and the light gray represents arts and humanities:1. Computer Science (23); 2. Physics (10); 3. History (8); 4. Art (6); 5. Chemistry (5); 6. Music(4); 7. Math (4); 8. Genetic Engineering (4); 9. Economics (4); 10. Software Engineering (3);11. Psychology (3); 12. Biology (3); 13. Game Design (2); 14. Literature (2); 15. Linguistics(2); 16. Information Technology (2); 17. Graphic Design (2); 18. Environment (2); 19.Engineering (2); 20. Education (2); 21. Aerospace (2); 22. Architecture (2); 23. Philosophy (1);24. Nuclear Physics (1); 25. Molecular Biology (1); 26. Medicine (1); 27. MechanicalEngineering (1); 28. Material Science (1); 29. Law (1); 30. Electrical Engineering (1); 31.Geography (1); 32. Design Engineering (1); 33. Dance (1)

Program Elements and Students’ Experiential LearningHarvard John A. Paulson School of Engineering and Applied Sciences (SEAS) undergraduateengineering curriculum is embedded within the traditional liberal arts education of HarvardFaculty of Arts and Sciences. Typically, students tend to take several courses outside ofengineering, including general education, humanities and social sciences. The liberal artseducation provides an interesting background that allowed us to offer complementary coursesthat enable developing and implementing proposals for mitigations to real challenges facingsociety. In addition, for the students to be effective in making their proposals, it is necessary totrain them on how to translate their theoretical learning to practical experiences. Thus, many ofSEAS courses have experiential components, some imbedded at the classroom, and others arepracticed at the teaching labs.Since our goal is to use the experiences of the pre-collegiate program as a guide to enhance thepractical experiences of Harvard College students, the pre-collegiate curriculum had to bemodified yearly to match the evolution of the new engineering courses at SEAS. Thus, throughthe pre-collegiate curriculum, we were able to test some of our hypotheses and the effectivenessof our pedagogy. First, we tested the interdisciplinarity of some of our courses, which isimportant for us as many of our courses are geared for a wide range of student concentrations,including, ME, EE, Bioengineering and Environmental Engineering, and students take thesecourses together and work collaboratively regardless of their concentration.Our program focuses on teaching systems analysis that emphasizes the importance ofinterdisciplinary collaborations in solving complex humanity challenges. Secondly, somestudents want to bring forward their innovative ideas to the commercialization stage, and wewant to support their aspirations. Thus, we included in the pre-collegiate curriculum someaspects of innovation and entrepreneurship. Thirdly, we consider design thinking as an importantenabler of innovation. Design thinking is an iterative and interdisciplinary collaborative processthrough which students are able to exercise and practice different types of thinking, includingdivergent, convergent, critical, analytical, and integrative thinking.Teaching DynamicsThe teaching methodology for the program assumes that students have no prior knowledge inany particular subject area, but through the workshops, mentorship, and the hands on activitiesoutlined in figure 2, they learn quickly. Two of the most important aspects of the curriculum,prior to their enrollment in the lab, include learning design methodologies (in Figure 2, 3.1-3.3)and understanding systems and systems analysis (in Figure 4.1-4.3). Through the design process,students can translate the technical and theoretical aspects of the curriculum and convert theminto built objects and working prototypes, and iterate on their work. After introducing thedefinition of design, students begin thinking through doing; starting with simple exercises suchas using marshmallow and spaghetti to build towers. Such challenges take only ten minutes,during which, groups build the largest freestanding structures. This gives students the chance to2

experience the nonlinear design process firsthand, which provides a foundation for introducingthe next steps of empathy, definition, ideation, prototyping, and testing. Once the process isdiscussed, the exercise of the spaghetti tower is introduced again with a new constraint of havingto hold the weight of a lime.WEEK ONEMondayTuesdayWednesdayThursdayFridaySession One10:00am 11:30am1.1Introduction tothe Program2.1Engineering andScience3.1Introduction toDesign Thinking4.1Introduction toSystems Design5.1Introduction toDrone TechnologySession Two1:00pm 2:00pm1.2Introduction toYYY ActiveLearning2.2Introduction toEntrepreneurship3.2Rapid PrototypingExercise4.2Learning AboutSystems Control5.2Learn and AssembleDrone Kit Exercise3.3Design Exercise4.3Learning AboutSystems Control5.3Learn and AssembleDrone Kit ExerciseSessionThree2:30pm 4:00pm1.3Team Forming2.3ExerciseLab Safety TrainingWEEK TWOMondayTuesdayWednesdayThursdayFridaySession One10:00am 11:30amLab 1: DesignDayLab 2: FabricationDayLab 3: Test andAnalysis DayLab 4: CompetitionDayLab 5: FinalPresentationSession Two1:00pm 2:00pmLab 1: DesignDayLab 2: FabricationDayLab 3: Test andAnalysis DayLab 4: CompetitionDayLab 5: FinalPresentationSessionThree2:30pm 4:00pmLab 1: DesignDayLab 2: FabricationDayLab 3: Test andAnalysis DayLab 4: CompetitionDayFigure 2: Week 1 and Week 2 daily schedule for the program.By going through the process again with the same team members, the students can apply whatthey have learned in the first experience and repeat the process. Following the second challenge,a longer discussion of different ways of prototyping with different materials takes place with ademonstration. And finally, the third module brings this learning together with a user interviewand brainstorming for students to use the systemized approach to ideate and prototype.A number of instructional modules were developed for the program. Chronologically, thesemodules include introductions to the several concepts, such as differences between science andengineering, innovation and relationships to entrepreneurship, rapid prototyping, and theoreticaland practical systems design. Following the content, we introduce exercises for team formation,understanding systems dynamics, and learning about the technology that is provided in thedrone kits.3

Week two of the program was structured to allow students to use their leaned skills, with toolsand techniques to complete a design project. Project topics vary each year. This year’s projectresembled a rescue mission to provide critical aid to victims of a recent earthquake. Studentswere tasked with designing solutions for rapid assistance by air for a disaster area in a remotelocation. To evaluate their design, the final design competition consisted of students utilizingbasic materials and rudimentary prototyping techniques to modify the framework of an existingquadcopter kit. Their design was intended to safely carry an egg payload to a designated landingzone while minimizing weight and maintaining maneuverability of their designed device. Alongwith the challenge of balancing those constraints in the complex quadcopter system, students hadto work alongside a multicultural team and work within a tight time constraint.Program Assessment and EvaluationAssessment was integrated into the pre-collegiate program to evaluate the three areas outlined inASEE guidelines [2]: a) student inputs, b) student outcomes, and c) the educational environment.As they entered, students were given a baseline assessment upon arrival to measure inputs, andas the program began, topic based assessments were given to students at the conclusion of eachworkshop to measure educational environments (Figure 3).Figure 3: The students’ average confidence levels collected from a daily survey in the selected areas of Applied Math, Computer Science,Engineering, Systems, Robotics, Design Thinking, and Prototyping throughout the program. Assessments included: gathering information onstudents’ exposure to and familiarity with topics prior to the start of the program, self-evaluations of the effectiveness of each segment andevaluations of teaching methods- the group dynamic, project based learning and comprehension of foundational concepts being introduced.Confidence levels were used as a formative metric to measure a student’s self-perceived abilityto measure the outcomes of each workshop analyzing the component level of the program [3,5].Instructors held discussion-based assessments at the conclusion of each day, and incorporated4

some of the most actionable feedback in real time into the teaching activities for that week. Wedo not conduct proficiency testing during this program. However, once the tested hands-onexercises are implemented into the curriculum, student’s proficiency is measured through avariety of course-based testing and evaluation method [4].Reflection on Students’ Level of ConfidenceWe focused on short term and long-term effects of the program on the students’ learning: shortterm defined as what happens during the program from day to day and week to week.Assessments on the longer term are defined as the inferred change from the pre-term and postterm assessment evaluation. We note that within the program, the first major increase in selfdetermined level of confidence for students was between their introduction to the program andthe completion on the first workshop on that specific subject. The average increase (Figure 4)from pre-term assessment to daily assessment after the respective workshop was 2.68 forEngineering, 3.09 Systems, 3.23 for Robotics, 3.00 Design Thinking, and 3.23 Prototyping.Figure 4: The average confidence level at each stage of evaluation for different Engineering and Applied Sciences topics.Applied Math and Computer Science were excluded because there were no dedicated workshopsfor those subject areas. This increase shows a raise in self-perceived confidence directly afterbeing taught the specific subject. As revealed in Figure 2, this single highest point of confidenceduring the workshop period decreases as students enter the lab days between day 6 and 7, wherethey were engaged in multiple subjects at once and surveyed thereafter. We are looking tovalidate this over multiple iterations of the program as currently this data is from one programwith n 22 students sampled. Our data and in-class observation show that students start with ahigh degree of confidence and then realize that their knowledge is lacking. In fact, we identify ageneral trend of decreased self-reported confidence levels as students progressed from week oneof skill-learning and the hands-on workshops, into week two of design-based open-endedcompetition days. Students begin asking in-depth questions as they obtain sufficient skillsets,5

tool and techniques. This normally takes place in week one workshops and toward thecompletion of design projects. We believe students who transition from knowledge gatheringinto skillset practice gain a realistic appreciation of their ability to analyze, evaluate, and applytheir expertise toward solving open-ended challenges, thus their reported level of confidenceincreases (Figure 3).Program Conclusion and Future PlanningTo incorporate active learning in our curriculum, we established state-of-the-arts teachingfacilities and infrastructure. We also created a human organization to enable the students andrespond to their specific needs. The active learning part of the curriculum is a result of a closeand tight collaboration between the faculty and the teaching staff, which led to a rapid increase inthe active learning activities over the past seven years. Piloting such new activities over thecourse of pre-collegiate programs has allowed us to effectively examine, revise, and implementdifferent active learning components that are well-aligned with the curriculum. This pilotingmechanism ensures a thorough testing of the new activities and provides us with detailed anduseful student feedback to refine our teaching skills and further develop learning exercises beforethey are fully implemented in our curriculum.The success of this pilot program depends on the effective transition from conceptual andtheoretical frameworks into applied hands-on experiences. As students were tasked to apply theknowledge they gained from classroom lectures in the lab section, the students were severelychallenged. This indicates that their own appreciation of the depth of their knowledge was not assolid as it can be. Furthermore, we observed a consistent increase in students’ level of confidencethroughout the second half of the program as prototypes were finished, tested, and measured. Weintend to continue using the SEAS Pre-Collegiate Program as a valuable educational test bed.Our future focus is to further improve the confidence level of enrolled students as they engage inthe designed workshop and the subsequent active learning exercises.References1., Shawna L., et al. "The WISE summer bridge program: Assessing student attrition, retention, andprogram effectiveness." Proceedings, American Society for Engineering Education. 2001.“Excerpts from a Working: Draft a Framework for the Assessment of Engineering Education.” ASEEPrism, vol. 5, no. 9, 1996, pp. 18–26.Hannan, J., et al. “An Engineering Design Summer Camp for a Diverse Group of High School Students.”Frontiers in Education Conference, 1997. 27th Annual Conference. Teaching and Learning in an Era ofChange. Proceedings., vol. 2, 1997, pp. 939–943.Purzer, Senay, Nicholas Fila, and Kavin Nataraja. "Evaluation of Current Assessment Methods inEngineering Entrepreneurship Education." Advances in Engineering Education 5.1 (2016): n1.Stevens, Floraline. "User-Friendly Handbook for Project Evaluation: Science, Mathematics, Engineering,and Technology Education." (1993).6

Program Structure , This program is structured as a pilot for curriculum development and is designed with flexibility in mind to create a cohesive cohort through team-based learning. It aims to offer our teaching staff the ability to select the topics they aim to pilot and test during the summer before they are implemented in our school curriculum.