Final Reflection

At the beginning of the semester, we were all focused on learning JavaScript on our own as a means to develop the apps we chose to prototype as the semester went on. While each of us is still interested in being better programmers, we felt that our inexperience with javascript was precluding us from doing the other things that we were passionate about, namely education and design. As soon as we put the idea of needing code behind us, we were able to examine these topics more thoroughly. By doing this, we were given the freedom to explore the literature around student motivation and actually try to incorporate what we found into our our designs. We made a lot of significant design decisions that we may have ignored if we had focused on coding. In the beginning stages of developing this prototype, we weren’t sure about giving points or badges for skills learned, but the research showed that this wasn’t a good idea.  

In addition, we also were able to think about the situation in schools and why a piece of technology made specifically for the classroom may not be the best approach to solving problems. This is due namely to the lack of proper integration we saw in the classroom and the amount of bureaucracy involved in getting any technology adopted. Teachers needed someone to go in and explain how to turn on the device, how to make lesson plans involving it, etc. This in and of itself isn’t really taking full advantage of the wonderful things these devices can do. Overall, we found that we weren’t particularly driven to try to make something for the classroom.

For many of us, this was the first really research driven designing we’ve done. We noticed that we have definitely grown as designers in terms of processing information and deciding how to use it. This is an imperative skill for any engineer or designer and we’re proud of the way we were able to incorporate what we learned into our final design. Our coding may not be our strongest front, but we’ve certainly learned about many of the difficulties surrounding making apps. As a whole, we would have appreciated a more balanced team formation. Both of the Javascript teaching assistants were placed in the same group making our group start off far behind and with far fewer resources at our disposal. Overall, we learned a lot about design, education, and the introduction of new technology to the educational world.

Our Final Prototype

As we developed our prototype, we began to wonder what happened between childhood and adolescence that caused people to no longer want to learn and explore the world. The classrooms we visited had kids who were really enjoying reading and doing activities, but we know that motivation is a serious issue in later education. What happened? Over the past couple of decades, there has been a significant shift in what researchers know about the roots of human motivation. The most interesting part about this shift is that it counters a lot of the structures that are most prevalent in our society. For instance, giving someone a reward, such as a high grade or sticker, can sometimes be very counterproductive. This is because it shifts the students motivation. Instead of reading a book because they are interested in what it has to say they instead will read it for the grade. In addition, this typically means that they will read less on their own time and cease to explore for fun. Even though this is extremely well tested and heavily researched there hasn’t been a reaction in most of the school systems. Most of this research was started by Edward L. Deci and Richard M. Ryan. Any of their papers on Self-Determination Theory are a fascinating read and leave the reader wondering why more hasn’t changed. Other readings include Drive, by Daniel H. Pink.

We felt that it would be important to motivate students by structuring our games such that users could track their improvement and thus feel competent. This idea was largely driven by the ideas laid out in a paper on Achievement Motivation Theory by John G. Nicholls, which can be found here. This paper explores how humans naturally attempt to improve mastery if presented with tasks that offer a moderate challenge without salient extrinsic factors. By this token, we wanted to allow students to track their improvement and develop their mastery of the skills intrinsic to these games.

On their own, the example games we’ve selected provide logic skill building and have educational value. However, we wanted to provide users with a learning value apart from the gaming experience itself. One example was choosing to provide ways to further explore topics related to these games without feeling bombarded by excess information. To accomplish this goal, we included links in each of the game’s wireframes to material that would relate to the game in an interesting way and engage the player to learn. Our short descriptions capture topics that relate to the games in ways that we believe will push our players to explore, learn, and even see these games in a new light. We also wanted to incorporate interactive components that would enhance the gaming experience itself. In mastermind, this manifests itself as a count of how many possible codes exist with each new clue throughout the game. In minesweeper, this takes the form of an animation that shows the changing probability distribution for each square on the board. Even without curiosity to explore external resources, we feel that these features would help users become better players and thinkers in a very organic way.

Below are example wireframes that explain how we envision our product and its content. We hope that you’ll note some of the motivational and educational design decisions that we’ve incorporated to scaffold the gaming experience.

Our Mockups

These wireframes include the aspects that we think will be most significant in providing our users with the experience we are trying to deliver. Each of the wireframes includes yellow notes that highlight and describe important features. We chose to focus on developing these designs as mockups rather than using our efforts for developing working prototypes in JavaScript.

Sudoku

Our first wireframe shows the screen on which a student could play Sudoku. The main part of the window on the right side includes the Sudoku game. The player can choose their level and hit the “New Game” button to start a game. From there, they can input numbers in squares. It will appear in black if it is correct, and red to indicate that the input is incorrect.

The left sidebar includes things to enhance the game experience and engage the player. On top, under “Become a Master”, will be a link to a page that details advanced techniques for Sudoku. If a player is interested, they can explore these topics and improve their Sudoku-solving skills. We believe that players will naturally feel more inclined to learn about these techniques once they begin to work up to more difficult levels. Underneath this section will be a statistic on how many squares can be filled in with complete confidence. Then on the very bottom, we include statistics on how the player’s average time to complete their past 5 (or however many games they have played total) games. This will allow them to track how they have improved their solving time as they solve more puzzles.

Minesweeper
Next is our mockup for a Minesweeper game. This wireframe focuses on information without bombardment. The box next to the actual game field shows some variable statistics such as height, width, and number of mines. By varying these, one may vary the complexity of the game. If one were so inclined, it is actually possible to use those numbers to do probability calculations, but by offering those numbers and not forcing people to learn about it, we intend people to learn about the probability more organically.

In addition, the bottom region, labelled (5) here is meant to have some additional content about the game and the mathematics surrounding it. Minesweeper has a lot to do with probability and this section will hopefully include a click through animation with the probability showing. The probability shifts with the amount of information known similar to Bayesian Probability. An example of what this might look like is shown below. NP complete problems are a fascinating area of mathematics and it is our intention that through the guise of minesweeper we may be able to incite people to explore the problems more.

A quick example of the shifting probability is minesweeper. If we say that the

one in the figure to the left is the same as the leftmost one in the figure to the right we can see that by the addition of the new information the probability of each square having a mine shifted.

Mastermind
The final wireframe we created was the game of mastermind, a code breaking game. The game itself would have six possible colors and four color codes, with repeating colors allowed. These are considered standard rules, although many versions have variations in the number of colors or length of the code. The left sidebar would have a rotation between a few facts, including the following:

-there are 1,296 possible codes in this game of MasterMind

-There are only 360 codes where no colors repeat

-Learn more about different secret codes here https://sites.google.com/site/codesforscouts/home

Below the facts is a progress bar underneath a count of how many possible codes are left, which updates for each guess. Players could measure their progress throughout the game and see how many possibilities were eliminated with their guess. Finally, the bottom graph displayed would contain their personal stats over the last 10 games, which would hopefully show improvement. This decision was driven by our motivation research.

Our Second Prototype Idea!

We have developed an idea for a prototype that examines education outside of the classroom. This prototype will allow us to incorporate a few of our other areas of opportunity in a very exciting way by bringing in elements about changing incentive structures, incorporating personal interests, and learning skills versus learning material.

Our idea is a platform that provides users with embedded games and resources to help them learn in a fun and engaging way. Our target audience will be kids in late elementary school or early middle school. The goal of our prototype is not to create new educational content, but rather to collect all kinds of resources that encourage logic, critical thinking, or creativity and deliver them in one central place. We want to avoid games and content that feel explicitly educational -- we’ve seen many variations on a solve this subtraction problem as quickly as you can for points, and see this is a common failure mode. Instead, we want to choose resources that teach more abstract skills, and that are compelling on their own. There is a wealth of games that encourage logical thinking, puzzles that draw on math skills, and fascinating stories with literary merit. We think that uniting it in one place will make for a great user experience.

For a first iteration, we would like our prototype to include a way for users to play Sudoku, Minesweeper, Planarity, and Mastermind. Our desired MVP is a front end that illustrates how we would like users to interact with our platform. This includes how they will navigate the site, play games, and be shown examples of how the skills they are learning can be applied in the real world. Once we reach our MVP, we would also like to include a back end such that an individual can log in and our database can track their progress.

At this point in time we are not sure about what form our incentive structure will take. Given the research done in the last couple decades about the true sources of human motivation, giving points for an activity may not be a good idea. This idea is outlined in Drive by Daniel H. Pink. By adding points for an activity, we may inadvertently turn an activity from something joyful to something that feels like work. In any case, at this point in time we know that we would like to have something resembling incentive to get people interested in new topics, but we are unsure about the right way to do that.

Education Outside the Institution

As a group, we had recently been feeling very discouraged by the School Shaped project. It was disheartening to see how often teachers avoided using technology in the classroom, and to get feedback that solutions that they were most interested in were ones that we couldn’t deliver on. For example, one teacher requested faster wifi which while possible to create is out of our control.  We observed in the classroom the administration taking a top-down approach to new technology in a classroom that typically took too long to introduce and involved insufficient training. Most teachers seemed intimidated enough by it to avoid using it all together. In interviews, one very experienced teacher explained that the only tool she need to rely on to get student attention was snapping her fingers. In general, implementing something in the classroom seems to require battling a ton of inertia in the system. With this in mind, we feel that the most impactful work we can do will not be implemented in classrooms. This led us to explore opportunities to encourage and support children’s learning outside of the educational institution.

There are several possible avenues of exploration for products within this area of opportunity. The first of these is to take subjects that are typically taught in the classroom and try to frame them in more engaging ways. Many of us remember using a scale to understand algebra, but this sort of idea is not always carried over to other subjects. In addition, many countries outside the United States will often try to teach upper level concepts such as those involved in relativity, calculus, or other base systems to children while they are still young without going into details. This means that when children see these concepts later they only need to learn implementation rather than implementation and concepts.

The second vein for education outside the classroom is teaching things that are not explicitly taught in school. This ranges from typical lessons like time management and organization all the way to body language, understanding culture, learning how to learn, and thinking logically and critically. We feel that these are often lessons that people pick up outside of the classroom setting. Since these skills are important for everyone, it is essential that everyone have at least some exposure to them. We have been unable to talk to students yet, so much of the evidence that we are drawing from is what methods we used to learn.

Cycle 2: Areas of Opportunity

Teaching across Cultural Barriers

Last week, we visited King school in Framingham, an elementary school which has many students who are English language learners. We were hoping to observe any nuances around teaching students from different cultural backgrounds. One important observation came during a Mandarin Chinese class. The teacher, who is originally from China, seemed to have trouble maintaining order in the classroom. We wondered how a classroom here would compare to a classroom in China, and what barriers might exist for a teacher or a student coming from a different background. In a later teacher interview, we talked to an English teacher originally from the United States who has taught all over the world, including Saudi Arabia, India, Singapore, and the US. We were interested in exploring in what ways classrooms change across cultures and what tools could lower those barriers.

Kinesthetic learning tools

This area of opportunity stemmed from watching teachers have trouble maintaining order in a large classroom full of very energetic first graders. For many of the children, it was difficult to get them to sit still for even a minute, let alone dedicate any attention to the subject they were learning. There is a mountain of research on kinesthetic learning tools -- toys or devices that allow children to be physically occupied so that they can mentally concentrate. There is evidence that introducing these tools could have huge behavioral impact. Even small changes such as a teacher snapping or clapping rather than speaking or yelling have anecdotally proven to be effective. We thought it could be interesting to explore tools that already exist but have not been introduced to the classroom that might help children concentrate.

Improving Incentive/Punishment Structure

During our recent visit to an elementary school, we realized that the traditional punishment structure is flawed. Traditionally a student that is misbehaving is often separated from the rest of the group and moved to the back of a class for some amount of time to “think about what they have done wrong.” While this may give students the ability to effectively sound apologetic it makes the student miss the lesson being taught to the rest of the class. This puts that student behind his or her peers making it less likely that he or she will be enthusiastic later on. In addition, the current incentive structure does not accurately represent what is currently known about the sources of human motivation. Giving an incentive such as grades or at younger levels, stickers, removes the sense of doing the activity because that activity is interesting and instead makes it about work. If an activity is sufficiently engaging students will learn it without having the joy of learning tempered by the feeling that they are doing work.

Incorporating personal interests

In the vein of the above area of opportunity, incorporating a student’s personal interests is one of the more reliable ways of keeping a student engaged. While visiting an elementary school this past week, we noticed that most students were very engaged in their reading. For instance, many of the students had books about the characters from the Lego world while others were reading about the orbits of planets. It seems obvious, but if there were a way for a teacher to tap into those interests in order to teach about a math concept then many students would find it more engaging. Educators have been attempting to do this for many years, but for the most part it is obvious that the introduced concept is not native to the material.

Learning skills vs material

Many students see school as simply a place they learn material. They don’t recognize the skills they can gain through the work they complete. For example, a student can learn organization and time management, invaluable skills, through the assignments they do in a class. These aren’t things that students can only learn on their own; the development of these skills can be incorporated into a classroom, through the work that is assigned and the environment the teacher creates. We think that more of an emphasis could be put into the incorporation of developing these skills through the material students learn.

Activities that work best for certain students

During our visit to an elementary school, we saw a few activities that didn’t seem to work as well for every student in the room. In one specific activity, two ELL students were playing a game with the teacher while the rest of the class did reading on their own. The teacher would place about ten index cards down on the table, each with a short word on it. She would then say the word, and whichever student pointed to the correct card first won a point. At the end of a round, the student with the most points won. The boy playing the game seemed to really enjoy it; he seemed to be at a level where he could enjoy playing the game, get something out of it, and still try to win. The girl, however, struggled with this game a lot. She panicked each time the teacher said a word and frantically pointed to as many cards as quickly as she could. With the pressure of winning, she wasn’t focusing on recognizing the words and finding the one that the teacher had just said. Because of this, the activity did not seem nearly as valuable or enjoyable to her as it was to the boy. As an observer, it seemed clear that this game only frustrated the girl and made her feel much less competent, but with all the responsibilities the teacher had, it seemed that she didn’t notice that this game was not helping one of the two students. We think it would be very valuable if there were a way for teachers to track how certain activities affect a student’s understanding of the material and improvement. With this knowledge, a teacher could more appropriately and intentionally design activities to fit specific students.

Reflecting Back on our First Cycle

Looking Back

It’s not a secret that our team loves reflection; there’s a reason we focused on student reflection in our first cycle. And given our first prototype, it’s only fitting that we reflect on our own experience as we move into cycle 2.

Although we love our app idea, we feel that we should have better iterated on and developed our idea throughout the cycle. It would have been great to take the idea back to teachers to ask for feedback, or even done an evaluation for ourselves in the middle of the cycle. Instead, we waited until we had some technical results to demonstrate, at which point there was too little time left in the cycle to do more design work.

On the more technical side of our prototype this past cycle, we worked well on bits and pieces of our node app, but never accomplished total integration. We ran into many problems with developing in node because we lacked experience with the tools we were trying to use. Rather than working towards our strengths or genuinely understanding what we needed to do before we approached implementation, we got stuck in the process of using brute force to implement aspects of our prototype.

Moving Ahead

Going forward, the principle goal of our team is to create impact. We would like to use tools that we are more comfortable with as learning a whole new skill in less than three weeks seems unlikely to pan out. Our team is more software centric, so it is likely that whatever route we take will be in the software realm. All of us have taken a software design course in the past, and so have a plethora of experience with python.

One side of this experience we have not yet explored involves designing for people other than teachers. Teachers are perhaps the easiest to design for as they are reasonably easy to access and have a direct interest in student development. However, there are also students, parents, administrators not to mention the rest of society that have some stake in our current education system. Students are difficult to design for as codesign is nearly impossible given a lack of access. Administrators are usually very busy and are hesitant to endorse anything which might not show a direct impact. Parents too, are very difficult to access and are usually very busy. However, it would be great to incorporate the ideas of at least some other stakeholders into our next cycle.

We’re excited for this next cycle and hope all our loyal fans are too!

Our First Prototype

For our first prototype, we wanted to expand on the area of opportunity around student accountability. Based on our own experiences and an interview with Becca, we thought that self reflection would be a powerful tool to promote student accountability. Becca had discussed a time when she asked students to reflect on their first project and then use those reflections to set personal goals for a second project. She saw measurable improvement both in students reaching their personal goals as well as in the quality of their projects. Based on this idea, Becca had proposed a tool that would promote for student reflection, and then return these reflections automatically before the next milestone (for example, at the start of a new project). We chose to expand on this idea for our first design cycle.

Process

The goal for our team’s first prototype took the form of an app that allowed students to submit a reflection and then receive their reflection back via email at a time designated by the teacher; we envisioned that this would be before a major project, test, or evaluation. Both the teacher and the student could read the reflections at any point.

At first, we decided to make a minimum viable product of a submission page for students, backend storage, and teacher’s page to view students’ submissions. The minimum version of the student’s page had a submission window and name area. We decided that this met our minimum deliverable of just making sure that the teacher could get the reflections without having to give out an email address. The backend simply had to store these reflections for the teacher’s evaluation. The minimum version of the teacher’s page displayed the students reflection and near the bottom only displayed reflections which contained certain words as a primary implementation of possible teacher analytics tools. A future iteration would most likely include automation to send the reflections back to students before a major project or at a prompt from the teacher.

At this point in time, the minimum student page and minimum teacher page are complete, however, the implementation of the backend is still unfinished. As the backend is more complicated to set up and our team does not have prior experience in working with backends, this is not unexpected. Once this is complete, the minimum version of our app should be more or less finished.

The work we have completed so far is located at https://github.com/rifkinni/SchoolShaped

Room for improvement

We also have discussed ideas for how we could improve upon this app in a future iteration. One idea that we would like to incorporate is a method for teachers to respond to and/or give feedback on student reflections that are sent to the students along with their initial reflections. If a teacher decides to respond to reflections in this way, they can emphasize a particular aspect of the reflection that they find important for the student moving ahead. Moreover, it can show the students that the teacher finds the reflections and the student’s growth significant. We believe this would push students to take the reflections more seriously, and therefore make our app and these student reflections more impactful.

We have also discussed implementing tools that allow for deeper analysis of the students reflection on the teacher’s side of this app. Possibilities for these tools include a built-in sentiment analysis tool that allows a teacher to see if a student’s reflection is primarily negative or positive.  This could be used to track overall trends in the entire class’s reflections over time.

Bootcamp

During these first few weeks of School Shaped “bootcamp”, we have had several conversations with different teachers about their challenges, successes, and just general approaches in the classroom. The teachers we have spoken with so far have covered a wide range of teaching backgrounds. Yet whether our interviewee taught integrated science at a Montessori middle school or physics at a public high school, we have already begun to see the same themes arise in each teacher’s experience. As we have identified these different themes and challenges that teachers face, we have discussed and developed our observations into areas of opportunity for improvement.

Accountability

One of the topics that naturally came up over and over during our conversations was an idea of student accountability. Not every teacher used the term accountability; one described students’ needs to feel agency, while another used reflection as a way to discuss the need for students to recognize ways they could improve. However, no matter how it was brought into the discussion, each teacher addressed the challenge of helping students feel accountable for their own work, successes, and even failures. We noticed that the two middle school teachers we spoke with put particular emphasis on the importance of their students developing a sense of accountability for their own work.

Consistency

A few teachers observed that students with inconsistent home lives have inconsistent behavior, inconsistent performance, and inconsistent personalities. A consistent educational experience can help support these students by providing them with much needed stability. Even students without serious issues at home can benefit from this type of support. This consistency can take many forms: within a single class, department, or even an entire school. One teacher with whom we spoke appreciated the cohesiveness that cross-disciplinary practices provided to a student. In her school, students practice certain skills such as as frameworks for note taking and scientific writing that are taught and used in multiple contexts. At this stage, we hope to discover more about what inconsistencies in schools can be harmful to a student’s development, and how to rectify these issues and provide stability.

OutcomesOne area of opportunity that is often contested in the more political realm is a definition of success for students. Policy makers and educational administrators make guidelines for schools and curriculum that teachers are often not just a suggestion, but a requirement that they must follow. In turn, these guidelines usually only have one metric for success. They often do not include whether a school is closing its achievement gap and pieces of the more complex puzzle. The teachers we talked to expressed a frustration reconciling the state and federal requirements with their own goals for their students, providing a challenge creating day-to-day lesson plans. The teachers and students would most likely benefit from some way of seeing how they are performing in regards to the entire system and see how far they are progressing regardless of their peers.

Opportunity

Nearly everyone agrees that each student should be given all appropriate opportunity to learn. This opportunity is not the same for every student. If a student has parents who are uneducated or unable to provide help on homework, that student receives less individualized help that can help them overcome a difficult problem or understand it in their own fashion. The ability to learn is not limited to those who are behind their peers, but also those who are ahead. Many teachers offer problems in addition to the standard set that require delving further into a subject. This method allows those students to continue to learn while their peers work to understand an already mastered subject. Some combination of these is already in effect. All of us have seen worksheets with the “extra credit” problems on them which students work on while a teacher walks around helping those that need the most attention. There is a lot more that may be done within this area of opportunity from reaching students at home in a more fulfilling fashion than homework and within the classroom.

Feedback

A theme that has come up during many of our interviews is feedback for both students and teachers. There are many facets of feedback that seem to be a broken part of the system. In one teacher’s school district, feedback from his supervisors came only once per semester and could have an effect on his salary or his opportunities for promotion. He described it as an “I gotcha” moment of sorts, whereas, he wished that the conversation could be more constructive and candid. He believed that more frequent and meaningful feedback from the school’s administration would help him improve his teaching.

Another teacher discussed the role of feedback given to students as a way of improving their learning. In her experience, she saw students improve their academic performance as a response to detailed constructive feedback. She also observed that the amount of feedback she was able to provide for her students was a reflection of her level of investment in their learning. She lamented when she moved to a larger school, where instead of writing detailed feedback for 30 students, she had to settle for very shallow feedback for 150 students. The need for improvement in this area comes in both terms of quantity and quality. For genuine improvement, both students and teachers need to receive constructive and frequent feedback on how they can progress.

Reflecting on Reflection

Accountability is a loaded word in the world of education. While teacher accountability is often held up as the solution to fixing our education system, we will not deal with holding teachers accountable for their students. Rather, we want to encourage students to take a stake in their own education.

The theme of accountability came up in a number of our interviews with teachers. Across the board, educators see it as a critical skill throughout a student’s education and life. One teacher put a focus on reflection as a means for building accountability. According to her, reflection helps students identify where they went wrong and fosters self-improvement. These observations are consistent with research; there is significant evidence that self reflection is linked to more intrinsic motivation and better academic achievement. A different teacher we spoke with gives students the ability to apportion their time as they see fit. This gives students a basic sense of time management as well as task handling.

For our first prototype, we envision a product that creates opportunities for students to reflect on their own progress. This may take several forms, but in the first iteration it will require that students write down what they believe they could have done better and what went well for each unit of material. We would then share this information with teachers so that they could see what each student thought of their own performance and give feedback on habits and skills. With each new unit, our prototype will automatically deliver back the students’ previous reflections at the appropriate time. This will allow students to look over their past reflections when the teacher sees fit, providing the students with an opportunity to remember past lessons and their desired improvements before diving into a similar task. We believe that students should often be given the opportunity to reflect and focus on making those positive changes.

Meet the Team

Madeleine is a junior studying electrical and computer engineering at Olin College with a love for learning and helping others learn as well. She REALLY likes quantum things and impersonating encyclopedias. She has missed throwing javelin and cats while in college (completely unrelated).

Nicole is a senior studying electrical and computer engineering at Olin College. She is passionate about software development, travel, and the intersection of technology and education. Her favorite color is bedazzled.  

Growing up, Robbie spent much of his time in the classroom dreaming up his own lesson plans instead of paying attention to what his teacher’s plan actually was. He hopes to use his passion for education design to help other distracted students overcome their personal obstacles.