How computing instructors plan to adapt to ChatGPT, GitHub Copilot, and other AI coding assistants (ICER 2023 paper): Guest blog post from Philip Guo
Hi everyone! This is Philip Guo from UC San Diego. My Ph.D. student (and soon-to-be faculty colleague) Sam Lau and I are presenting a paper at ICER 2023 on a topic that’s been at the top of many of our colleagues’ minds in recent months:
How are instructors planning to adapt their programming-related courses as more and more students start using AI coding assistance tools such as ChatGPT and GitHub Copilot?
A series of studies throughout 2021 and 2022 showed that GitHub Copilot can solve many kinds of CS1 and CS2 homework problems (see Section 3 of our paper for a summary of these studies). However, Copilot still requires users to install, configure, and activate it within an IDE like Visual Studio Code, which can be hard for novices to do. So when ChatGPT launched at the end of 2022, it made this AI technology far more accessible and easier to use. Now any student can visit the ChatGPT website, copy-paste in their homework assignments and instructor-provided starter code, and watch ChatGPT generate the solutions for them. Given this new reality we’re all now living in, Sam and I wanted to know what computing instructors are planning to do to ensure that their students are still learning well.
To gather a diverse sample of perspectives, we interviewed 20 university CS1/CS2 instructors across 9 countries (Australia, Botswana, Canada, Chile, China, Rwanda, Spain, Switzerland, United States) spanning all 6 populated continents. To our knowledge, our paper is the first empirical study to gather instructor perspectives about these AI coding tools that more and more students will likely have access to in the future.
Here’s a summary of our findings:
Short-term, many planned to take immediate measures to discourage AI-assisted cheating by weighing exams more, trying to ban these tools, or showing students their limitations. Then opinions diverged sharply about what to do longer-term, with one side wanting to resist AI tools by creating more AI-resistant assignments and exams, and the other side wanting to embrace these tools by integrating them deeply into introductory programming courses.
Our study findings capture a rare snapshot in time in early 2023 as computing instructors are just starting to form opinions about this fast-growing phenomenon but have not yet converged to any consensus about best practices. Using these findings as inspiration, we synthesized a diverse set of open research questions regarding how to develop, deploy, and evaluate AI coding tools for computing education. For instance, what mental models do novices form both about the code that AI generates and about how the AI works to produce that code? And how do those novice mental models compare to experts’ mental models of AI code generation? See Section 7 of our paper for more examples.
We hope that these findings, along with our open research questions, can spur conversations in our community about how to work with these tools in effective, equitable, and ethical ways.
Check out our paper here and email us if you’d like to discuss anything related to it!
Sam Lau and Philip J. Guo. From “Ban It Till We Understand It” to “Resistance is Futile”: How University Programming Instructors Plan to Adapt as More Students Use AI Code Generation and Explanation Tools such as ChatGPT and GitHub Copilot. ACM Conference on International Computing Education Research (ICER), 2023.
Broadening Participation in Computing by Moving Away from Computer Science: Information, Arts, Humanities, and Sciences offer better models for #CSforAll
In April, I gave a talk at Carnegie Mellon University’s Software and Societal Systems Department (S3D) “Broadening Participation in Computing by Moving Away from Computer Science” — slides available here, and video available here.
The argument I’m making is that computer science as a field has become more narrow over time. I wrote a CACM Blog Post last month where I provided several definitions of computer science: “Education is always changing: We need to define CS to keep the good stuff.” The earliest definitions of computer science described it as something to be taught to everyone, a critical literacy for 21st century citizenry, and touching on many different aspects of modern life. (I tell the story of those early definitions in this blog post.) More modern computer science definitions are much more narrow.
Computer science departments perform a critical function. They produce software professionals. Our society needs those. But that’s not the only societal need for computing education. CS departments also perform a gatekeeping function so that they can certify their graduates as ready for professional programming. If we want “Computing Education for All” with alternative endpoints, we need less of that.
During the pandemic, I worked with a Computing Education Task Force at my University to discover what kinds of computing education was currently available (see that story here and our final report here) across the campus. There’s a lot, even outside of CS. Arts teaches wonderful courses on expression with computing. Sciences teach how to discover with computing. Humanities reflects on the role of computing in society and critiques our digital and computational systems. Our School of Information teaches similar introductory computing courses to what we offer in CS, but with a focus on data science, user experience, and impact on society.
I suggest in my talk that new initiatives like our Program in Computing for the Arts and Sciences in University of Michigan’s College of Literature, Science, and the Arts (LSA) is more likely to invent computing education for everyone than will CS departments. The first half of my talk is about the history of computing education and about the narrowing of definitions (for which I use the U-M CSE standard slide templates), and the second half (for which I use the U-M LSA standard slide templates) is about PCAS, our new courses, and how this serves to broaden access to and participation in computing education.
Let’s think about how we could broaden our goals beyond CS departments and CS majors in K-12 education. Advanced Placement CS A and CS Principles are tightly tied to CS curriculum. What would it look like to create Advanced Placement exams for other needs for computing education? For example, what would an AP in Computational Science look like? What would it mean to value alternative endpoints in “Computing Education for All”?
Participatory Design to Support University to High School Curricular Transition/Translation in FIE 2022
Here’s my second blog post on papers we presented during the first year of PCAS. Emma Dodoo is an Engineering Education Research PhD student working with me and co-advised with Lisa Lattuca. When she first started working with me, she wanted a project that supported STEM learning in high school. We happened upon this fascinating project which eventually led to an FIE 2022 paper (see link here).
The University of Michigan Marsal Family School of Education has a collaboration with the School at Marygrove in the Detroit Public Schools – Community District. The School at Marygrove requires all high school students to take a course in Engineering every year. The school is new, so when we came into the story, they were just starting to build an 11th grade Engineering curriculum. Where does an innovative K-12 school find curriculum for not-often-taught subjects like Engineering? It seemed natural to look to the partner university.
The University of Michigan has recently established a Robotics Department with an innovative undergraduate curriculum. The leadership in the U-M School of Education and the School at Marygrove decided to use some projects from the undergraduate curriculum for the 11th grade Engineering curriculum. Emma and I came in to run participatory design sessions to help with supporting the high school in adopting the university curriculum. We focused on one project in particular, where students would input data from a LIDAR sensor on a robot, then visualize the results. What we wanted to know was: What are the issues that come up when using a university curriculum to inform an innovative high school curriculum?
We started out with a set of interviews. We talked to undergraduates who had been in the Robotics curriculum and asked them: What was hard about the robotics projects? What were things that you wished you knew before you started? They gave us a list of issues, like the realization that equations on a plane could be used to define regions of a picture, that colors could be mapped to numbers and equations, and that pixels could be queried for their colors.
We talked to high school educators about what they wanted students to learn from the project. They were pretty worried about the mathematics required for the project, especially after the students had spent all of 10th grade on-line during the pandemic.
Emma took these objectives and concerns, and generated a set of possible activities to be used in the class. She used Desmos, Geogebra, and our new Pixel Equations teaspoon language. (I mentioned back in this blog post that we were using Pixel Equations in participatory design sessions — that’s when this study was happening.)
Pixel Equations was developed explicitly for the concerns that the undergraduates were raising and that the math educators cared about. Users specify the pixels they want to manipulate (leftmost column) by providing an equation on a plane or an equation based on the RGB channels in the color in the pixel. They specify the desired color changes in terms of the red, green, and blue channels (three columns on the right). The syntax for the boolean expressions and the equations for calculating colors is the same as what students would see in Java, C, or JavaScript. But there are no explicit loops, conditionals, or data — it’s a teaspoon language.
Emma ran participatory design sessions with stakeholders, including teachers from the School at Marygrove. Her goal was to identify the features that the stakeholders would find valuable, and in particular, to identify the concerns of the high school educators that may not be addressed in the university curriculum. She identified four sets of issues that were important to the stakeholders when transferring curriculum from the university to the high school:
- Prior Knowledge: The students knew Desmos. Using a tool they used before would help a lot when dealing with the new robotics concepts.
- Priming: The computing must be introduced so that there is time for the students to become familiar with it.
- Motivational Play: High school students need more opportunities to see the fun in an activity than undergraduate students.
- Self-efficacy: It’s important for students to feel that they can succeed at the activities.
That’s where the paper ends
All of this design and development happened during the pandemic. We didn’t hear much about what was going on in the high school, and we couldn’t visit. When we talked to our contacts at the School of Education, we found that they didn’t have much news either. It wasn’t until much later that we saw this news item. The 11th grade Engineering class actually didn’t do any of the mathematics activities that we’d helped with. Instead, the school got a grant for a bunch of robots, and the classes focused on directly programming the robots instead. It’s disappointing that they didn’t use any of the things that we worked on, but as I’ve been mentioning in this blog, we find that adoption is really hard. Other factors (like grants and the wow factor of programming a robot) can change priorities.
The paper is interesting for investigating stakeholder issues when transferring activities from university to high schools. Those are useful issues to know about, but even if you address all the issues, you still might not get adoption.
The information won’t just sink in: Helping teachers provide technology-assisted data literacy instruction in social studies
Last year, Tammy Shreiner and I published an article in the British Journal of Educational Technology, “The information won’t just sink in: Helping teachers provide technology-assisted data literacy instruction in social studies.” (I haven’t been able to blog much the last year while starting up PCAS, so please excuse my tardiness in sharing this story.) The journal version of the paper is here, and our final submitted version (not paywalled) is available here.
Tammy and I used this paper to describe what happened (mostly during the pandemic) as we continued to provide support to in-service/practicing social studies teachers to adopt data literacy instruction in their classes. Since this was a journal on educational technology, we mostly focused on two technologies:
- The OER Tammy created to support data literacy in social studies education — see link here.
- DV4L, the Data Visualization for Learning tool that we created explicitly for social studies teachers — see link here.
When we started collaborating together, we looked for a theoretical model could inform our work. The end goal was easy to describe: we wanted social studies teachers to teach data literacy. But it’s hard to measure progress towards that big, high-level goal. Teachers are teaching data literacy, or they’re not. How do you know if you’re getting closer to the goal? We structured our work and our evaluation around the Technology Acceptance Model (TAM). TAM suggests that adoption of a new technology boils down to two questions: (1) is the technology actually useful in solving a problem that users care about, and (2) is the technology usable by the users? Those were things that we could measure progress towards.
During the pandemic, we ran several on-line professional learning opportunities — a workshop where practicing teachers could try out the OER with some guidance (e.g., “Make sure you see this” and “Why don’t you try that?”), and kick the tires on a bunch of tools including DV4L. We gathered lots of data on those teachers, and Tammy did the hard work of analyzing those data over time. We made progress on TAM goals — our tools got more usable and more useful.
But we still got very little adoption. TAM didn’t work for us. Adoption didn’t increase as usability and usefulness increased.
Why not? That’s a really big question, and we barely touch on it in this paper. It’s now a couple of years since we wrote the BJET article, and I could now tick off a dozen bullet points of reasons why teachers do not adopt, despite a technology being both useful and usable. I’m not going to list them here, because there are other publications in the pipeline. Bahare Naimipour, the EER PhD student working on our project, is finishing a case study of some teachers who did adopt and how their beliefs about data literacy changed.
I can give you a big meta-reason which probably isn’t a surprise to most education researchers but might be a surprise to many computer scientists: It’s not all about (or even mostly about) the technology. I led the group that worked on DV4L, and I’ve been directing students who have been helping Tammy make the OER more usable and useful (including build new tools that we haven’t yet released). TAM matters, but the characteristics of the individual teachers and the context of the teacher’s classroom are critical factors that technology is unlikely to overcome.
This is work funded in part by our National Science Foundation grant, #DRL2030919
A Workshop on Slow Reveal Graphs for Social Studies Teachers
My collaborator, Tammy Shreiner, is running a workshop for social studies educators on teaching with Slow Reveal Graphs. The idea of slow reveal graphs is that visualizations are just too complex for students to pick out all the visual elements. Instead, a slow reveal graph is presented in stages, and at each stage, students are prompted to reflect (and discuss, or write about), “What do you notice now? What do you wonder about?”
Tammy has been building a bunch of slow reveal graphs that really fascinating. I’m particularly amazed at the ones that she and her colleague Bradford Dykes have been building. They are taking hand-drawn visualizations (like the fascinating ones by W.E.B. Du Bois) and recreating them in R, so that they can generate the slow reveal process.
She’s offering a workshop in January that I highly recommend.
Dear friends,
I am writing to share information about a professional learning opportunity focused on teaching primary source data visualizations using the “slow reveal” process. The PLO will take place on Zoom over two Saturdays, January 21 and 28, 9:00-noon. It is open to teachers inside and outside of Michigan.
Please share the attached flyer with your social studies teacher colleagues. A sneak peak of the website that we will share with participants is below.
Thanks for sharing!
Tammy
Launching PCAS, the first two COMPFOR classes, and hiring our first lecturer
I last gave an update on the Program in Computing for the Arts and Sciences (PCAS) here in February (see blog post). Since then, it’s become real. I was hired as of July 1 as the Director of PCAS. My Computing Education Task Force co-chair, Gus Evrard, is the Associate Director. We even have a website: https://lsa.umich.edu/computingfor
I am building and teaching our first two courses now. I love our course code: COMPFOR. It stands for “COMPuting FOR…” The two courses are:
- COMPFOR 111 Computing’s Impact on Justice: From Text to the Web
- COMPFOR 121 Computing for Creative Expression
I have never worked harder than this semester– building these two courses, teaching both courses at the same time, learning how to be a program director (e.g., explicit classes and workshops on academic leadership, on evaluating faculty, and on how University of Michigan budgets work), and creating the program. I am having enormous fun.
I plan to write more about the two courses here and our innovations in teaching them. Here’s a brief summary. We are using teaspoon languages to introduce concepts, Snap for programming assignments, and Runestone ebooks for helping students to transfer their knowledge from blocks to traditional textual languages (Python, Processing, and SQL). I gave a talk for the CS for Michigan Conference a few weeks ago, and for the attendees, I created a page connecting to some of what we’re building and a narrative account of a couple of the units: https://guzdial.engin.umich.edu/cs4mi-pcas/.
We have been given permission to hire our first lecturer: https://careers.umich.edu/job_detail/226551/leo-lecturer-i-compfor-111-and-compfor-121. Right now, it’s a one year position for 2023-2024, but if enrollments grow, we have been encouraged to request three year positions starting in Fall 2024. Please do forward this job announcement to anyone you think might be interested. It’s also available in other places like CRA: https://cra.org/job/university-of-michigan-lecturer/.
Doing a Little Housekeeping and Rebranding
I discovered today that I have written over 2,500 blog posts here on WordPress, starting in June 2009. There was a time when I was writing daily. This is the first post I’ve written here since June. From a pace of a new post every day, to once every six months.
Our lives change so much from year to year. Thirteen years feels like so many changes ago. I live in a different state, working for a different University. Even the name of the department where I work has changed — I was in the School of Interactive Computing at Georgia Tech. Now I’m in the Division of Computer Science and Engineering in the College of Engineering at the University of Michigan and I direct the Program in Computing for the Arts and Sciences.
I changed the name of this blog once before. When I started out here in 2009, there were few websites and blogs on computing education. I just called it the Computing Education Blog. But as the field grew and there were more and more terrific sites helping teachers do computing education, I renamed the site Computing Education Research Blog. My focus was on the research, and not on how to be a great computing educator.
Today, there are many great resources even on Computing Education Research. I particularly recommend https://csedresearch.org/. I attended one of their on-line panels this last week — such interesting ideas and such a wonderful and diverse group of researchers.
I have been reticent to post under a banner saying “Computing Education Research” because this site has pretty broad visibility now. There are many more subscribers than the first few years. Newcomers might come here with that title and expect to read a newsletter or an authoritative perspective on the field — that’s an overwhelming responsibility. I recognize that I’m a senior (read: “old”) voice in the field, but I am just one of many voices in the field. Like any academic, I want to share what we’re working on and what I’m thinking about. I do not want to my posts here to appear like I’m speaking for the field.
So, I have renamed the blog for a second time: Computing Ed Research – Guzdial’s Take. This blog represents my perspective. That’s how I’ve always thought of the blog, but I want to make it explicit.
I have updated the Guzdial Papers page for the first time in a decade, and changed the About page.
Thanks for reading!
New ICER paper award for Lasting Impact: Guest blog post from Quintin Cutts
I serve on the ACM SIGCSE International Computing Education Research (ICER) conference steering committee. Quintin Cutts is Chair of the Steering Committee. I offered to share his announcement of a new Lasting Impact paper award here in the blog.
This is an invitation to nominate a paper for the ICER Lasting Impact Award 2022, or to offer to serve on the judging panel.
Which ICER paper has caused you to change the way you teach, or the direction of your research? Which has helped you to see and understand CS education more clearly? Has it also had an impact right across the community? I know which paper I would nominate, if I were allowed (but I’m not! – see below). It’s been a game-changer for me, and across CS education. Which one has done this for you and others?
1. Description of the award
The ICER Lasting Impact Award recognizes an outstanding paper published in the ICER Conference that has had meaningful impact on computing education. Significant impact can be demonstrated through citations, adoptions and/or adaptations of techniques and practices described in the paper by others, techniques described in the paper that have become widely recognized as best practices, further theoretical or empirical studies based on the original work, or other evidence the paper is an outstanding work in the domain of computing education research. The paper must have been published in ICER at least 10 years prior (i.e., for the 2022 award, papers must have been published in or before ICER 2011.)
2. Requirements for nominating a paper
a. An ACM Digital Library link to the paper being nominated.
b. A brief summary of the technical content of the paper and a brief explanation of its significance (limit 750 words).
c. Signatories to the summary and significance statement, with at least two current SIGCSE members. The name, contact email address and affiliation of each person who has agreed to sign the endorsement is acceptable.
3. Nominating yourself as a potential award judge
Please consider nominating yourself as a potential judge. We are seeking judges who have significant experience in the ICER community. We will ask judges to serve who do not have nominated papers, to avoid conflicts of interest.
4. Additional Notes
a. ICER Steering Committee members cannot nominate papers.
b. The ICER Steering Committee chair, Quintin Cutts, will run the process this year, and his papers cannot be nominated.
c. In this inaugural year of the Award, we will not have a pre-defined rubric. We will ask the judges to report the rationale for their decision, and the report will be made public when we announce the winner.
5. Timetable (all times, 23.59 AoE)
17th July: Nominations close.
18th July: Judging panel selected from the candidate pool, depending on the number of nominations and conflicts. Papers sent out to the judges.
2nd August: Judging panel sits to deliberate and makes a decision, which is passed to PC chairs. Winner notified.
The award will be presented at the ICER 2022 conference in Lugano, Switzerland either in person or on-line.
6. Submitting nominations
Please send both paper nominations and judging self-nominations to me, Quintin Cutts at Quintin.Cutts@glasgow.ac.uk
Programming in blocks lets far more people code — but not like software engineers: Response to the Ofsted Report
A May 2022 report from the UK government Research Review Series: Computing makes some strong claims about block-based programming that I think are misleading. The report is summarizing studies from the computing education research literature. Here’s the paragraph that I’m critiquing:
Block-based programming languages can be useful in teaching programming, as they reduce the need to memorise syntax and are easier to use. However, these languages can encourage pupils to develop certain programming habits that are not always helpful. For example, small-scale research from 2011 highlighted 2 habits that ‘are at odds with the accepted practice of computer science’ (footnote). The first is that these languages encourage a bottom-up approach to programming, which focuses on the blocks of the language and not wider algorithm design. The second is that they may lead to a fine-grained approach to programming that does not use accepted programming constructs; for example, pupils avoiding ‘the use of the most important structures: conditional execution and bounded loops’. This is problematic for pupils in the early stages of learning to program, as they may carry these habits across to other programming languages.
I completely agree with the first sentence — there are benefits to using block-based programming in terms of reducing the need to memorize syntax and increasing usability. There is also evidence that secondary school students learn computing better in block-based programming than in text-based programming (see blog post). Blanchard, Gardner-McCune, and Anthony found (a Best Paper awardee from SIGCSE 2020) that university students learned better when they used both blocks and text than when they used blocks alone.
The two critiques of block-based programming in the paragraph are:
- “These languages encourage a bottom-up approach to programming, which focuses on the blocks of the language and not wider algorithm design.”
- “They may lead to a fine-grained approach to programming that does not use accepted programming constructs…conditional execution and bounded loops.”
Key Point #1: Block-based programming doesn’t cause either of those critiques. What about programming with blocks rather than text could cause either of these to be true?
I’m programming a lot in Snap! these days for two new introductory computing courses I’m developing at the University of Michigan. I’ve been enjoying the experience. I don’t think that either of these critiques are true about my code or that of the students helping me develop the courses. I regularly do top-down programming where I define high-level custom blocks, as I design my program overall. Not only do I use conditional execution and bounded loops regularly, but Snap allows me to create new kinds of control structures, which has been a terrific help as I create block-based versions of our Teaspoon languages. My experience is only evidence that those two statements need not be true, just because the language is block-based.
I completely believe that the studies being cited in this research report saw and accurately describe exactly these points — that students worked bottom-up and that they rarely used conditioned execution and bounded loops. I’m not questioning the studies. I’m questioning the inference. I don’t believe at all that those are caused by block-based languages.
Key Point #2: Block-Based Programming is Scaffolding, but not Instant Expertise. For those not familiar, here are two education research terms that will be useful in making my argument.
- Scaffolding is the support provided by a learner to enable them to achieve some task or process which they might not be able to achieve without that support. A kid can’t hop a fence by themselves, but they can with a boost — that’s a kind of scaffolding. Block-based programming languages are a kind of scaffolding (and here’s a nice paper from Weintrop and Wilensky describing how it is scaffolding — thanks to Ben Shapiro for pointing it out).
- The Zone of Proximal Development (ZPD) describes the difference between what a student can do on their own (one edge of the ZPD) and what they might be able to do with the support of a teacher or scaffolding (the far edge of the ZPD). Maybe you can’t code a linked list traversal on your own, but if I give you the pseudocode or give you a lecture on how to do it, then you can. But the far edge of ZPD is unlikely to be that you’re a data structure expert.
Let’s call the task that students were facing in the studies reviewed in the report: “Building a program using good design and with conditioned execution.” If we asked students to achieve this task in a text-based language, we would be asking them to perform the task without scaffolding. Here’s what I would expect:
- Fewer students would complete the task. Not everyone can achieve the goal without scaffolding.
- Those students who do complete the task likely already have a strong background in math or computing. They are probably more likely to use good design and to use conditioned execution. The average performance in the text-based condition would be higher than in the block-based condition — simply because you’ve filtered out everyone who doesn’t have the prior background..
Fewer people succeed. More people drop-out. Pretty common CS Ed result. If you just compare performance text vs. blocks, text looks better. For a full picture, you also have to look at who got left out.
So let’s go back to the actual studies. Why didn’t we see good design in students’ block-based programs? Because the far edge of the ZPD is not necessarily expert practice. Without scaffolding (block-based programming languages), many students are not able to succeed at all. Giving them the scaffolding doesn’t make them experts. The scaffolding can take them as far as the ZPD allows. It may take more learning experiences before we can get to good design and conditioned execution — if that even makes sense.
Key Point #3: Good software engineering practice is the wrong goal. Is “building a program using good design and with conditioned execution” really the task that students were engaging in? Is that what we want student to succeed at? Not everyone who learns to program is going to be a software engineer. (See the work I cite often on “alternative endpoints.”) Using good software engineering practices as the measure of success doesn’t make sense, as Ben Shapiro wrote about these kinds of studies several years ago on Twitter (see his commentary here, shared with his permission). A much more diverse audience of students are using block-based programming than ever used text-based programming. They are going to solve different problems for different purposes in different ways (a point I made in this blog post several years ago). Few US teachers in K-12 are taught how to teach good software engineering practice — that’s simply not their goal (a point that Aman Yadav made to me when discussing this post). We know from many empirical studies that most Scratch programs are telling a story. Why would you need algorithmic design and conditioned execution for that task? They’re not doing complicated coding, but the little bit of coding that they’re using is powerful and is engaging for students — and relatively few students are getting that. I’m far more concerned about the inequitable access to computing education than I am about whether students are becoming good software engineers.
Summary: It’s inaccurate to suggest that block-based programming causes bad programming habits. Block-based programming makes programming far more accessible than it ever has been before. Of course, we’re not going to see expert practice as used in text-based languages for traditional tasks. These are diverse novices using a different kind of notation for novel tasks. Let’s encourage the learning and engagement appropriate for each student.
Getting feedback on Teaspoon Languages from CS educators and researchers at the Raspberry Pi Foundation seminar series
In May, I had the wonderful opportunity to speak at the Raspberry Pi Foundation Seminar series. I’ve attended some of these seminars before. I highly recommend them (see past seminars here). It’s a terrific format. The speaker presents for up to a half hour, then everyone gets put into a breakout room for small group discussions. The participants and speaker come back for 30-35 minutes of intensive Q&A — at least, it feels “intensive” from the speaker’s perspective. The questions you get have been vetted through the breakout room process. They’re insightful, and sometimes critical, but always in a constructive way. I was excited about this opportunity because I wanted to make it a hands-on session where the CS teachers and researchers who attended might actually use some Teaspoon Languages and give me feedback on them. I have rarely had the opportunity to work with CS teachers, so I was excited for the opportunity.
Sue Sentance wrote up a very nice blog post describing my talk (thank you!) — see here. The video of the talk and discussion is available. You can watch the whole thing, or, you can read the blog post then skip ahead to where the conversation takes place (around 26:00 in the video). If you have been wondering, “Why isn’t Mark just using Logo, Scratch, Snap, or NetLogo? We already have great tools! Why invent new languages that are clearly less powerful than what we already have?”, then you should jump to 34:38 and see Ken Kahn (inventor of ToonTalk) push me on this point.
The whole experience was terrific for me, and I hope that it’s valuable for the viewer and attendees as well. The questions and comments indicated understanding and appreciation for what I’m trying to do, and the concerns and criticisms are valuable input for me and my team. Thanks to Sue, Diana Kirby, the Raspberry Pi Foundation, and all the attendees!
Ruthe Farmer’s important big idea: The Last Mile Education Fund to increase diversity in STEM
I met Ruthe Farmer (Wikipedia page) when she represented the Girl Scouts in the early days of the NSF Broadening Participation in Computing (BPC) alliances. She played a significant role in NCWIT. I had many opportunities to interact with her in her roles at NCWIT and CSforAll. Ruthe organized the White House summit with ECEP in 2016 (see blog post) when she was with the Office of Science and Technology Policy in the Obama administration. Her latest project may be the one that’s closest to my heart.
Ruthe has founded and is CEO of the Last Mile Education Fund. Their mission is:
The Last Mile Education Fund offers a disruptive approach to increasing diversity in tech and engineering fields by addressing critical gaps in financial support for low-income underrepresented students within four semesters of graduation.
I was still at Georgia Tech when I heard about the completion microgrant program at Georgia State. Georgia State was (and still is) making headlines for their use of big data to boost retention and get students graduated. Georgia State is the kind of institution where over half of their students are classified as low-income. There is a huge social benefit when GSU can improve their retention statistics. The completion grant program was started in 2011 and focuses on students who could graduate (e.g., their grades were fine), but they had run out of money before they finished. The grant program gave no more than $2,500 per student (Inside Higher Education article). Today, we know that the average grant has actually been $900. That’s a shockingly low cost for getting students the rest of the way to their college degree. It’s a great idea, and deserves to be applied more broadly than one university.
The Last Mile Education Fund especially focuses on getting students from diverse backgrounds into STEM careers. These last gaps in funding are among the barriers that keep girls out from STEM careers (where Ruthe’s focus was in the Girl Scouts and NCWIT) but also low-income students and people of color.
I was reminded to write about the Last Mile Education Fund by Alfred Thompson’s blog (see post here). He’s got a lot more information about the Last Mile Education Fund there.
I am a first generation graduate. My parents and I had no idea how to even apply to college. I am forever grateful that Wayne State University found me in my high school, guided me to applying, and gave me a scholarship to attend. I’m a privileged white guy. Not everybody gets the opportunities I had. It’s critical to extend the opportunity of a higher education degree to a broader and more diverse audience.
The Last Mile Education Fund is important for closing the gap for students from diverse backgrounds. I’m a monthly supporter, and I encourage you to consider giving, too.
Three types of computing education research: for CS, for CS but not professionally, and for everyone
In February, I was invited to give a lecture at the University of Washington’s Allen School. I had a great day visiting there, even though it was all on Zoom. My talk is available on YouTube:
I got a chance to talk to Jeff Heer and Amy Ko before my visit. The U-W CSE department had been thinking about making a push into computing education research. They suggested that I describe the lay of the land — and particularly, to identify where I fit in that space. What I do these days (e.g. Teaspoon languages for history and mathematics classes) isn’t in the mainstream of computing education research, and it was important to tell people unfamiliar with the field, “There’s a lot more out there, and most of it doesn’t look like this.”
CS Education research dates back to the late 1960’s (see the history chapter that Ben du Boulay and I wrote). ACM SIGCSE started in 1968 with a particular focus on how to teach Computer Science and Information Technology majors. Much of what SIGCSE has published is focused even more specifically on the first course, which we now call CS1. This is a big and important space. These majors will be significant drivers of the world’s infrastructure.
There is a growing trend in computing education research to look at people who are learning programming (like in the first circles), but not for the purpose of becoming technology professionals. This includes K-12 CS teachers, end-user programmers, and conversational programmers. This kind of research sometimes appears in venues like CHI, CSCW, and VL/HCC, and occasionally in venues like SIGCSE, RESPECT, and ITiCSE. These circles aren’t scaled correctly by size of potential student population. By most measures, the outer circle (of people learning programming but who aren’t going to become technology professionals) is at least ten times the size of the student population inside the first circles.
My research is one level further out. I’m interested in studying what should we be teaching to everyone, whether or not they’re going to program like professionals, and how do we facilitate that learning. These students might not use the same tools or languages, and certainly have different goals for studying computing. I offer three reasons for the broader “everyone” to learn computing (drawn from the work of C.P. Snow, Alan Perlis, Peter Naur, and Seymour Papert — see this earlier blog post):
- To make sure that technology is controlled by a democracy.
- To support new ways of thinking and learning.
- To be part of a new computational literacy, a new tool for human expression.
This outer circle is far bigger in terms of number of students potentially impacted than any of the inner circles. But it’s also where we know the least in terms of research results.
Take a look at the talk for more on this way of thinking about the field, and how I connect that to existing research. I’d be interested in your perspective on this framing.
College Board stops sharing data on Advanced Placement Computer Science exams
Barb Ericson has been gathering data on the Advanced Placement exams in Computer Science for a decade. The College Board made available data about who took the exam (demographic statistics) and how well they did for each state, for AP CS Level A and then for AP CS Principles when that exam started. When she first started in 2010, she would download each state’s reports, then copy the data from the PDF’s into her Excel spreadsheets. By the time she processed the 2020 data, it was mostly mechanized. Her annual reports on the AP CS exam results were posted here until 2018. She now makes her reports and her archived data collection available at her blog.
However, the 2020 data she has posted are now the last data that are available. The College Board is no longer sharing data on AP CS exams. The archive is gone, and the 2021 data are not posted.
Researchers can request the data. Barb did several months ago. She still hasn’t received it. She was told that they would sign an agreement with the University of Michigan to give her access to the data — but not to her personally. She would also have to promise that she wouldn’t share the data.
Barb talked to someone at the College Board who explained that this is a cost-saving measure — but that doesn’t make much sense. The College Board still produces all the reports and distributes them to the states. They have just stopped making them publicly available.
I agree with Joanna Goode in this tweet from April:
The National Science Foundation paid for the development of the AP CS Principles exam explicitly to broaden participation in computer science. The goal was to create an AP CS exam that any high school could teach, that would be welcoming, and that would encourage more and more diverse students to discover computing. But now, the data showing us whether that’s working are being hidden. Why?
Updates: Workshop on Contextualized Approaches to Introduction to Computing, from the Center for Inclusive Computing at Northeastern University
From Nov 2020 to Nov 2021, I was a Technical Consultant for the Center for Inclusive Computing at Northeastern University, directed by Carla Brodley. (Website here.) CIC works directly with CS departments to create significant improvements in female participation in computer science programs. I’m no longer in the TC role, but I’m still working with CIC and Carla. I’ll be participating in a workshop that they’re running on Monday March 21. I’ll be talking about Media Computation in Python, and probably show some of the things we’re working on for the new classes here at Michigan.
https://www.khoury.northeastern.edu/event/contextual-approaches-to-introduction-to-computing/
Contextual Approaches to Introduction to Computing
Monday 3/21/22, 3pmEST/12pmPST
Moderator: Carla Brodley; Speakers: Valerie Barr, Mark Guzdial, Ben Hescott, Ran Libeskind-Hadas, Jakita Thomas
Brought to you by the Center for Inclusive Computing at Northeastern University
In this 1.5 hour virtual workshop, faculty from five different universities in the U.S. will present their approach to creating and offering an introductory computer science class (CS0 or CS1) for students with no prior exposure to computing. The key differentiator of these approaches is that the introduction is contextualized in one area outside of computing throughout the semester. Using the context of areas such as cooking, business, biology, media arts, and digital humanities, these courses appeal to students across the university and have realized spectacular results for student retention in CS0/CS1, persistence to taking additional CS courses, and declaring a major or minor in computing. The importance of attracting students to computing after they enter university is critical to moving the needle on increasing the demographic diversity of students who graduate in computing. Interdisciplinary introductory computing classes provide a pathway to students discovering and enjoying computing after they start university. They also help students with no prior coding experience gain familiarity with computing before taking additional courses required for the CS major. The workshop will begin with a short presentation by each faculty member on their approach to contextualized CS0/CS1 and will touch upon the university politics involved in its creation, the curriculum, and the outcomes. We will then split into smaller breakout sessions five times to enable participants to meet with each of the five presenters for questions and more in-depth conversations.
Updates: Dr. Barbara Ericson awarded ACM SIGCSE 2022 Outstanding Contributions to Education
March 2-5 is the ACM SIGCSE Technical Symposium for 2022 in Providence, RI. (Schedule is here.) I am absolutely thrilled that my collaborator, co-author, and wife is receiving the Outstanding Contributions to Education award! She is giving a keynote on Friday morning. Her abstract is below.
She’s got more papers there, on CS Awesome, on her ebooks, and on Sisters Rise Up. I’m not going to summarize them here. I’ll let you look them up in the schedule.
A couple of observations about the SIGCSE Awards this year that I love. Both Barb and the Lifetime Service to the Computer Science Education Community awardee, Simon, earned their PhD’s later in life, both within the last 10 years. Barb is the first Assistant Professor to win the Outstanding Contributions award in the 40 year history of the award.
I have one Lightning Talk. The work I’m doing these days is computing education, but it’s not in the mainstream of CS education — I focus on computing education for people who don’t want to study CS. So, I’m doing a five minute lightning talk on Teaspoon languages as provocation to come talk to me about this approach to integrating computing into non-CS subjects. You can see the YouTube version here. This is my attempt to show that each Teaspoon language can be learned in 10 minutes — I describe all of two of them in less than five minutes!
Outstanding Contribution Plenary
Friday, March 4 / 8:15 – 9:45
Ballroom A-E (RICC)
Barbara Ericson (University of Michigan)
Improving Diversity in Computing through Increased Access and Success
My goal is to increase diversity in computing. In this talk I explain why diversity is important to me. My strategy to improve diversity is to increase access and success. This work includes teacher professional development, summer camps, weekend workshops with youth serving organizations, curriculum development, helping states make systemic changes to computing education, publicizing gender and race issues in Advanced Placement Computer Science, creating free and interactive ebooks, testing new types of practice problems/tools, and offering near-peer mentoring programs.
Barbara Ericson is an Assistant Professor in the School of Information at the University of Michigan. She conducts research at the intersection of computing education, the learning sciences and HCI, to improve students’ access to and success in computing. With her husband and colleague, Dr. Mark Guzdial, she received the 2010 ACM Karl V. Karlstrom Outstanding Educator Award for their work on media computation. She was the 2012 winner of the A. Richard Newton Educator Award for her efforts to attract more females to computing. She is also an ACM Distinguished Member for Outstanding Educational Contributions to Computing.
Computing Ed Research - Guzdial's Take 
Recent Comments