Koli Calling 2024 Trip Report
in Posts
Photo Credits: Nickolas Falkner
Thursday: Arriving at Koli
I caught the IC1 train at 6:54 AM from Helsinki Central Station with VR. As someone who pays close attention to app usability, I was genuinely impressed with Finland’s train service (VR) app. Not only do they offer free WiFi on long-distance routes and maintain impressive punctuality, but their app also has an incredible UX/UI: I could see if someone was sitting next to me, access a help chat, track my route, and activate tickets (when activated, the conductor doesn’t even need to stop by).
An amusing incident occurred during the journey when a postdoc from the University of Helsinki, who was also heading to the conference, spotted me reading the proceedings. He posted in the Discord: “One laptop with an ACM paper open spotted in the train.” I turned around, and we ended up having coffee while discussing our projects.
I arrived at Koli on a bus that was waiting for us in Joensuu - everything was incredibly well-organized. Koli is a special place, situated in Koli National Park, a location of great historical and cultural significance for the Finnish people. It has inspired artists, writers, and composers like Jean Sibelius, who found creativity in its beauty. Today, Koli remains an emblem of Finnish identity and a reminder of the relationship between humans and their natural environment.
First of all, they welcomed us with lunch with these views.
Opening session
Then we dove right in. Juho Leinonen and Andreas Mühling did the introduction with a nice tribute to Brett Becker, a researcher who passed away very recently, a great loss for the CSEd research community.
Then they gave us some interesting data: this conference is organized jointly by the Leibniz Institute for Science and Mathematics Education / Kiel University (Germany), Aalto University, and the University of Eastern Finland (Finland). This year there are 74 participants, of which 61 are in person, from all over the world (Germany, USA, Netherlands, Switzerland…).
Session 1 (Regular): Problem Solving and Understanding – Session Chair Ilkka Jormanainen
After the introduction, we started the sessions which are the conference’s strong point. In the first one about problem solving and understanding, it was Eduardo Carneiro from Utrecht’s turn who told us about how students reason when refactoring code. Using the “think-aloud” method, they investigated students’ actions and reasoning while improving code.
The second session was given by Imke de Jong from Utrecht as well and was about how Computational Thinking (CT) skills are used in solving Bebras tasks.
The last one was by Joey Bevilacqua from Lugano about understanding programming expressions through the analysis of university exams.
Session 2: Doctoral Consortium Lightning Talks and Posters – Session Chair Nick Falkner
We finished with PhD student presentations about various research projects focused on improving programming education, including mobile escape rooms, interactive visualizations, industrial code quality tools, and applications in secondary school mathematics.
One could go to each poster asking questions, and the research covered from K-12 education to university level, emphasizing different aspects like computational thinking, mental models, and digital humanities.
Two particularly caught my attention:
The first one was by Naaz from the University of Toronto, who presented a way to improve novice programmers’ understanding through interactive visualizations and multiple representations.
The second one also blew my mind: it’s about integrating static analysis with large language models (LLMs). Their research shows that combining LLMs with static analysis allows significantly improving the quality of suggestions provided to students.
The evening ended with a wine tasting session led by Nick Falkner.
And I had a great conversation about life with Max Barth from ETH.
Did we have a Northern Light? Kind of.
Day Two: Friday
Keynote and Discussion – Session Chair Juho Leinonen
So let me start from the beginning of the day: Lauri Malmi, from Aalto University, did a very interesting review of Koli Calling’s 24-year history. In his presentation, he covered the conference’s origins, its evolution, the meaning of its name, as well as data about participants, publications, and other aspects.
That journey served as a basis for making several thought-provoking reflections about the development of the computing education area over time, addressing the topics that have been subject to research: K-12, tools for university students, university courses, curriculum, stakeholders, research, and other topics. It was an opportunity to reflect on how the field has advanced and where it might be heading.
Session 3 (Regular): Student Conceptions – Session Chair Hieke Keuning
The first block of sessions of the day was about student conceptions and had four papers.
The first was by Marco Hartmann, from ETH, who talked about the prevalence of misconceptions in programming among primary education students with xLogo and Scratch, touching on topics like sequentiality, loops, conditional structures, and the concept of ‘superbug’ (attributing interpretative capability to the computer).
The second talk was by Marko Schmellenkamp, from Ruhr-Universität Bochum, who addressed the types of errors in context-free grammars among secondary school students (10-12 years old).
The third talk was given by Tina Vrieler from Uppsala and was about applying the CSC framework to understand factors that facilitate children’s decisions to participate in code clubs: teachers, family members, friends, video games, caregivers.
The last session of the block explored, through Bostjan Bubnic from the University of Maribor, whether metacognition (behavior) can predict success in solving object-oriented programming problems.
Session 4 (Panel): Generative AI – Session Chair Otto Seppälä
The second block of sessions of the day was a panel discussion on Generative AI led by Otto Seppälä from Aalto.
Before my session, Isaac, a Costa Rican professor at Utrecht who specializes in computational thinking and GenAI, and Hieke from Utrecht talked about students’ perceptions (concerns) and use of generative AI tools (strategies, main uses) for programming in different computer science courses.
Then two researchers from the University of Helsinki talked about the unrestricted use of LLMs in a software projects course and student perceptions about learning and its impact on course performance.
Finally, it was my turn, where I shared a use case with 15 CS1 students who completed three programming tasks with the possibility of generative AI assistance and the learnings from this coexistence with AI.
This was special: I gave my first talk at a conference of this type and, additionally, I participated in my first panel, where we talked about the impact of GenAI on CSEd. I’ve uploaded the video of my presentation to YouTube, in case you’re curious and want to take a look: Watch here
The talk seems to have been quite well received based on the feedback I got at the end.
Lauri Malmi gave us a great moment at the piano while we had our afternoon snack.
Learning Analytics
The last block of sessions was about Learning Analytics.
The first one was with Fabian, super nice German guy I’ve talked quite a bit with during the conference, about a matrix factorization topic.
The second was given by my train colleague Leo Leppänen about flow analysis in student admission itineraries. Now I’m almost an expert in the Finnish educational system and its admissions hehe.
Cyrille gave us a fantastic session about comparing responses generated by beginner programmers and AI, pattern detection, clustering.
Finally, Elizaveta Artser, from JetBrains, presented Rhubarb, a custom clustering tool for selecting diverse Python solutions to show in MOOCs.
To end the day, there were board games and a party with a band included (apparently they were famous in Finland).
Saturday:
Session 6 (Regular): Teaching Tools – Session Chair Leo Leppänen
The first block today was dedicated to teaching tools. I enjoyed it a lot, I think there’s a great opportunity here.
We started with a block-based testing framework for Scratch presented by Ute Heuer from Passau (Germany). Very solid.
The second was about a tool created for LogEx problems and Parsons for equivalence demonstrations in logic by Ebrahim Rahimi, from Open University (Netherlands). I loved his way of explaining Parsons problems and his tool demo.
The third was about a tool for direct manipulation for teaching basic and advanced software testing presented by Maximilian Georg Barth from ETH.
The last paper of this block fascinated me with its approach. The basic idea is to combine LLMs and static analysis to generate progressive hints in programming tasks. Thank you, Anastasiia, great presentation!
Session 7 (Panel): Teaching Approaches – Session Chair Brian Dorn
The next block of the conference was panel style and focused on pedagogical approaches.
The first session was about cognitive load, increasing self-efficacy, and promoting conceptual understanding in basic programming, by Noah Cowit.
The second was about the limited impact that spatial skills training has on performance in a non-programming CS course and the factors that impact those skills to improve them.
The third session was given by Max Sölch, a super nice guy doing his PhD at TUM (Munich), and presented a tool to make exams more computerized under the Bring Your Own Device (BYOD) framework. His great team has created Artemis
We finished with a panel discussing topics like the time it takes to adopt the discussed practices, how to make the tool fair and work for everyone (an idea from Noah was representing different groups in the sample, for example), whether some programming languages are more appropriate for these pedagogical approaches, or how to make solutions work for all types of computers (and make it more equitable).
Then we enjoyed a Koli walk with spectacular views (if it weren’t for the fog). Still, a gift.
Great guy Max. We really hit it off.
Another of the stars I met was Fabian who works at Volkswagen but also dabbles in the field. He won best paper at the conference!
Session 8: Posters (including afternoon coffee) – Session Chair Andreas Mühling
We closed with some final posters. The winners for best in-person poster voting were: Nick Falkner and Ute Heuer. Congrats guys!
Session 9 (Panel): New Didactic Approaches – Session Chair Lauri Malmi
First was Anna Sollazzo’s turn: Towards a theory of humanistic computer science and its teaching. This talk broke me with the depth of the topic (I think everyone) but was revealing because it made us think about the intersection of both disciplines and about what it means to be human… BIG TOPIC.
Carsten Schulte closed with a very well-presented and quite theoretical session with new perspectives on the future of Computing Education: exploratory models in teaching and learning.
After the closing dinner, we also had the Koli Calling 2024 awards where the best paper winner was announced.
It went to Fabian Pfütsch and Frank Höppner with the paper “Estimating and Differentiating Programming Skills from Unstructured Coding Exercises via Matrix Factorization”. Congratulations, Fabian Pfütsch and Frank Höppner! I’m so happy about this well-deserved award, colleagues. Since I was having dinner next to them, I just took their picture:
Final Reflections
Finally, I’d like to tell you the best things I take away from my first experience at an academic conference:
- I realized the breadth and dimension the field has.
- I demystified professors and researchers, seeing them as approachable and collaborative.
- I came away with a better approach to structuring each section of a paper.
- Unlike industry conference talks, the sessions at this conference were excellent because they underwent a rigorous selection process beforehand. Usually, the first authors were present, so they had deep knowledge of the study and could answer even the most challenging questions.
- The experience benefited me in several ways: it supported my personal projects, strengthened my doctorate application, and helped me connect with potential collaborators.
- I thoroughly enjoyed every conversation. Being a conference held in a small, crowded classroom, it fostered an intimate atmosphere that encouraged lively and positive discussions. It also allowed for easy and direct interaction with all attendees. Koli brought together people who shared a genuine enthusiasm for the same topics, creating a delightful feedback loop of energy and ideas.
I hope this post helps you experience a bit of what it was like.
