Attractive Educational Robotics Motivates Younger Students to Learn Programming and Computational Thinking

(1)

UEF//eRepository

DSpace https://erepo.uef.fi

Rinnakkaistallenteet Luonnontieteiden ja metsätieteiden tiedekunta

2020

Attractive Educational Robotics

Motivates Younger Students to Learn

Programming and Computational Thinking

Jormanainen, Ilkka

ACM

Artikkelit ja abstraktit tieteellisissä konferenssijulkaisuissa

© 2020 Authors All rights reserved

http://dx.doi.org/10.1145/3434780.3436676

https://erepo.uef.fi/handle/123456789/24441

Downloaded from University of Eastern Finland's eRepository

(2)

Learn Programming and Computational Thinking

Ilkka Jormanainen

School of Computing University of Eastern Finland

Joensuu, Finland ilkka.jormanainen@uef.ﬁ

Markku Tukiainen

School of Computing University of Eastern Finland

Joensuu, Finland markku.tukiainen@uef.ﬁ

ABSTRACT

Educational robotics and physical computing have proved to be good sources of motivation for students of all ages and school levels. We conducted a series of workshops in the primary schools of city of Joensuu in eastern Finland, focusing on training the fundamental computational thinking (CT) skills by using a programmable and interactive Teddy Bear toy. Educational robotics and physical computing devices have proved to be an efficient way to teach these skills regardless the students’ age group or previous background. To assess the students’ intrinsic motivation towards Teddy Bear programming, we devised a survey for workshop participants and conducted a statistical analysis to compare differences between the genders and age groups. The results of the large-scale empirical study (n=1440) show that the students at the age of 9-10 years (Grades 3-4) are significantly more motivated towards such a learning tool than the students of age 11-12 years (Grades 5-6). Furthermore, we show that especially young girls find the Teddy Bear programming motivating and they are eager to learn more. This indicates that appealing tools play a key role when teaching programming and CT concepts to young school children.

CCS CONCEPTS

• Social and professional topics~Professional topics~Computing education~Computational thinking • Social and professional topics~Professional topics~Computing education~K-12 education

KEYWORDS

Computational thinking, physical computing, robotics, Arduino, programming education, K-12 education, motivation, IMI instrument

ACM Reference format:

Ilkka Jormanainen and Markku Tukiainen. 2020. Attractive Educational Robotics Motivates Younger Students to Learn Programming and Computational Thinking. In Proceedings of the Eight International Conference on Technological Ecosystems for Enhancing Multiculturality (TEEM 2020) (Salamanca, Spain, October 21-23, 2020), ACM, New York, NY, USA, 7 pages. https://doi.org/10.1145/1234567890

1 Introduction

Increased motivation is one of the often-mentioned benefits when using educational robotics in a school setting. Usage of robotics solutions or physical computing devices range from teaching basic computational thinking skills and programming in primary schools to advanced computer science and engineering topics in tertiary level education. Similarly, range of available tools is wide.

Young school kids may use ready-made robotics kits such as Lego Mindstorms or BeeBots [11] whereas older students may opt flexibility and versatility of the tools over the learning curve and use for example hand-crafted Arduino-based custom robots [13].

No matter what the context or utilised tools are, educational robotics has been seen to increase students’ motivation towards solving the task. For example, Chin et al. [4] found out that an educational robotics tools in elementary school level demonstrate a significant advantage in by improving students’ overall learning interest and motivation. In other contexts, for example Toivonen et al. [20] demonstrated that robotics environment motivates also university students when they are learning complex and abstract topics, such as artificial neural networks.

However, in many of the reported cases the participants of the robotics activities are selected on volunteer basis, activities are for example part of after-school robotics clubs or a special in- curriculum course. This implies that the students are readily motivated towards the activities. What is not so often reported in the literature, are large-scale experiments where all students in a certain school level or grade would be engaged in the robotics activities. In this paper, we report a study with primary school children, where we reached all pupils in all school in the grades 3-

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s).

WOODSTOCK’18, June, 2018, El Paso, Texas USA

(3)

TEEM’20, October 2020, Salamanca, Spain I. Jormanainen and M. Tukiainen 6 in Joensuu, eastern Finland. This intervention resulted to a large

experiment (n=1140) in which we used Intrinsic Motivational Inventory (IMI) instrument [16] to assess how motivated the school children between 9-12 years in general are towards educational robotics and physical computing when learning basics of computational thinking and programming.

In this study, we seek answers to the following research questions:

RQ1: Are younger students (Grades 3-4) more motivated towards a toy-like programmable device such as Teddy Bear than the students in Grades 5-6?

RQ2: Are there differences between girls and boys in their motivation towards the Teddy Bear programming?

This paper is organized as follows. First, we contextualise and position our study with the existing literature, especially from the viewpoint of how educational robotics affects to students’

motivation. Then we present the robotics solution used in the experiment, the programmable Teddy Bear. We describe the method, the survey instrument and implementation, and analysis of the survey results. We conclude the paper by presenting suggestions for the future work in the field.

2 Background

The concept of computational thinking (CT) has been discussed widely since it was introduced to the community by Jeanette Wing almost 15 years ago [22]. Researchers’ and practitioners’

community is yet to come with a unified consensus about the CT concepts, evaluation of CT competencies, and transferability of CT skills to other subject matters [9, 12]. However, as computing evolves, new tools and technologies have been introduced to teach CT concepts in all school levels, including K-12 and K-9.

Educational robotics and other physical computing devices as a platform have a well-established status as a tool to teach CT and programming principles [1]. Several initiatives and projects, such as TACCLE3 – Coding [10] and RoboSTEAM [5] (both funded by European Union from the Erasmus+ framework) are seeking internationally applicable methodological and technological solutions for the CT teaching and learning challenges.

Usage of educational robotics learning environments in different contexts has been reported widely in research literature, and examples of conducted interventions can be found from all school levels [6]. However, amount of reliable empirical evidence confirming that robotics is generally an effective tool remains limited [3, 4], but based on literature, a conclusion can be drawn that robotics environments do motivate students in specific contexts. There are studies using quantitative metrics measuring students’ motivation, such as a study by Chin et al [4], where the authors used Instructional Materials Motivation Survey (IMMS) to assess an educational robotics setting in elementary education.

The results show that overall, educational robot-based system improves students’ learning interest and motivation.

Another study by Cross et al. [7] reports development of specific tool, the robotics activity attitudes scale (RAAS), for

middle-school students. Similarly, Kaloti-Hallak et al. [14] report a study with middle-school students where they found out that students’ attitude to learn STEM and computer science subjects was increased when using robotics as a learning environment.

The abovementioned literature report studies in primary or secondary school levels. In higher education context, studies such as conducted by Verner and Ahlgren [21] suggest that robotics project have positive impact to learn science and engineering.

More recently, results by Apiola et al. [2] show that practically oriented robotics learning environment trigger students’

motivation in a computer science course.

In contrast to [4], our study population represents the whole age group in all schools of the authors’ city, hence leading to a large-scale analysis (n=52 in [4] vs. n=1140 in our study). We assume that the large population helps to strengthen the results and generalizes the assumption that educational robotics motivates students to learn principles of computational thinking.

What is more, our participants were not selected based on volunteer basis to join for example in a robotics competition (such as in [14]). Instead, the intervention was part of in-school curriculum for all students in the grades 3-6, hence providing a good representation of the whole student population.

3 Programmable Teddy Bear

The context of the study was built around a programmable, interactive Teddy Bear toy. The system was built as a prototype of a commercial product by Opinsys Ltd., a local IT company working in the field of educational technology. In the solution, the stuffed teddy bear was modified so that it was able to house Arduino microcontroller, alongside with the needed sensors and actuators (Figure 1).

Figure 1: Installation of a servo motor inside of the Teddy Bear.

Hardware set embedded in the Teddy Bear had the following components, and they were configured according to the schematics depicted in Figure 2.

• Arduino UNO

• 2 x Hitech HS-422 servos

• LDR07 photoresistor

• Piezo speaker

• Tilt sensor cw1300

• Resistors and jumper cables, breadboard

(4)

Learn Programming and Computational Thinking

Figure 2: Schematic for the hardware connections The interactivity with the Teddy Bear is accomplished in two different ways. First, the Teddy Bear has built-in sensors to detect if someone approaches or passes by close to it. The servos are built to move Teddy Bear’s paws, to create a sense of physical interaction. The piezo speaker enables the Teddy Bear to play sounds. Second, to enable the users to control the actions, an extension for Snap4Arduino programming environment has been developed with the needed Snap! blocks for controlling the Teddy Bear (Figure 3). The blocks control digital and analogue input and output pins of the Arduino board, although the low-level functionality was abstracted into higher level actions suitable for controlling the Teddy Bear. This abstraction for example allows users to program action where the Teddy Bear moves one of its paws to the desired position within servo motor’s range -180 – 180 degrees (Figure 4).

Figure 3: Snap4Arduino programming environment with the Teddy Bear extensions.

Figure 4: The readymade programmable Teddy Bear waving one of its paws.

4 Method

In this study, we used a modified Intrinsic Motivation Inventory (IMI) instrument to assess participants’ perception towards the programmable teddy bear and related tasks.

4.1 Context description

Survey data was collected during a series of workshops that were part of a project for introducing and improving technology- enhanced learning in the local primary schools. Staff members of the local IT company manufacturing the Teddy Bear produced content and lesson plan for a 30 minutes long workshop, which was conducted in a similar way on each school so that eventually all students of the schools (nschools=20) were reached.

The sessions were conducted for all six grades (1-6) of the primary schools, but for comparability and coherence, answers for grades 3-6 were only included in the analysis. For the lower grades, the survey was slightly modified, and hence not included in this study. The workshop was part of the in-curriculum activities, and sessions were guided by the teachers. The tasks in the workshop were simple, such as

“The Teddy Bear waves both of its paws so that they are in opposite positions. Both paws have only two positions that will be changed every two seconds.”

or

“The Teddy Bear waves for the persons passing by. The light sensor is used to detect the motion and waving the paw starts.”

During the workshop, the students programmed the Teddy Bear to complete the tasks, and they were instructed by the teachers and personnel from the company. The surveys were devised to the students at the end of sessions, and they answered to them immediately.

Altogether, 1385 answers were collected from the 3-6 graders.

Of this amount, 245 were considered as invalid and removed from the analysis. The invalid answers had either incomplete answers, missing grade or gender information, or multiple answers for a

(5)

TEEM’20, October 2020, Salamanca, Spain I. Jormanainen and M. Tukiainen single question. The total amount of the valid answers for the

analysis was hence 1140.

4.2 Self-Determination Theory and Intrinsic Motivation Inventory

Theoretical background of the research lies in self-determination theory (SDT), which provides a framework to study an individual’s motivation and emotions, and development of these aspects. In the core of the model are the individual’s motivation, personality, and optimal actions. Self-motivation and self- determination are defined by the intrinsic and extrinsic factors. It has been shown that intrinsic motivation has a direct connection to individual’s optimal learning and skill development. It has been argued by Deci and Ryan [8] that development of students’

intrinsic motivation should be a priority in teaching and learning processes. Intrinsic motivation supports activities that are naturally enjoyable and interesting for an individual. Also, personal values have a strong influence on individual’s intrinsic motivation [17].

The main research instrument used in the study was Intrinsic Motivation Inventory (IMI) [16, 18]. The IMI is a multidimensional instrument to assess participants’ self-determination and subjective experience related to a task. The IMI instrument has been used successfully in different contexts, also in computing [4, 19]. The instrument’s validity has been also verified [15].

4.3 Survey implementation

The IMI instrument is often modified to suit for a specific purpose.

From the seven subscales of IMI (Interest/Enjoyment, Perceived Competence, Effort/Importance, Pressure/Tension, Perceived Choice, Value/Usefulness, Relatedness) we chose the following to assess the students’ motivation towards the Teddy Bear programming session and this type of instruction model:

• Interest/Enjoyment

• Value/Usefulness

• Perceived Competence

The whole pattern of IMI instrument has more than 20 different questions / statements categorized into the abovementioned seven categories. For our purposes, and taking into account the participants’ age group, we selected six questions into the questionnaire. The questionnaire was translated to students’ native language, so the students were able to answer by using language familiar for them. All the statements were answered with five-step Likert scale (Strongly disagree .. Strongly agree). As it is typical for the IMI instrument, there are overlapping questions between and in the categories (for example questions Q3 and Q5, see below). The questions were presented to the students in mixed order to make this overlapping less salient for the participants.

Interest / Enjoyment

• Q1: Programming the Teddy Bear was fun.

• Q4: I found the programmable Teddy Bear interesting.

Value / Usefulness

• Q6: I would like to learn more about Teddy Bear programming.

• Q2: I learnt something new in the Teddy Bear programming workshop.

Perceived Competence

• Q3: I was good at programming.

• Q5: I was skilled when programming the Teddy Bear.

In addition to the IMI scale questions, the participants were asked to identify their school grade and gender. This information was used to categorize the responses for the analysis phase. As the participants of the study were minors, the consent from parents were sought and handled by teachers.

4.4 Preliminary study

Before the actual workshops and conducting survey with all students, it was tested with a subset of 150 participants during the preliminary workshops. The instrument categories were found to be meaningful, and they corresponded our observations we made during the workshop. Based on our observations during the preliminary survey, we changed for example statement “I found the Teddy Bear interesting” to the final form “I found the programmable Teddy Bear interesting” to make the questionnaire less confusing and keeping the students’ focus more strictly on programming aspects of the workshop. After analysing the results of the preliminary questionnaire, we concluded that the IMI instrument would be suitable for collecting subjective experiences and measuring intrinsic motivation in our experimental context.

4.5 Analysis

The survey forms were collected after each session, and results were inserted to an Excel table. Invalid answers were cleaned from the data. Altogether, 1140 valid answers were recorded from the students of 3-6 grades (9-12 years). Table 1 shows the demography of the study population.

Table 1. Participant demography.

Grade 3-4 (9-10 years)

Grade 5-6 (11-12 years)

Female 325 288

Male 265 262

Total 590 550

In the analysis phase, we compared if there were differences between the age groups (3-4 graders vs. 5-6 graders) and between the genders within the age groups.

5 Results

5.1 Descriptive statistics

Descriptive statistics from the answers is shown in Table 2. These statistics show that, in general, the students were satisfied and

(6)

motivated towards the Teddy Bear programming. They considered the programming fun (Q1), but on the same time they did not feel to be fully skilled to program the given tasks (Q3, Q5).

They found the programmable Teddy Bear interesting (Q4) and would like to learn more about it (Q6). The students also agreed that they learnt something new during the workshop (Q2;

although this answer has a higher standard deviation when comparing to other questions, indicating that the tasks were rather easy for a part of the students).

Table 2. Descriptive statistics from the survey.

Grade 3-4 Grade 5-6

Question Mean SD Mean SD.

Q1 4.62 0.85 4.49 0.87

Q2 4.15 1.20 4.06 1.21

Q3 4.07 1.09 3.90 1.07

Q4 4.58 0.90 4.29 1.03

Q5 3.99 1.08 3.90 1.02

Q6 4.26 1.16 4.15 1.17

5.2 Statistical analysis

The survey data was analysed with SPSS statistical software by using Mann-Whitney U test. We conducted the following tests to see if there was a statistically significant difference (p < .05) between the genders within the grades 3-4 and 5-6. Furthermore, we were interested in to see if there were differences between the age groups (3-4 graders vs. 5-6 graders). Table 3 presents the results from the statistical analysis for Grades 3-4.

Table 3. Grades 3-4, differences by the gender (significance level p < .05).

Grades 3-4 Questions by the

IMI categories

Mann- Whitney

U

Mean rank girls

Mean rank boys

p

Interest / enjoyment

Q1 40533.00 303.28 285.95 .092

Q4 41854.00 299.22 290.94 .424

Value / usefulness

Q2 41192.50 301.25 288.44 .311

Q6 41127.00 301.46 288.20 .276

Perceived competence

Q3 49269.50 276.40 318.92 .001

Q5 46191.50 285.87 307.31 .108

Girls in grades 3-4 considered Teddy Bear programming slightly more fun and interesting than boys of the same age, but there was no significant difference (Q1 and Q4 Mean rankgirls > Mean rankboys, p > .05). On the other hand, boys of the same age considered themselves significantly better programmers in

general (Q3 Mean rankgirls < Mean rankboys, p = .001) and better also in programming the Teddy Bear (Q5 Mean rankgirls < Mean rankboys, p > .05). The girls considered slightly more than boys that they had learnt something new and would like to learn more about Teddy Bear programming, however there was no significant difference (Q2, Q6 Mean rankgirls > Mean rankboys, p > .05).

Results of the analysis of the students at Grades 5-6 show that there is more diversion between girls and boys than with the younger students in Grades 3-4 (Table 4). Boys in Grade 5-6 considered programming more fun than girls did (Q1 Mean rankgirls < Mean rankboys, p = .005), but when considering how interesting the programmable Teddy Bear was, the result is aligned with lower grade students where the girls found the Teddy Bear more interesting (Q4 Mean rankgirls > Mean rankboys, p

> .05), however there is no significant difference. In the group of older students, boys considered themselves significantly more competent programmers than girls (Q3, Q5 Mean rankgirls < Mean rankboys, p = .001). This reflects to the finding that the girls considered more than boys that programming the Teddy Bear was more useful for them (Q2, Mean rankgirls > Mean rankboys, p > .05), but on the other hand, boys are more eager to learn more about Teddy Bear programming (Q2, Mean rankgirls < Mean rankboys, p >

.05).

Table 4. Grades 5-6, differences by the gender (significance level p < .05).

Grades 5-6 Questions by

the IMI categories

Mann- Whitney

U

Mean rank girls

Mean rank boys

p

Q1 42048.50 260.50 291.99 .005

Q4 37357.50 275.79 274.13 .891

Q2 35146.50 284.46 265.65 .133

Q6 40805.00 264.82 287.24 .068

Q3 48622.50 236.53 317.45 .001

Q5 49404.50 234.96 320.07 .001

The clearest and most significant result in the statistical analysis is revealed when comparing younger students to older ones (Grades 3-4 to Grades 5-6). Table 5 summarizes the findings from this analysis. There were statistically significant differences (p < .05) in all but one question (Q2 “I learnt something new in the Teddy Bear programming workshop”). In all cases, Mean rankGrade3-4 > Mean rankGrade5-6, p < .05 (except of Q2, where p = .108), strongly indicating that the younger students were more determined and motivated towards the Teddy Bear programming workshop.

(7)

TEEM’20, October 2020, Salamanca, Spain I. Jormanainen and M. Tukiainen Table 5. Grades 5-6, differences by the gender (significance

level p < .05).

Grades 3-4 vs. Grades 5-6 Questions

by the IMI categories

Mann- Whitney

U

Mean rank Grade 3-

4

Mean rank Grade 5-

6

p

Q1 145633.00 598.66 540.29 .001

Q4 131989.50 620.79 515.42 .001

Q2 154128.50 584.27 555.73 .108

Q6 151179.00 589.26 550.37 .025

Q3 145752.50 597.46 540.49 .002

Q5 151412.50 588.87 550.80 .040

5.3 Limitations of the study

The main limitations of the study are connected to the short duration of the workshop and the composition of the survey form.

The students were exposed to the Teddy Bear programming only during one workshop session of 30 minutes. It could be argued that this gave them only a superficial understanding about the technology and the topics. On the other hand, answering to the IMI survey after only a short session may capture students’ first- hand views and motivation towards the Teddy Bear programming. The IMI composition was reduced from the original instrument quite much (from more than 20 questions and seven categories to only six questions in three categories). The IMI instrument, however, is originally designed so that it can be modified and only subsets of it can be used. The short length of the survey helped us to ensure as complete answers as possible.

6 Discussion

The obtained results from the survey indicate clear differences on the students’ intrinsic motivation between the different genders and age groups. In general, the students’ motivation towards the Teddy Bearn programming workshop was in a good level. They enjoyed the workshop and considered programming the Teddy Bear fun. The results of the statistical analysis with n=1140 appropriately filled IMI surveys show the following answers to the research questions.

RQ1: Are younger students (Grades 3-4) more motivated towards a toy-like programmable device such as Teddy Bear than the students in Grades 5-6?

Yes. According to the results, the students in Grades 3-4 are significantly more motivated towards the Teddy Bear programming. They enjoyed the workshop more, they perceived it more useful, and they also would like to learn more about programming, when comparing to the students in Grades 5-6.

RQ2: Are there differences between girls and boys in their motivation towards the Teddy Bear programming?

The results show a difference between the genders, although the results are not so clear and significant than in RQ1. The difference is more visible with older students, where boys in Grades 5-6 were more motivated than girls of the same age.

The findings for the research questions were somewhat expected. Especially younger girls seemed to enjoy working with an appealing and toy-like programmable device. Simplicity of the Teddy Bear and limited possibilities for the programming were probably more attractive for the younger students, whereas the older students possess readily programming skills going beyond the given tasks, hence showing significantly weaker motivation towards the programming (although it must be noted that older students’ motivation was indeed in a good level, as it can be seen in Table 2).

7 Conclusions

In this paper, we have reported a large quantitative study where we explored students’ intrinsic motivation towards a programmable and interactive Teddy Bear toy. A series of workshops were conducted, and students’ motivation was measured by using the Intrinsic Motivation Inventory (IMI) instrument. Analysis of the results shows clearly that especially the younger students are motivated towards the toy-like programmable device. This strengthens the earlier findings that educational robotics and physical computing devices are suitable tools for teaching computational thinking and programming fundamentals also for young students. These tools motivate the students to learn the abstract topics, and younger girls in particular seem to enjoy when working with devices with “softer”

look and outfit.

ACKNOWLEDGMENTS

We thank all students and teachers in the primary schools of City of Joensuu (Finland) for participating to the study, as well as Opinsys Ltd. for providing the tools and materials for the experiment. We thank January Collective for their continuous support and encouragement for our research eﬀorts.

REFERENCES

[1] Charoula Angeli, and Nicos Valanides. 2020. Developing young children's computational thinking with educational robotics: An interaction effect between gender and scaffolding strategy, Computers in Human Behavior, Volume 105, 105954, ISSN 0747-5632, https://doi.org/10.1016/j.chb.2019.03.018.

[2] Mikko Apiola, Matti Lattu, and Tomi A. Pasanen. 2012. Creativity-Supporting Learning Environment – CSLE. ACM Transactions on Computing Education. 12, 3, (July 2012), 25 pages. DOI=http://dx.doi.org/10.1145/2275597.2275600 [3] Fabiane Barreto Vavassori Benitti. 2012. Exploring the educational potential of

robotics in schools: A systematic review. Computers & Education, 58, 3, pp 978- 988, ISSN 0360-1315. DOI=https://doi.org/10.1016/j.compedu.2011.10.006.

[4] Kai-Yi Chin, Zeng-Wei Hong, and Yen-Lin Chen. 2014. Impact of Using an Educational Robot-Based Learning System on Students’ Motivation in Elementary Education. IEEE Transactions on Learning Technologies, 7, 4, 333-345.

DOI=https://doi.org/10.1109/TLT.2014.2346756

[5] Miguel Á. Conde, Camino Fernández, Jonny Alves, María-João Ramos, Susana Celis-Tena, José Gonçalves, José Lima, Daniela Reimann, Ilkka Jormanainen, and Francisco J. García Peñalvo. 2019. RoboSTEAM - A Challenge Based Learning Approach for integrating STEAM and develop Computational Thinking. In Proceedings of the Seventh International Conference on Technological Ecosystems for Enhancing Multiculturality (TEEM'19). Association for

(8)

Computing Machinery, New York, NY, USA, 24–30.

DOI:https://doi.org/10.1145/3362789.3362893

[6] Miguel Á. Conde, Francisco J. Rodríguez-Sedano, Camino Fernández-Llamas, José Gonçalves, José Lima, and Francisco J. García-Peñalvo. 2021. Fostering STEAM through Challenge Based Learning, Robotics and Physical Devices: A systematic mapping literature review. Computer Application in Engineering Education. In press.

[7] Jennifer Cross, Emily Hamner, Lauren Zito, Illah Nourbakhshh, and Debra Bernstein. 2015. Development of an assessment for measuring middle school student attitudes towards robotics activities. IEEE Frontiers in Education Conference (FIE), Erie, PA, USA, pp. 1-8.

[8] Edward L. Deci and Richard M. Ryan. 1987. The support of autonomy and the control of behavior. Journal of personality and social psychology 53, 6 (1987), 1024.

[9] Peter J. Denning, and Matti Tedre. 2019. Computational thinking. MIT Press.

[10] Francisco José García-Peñalvo and Antònio José Mendes. 2018. Exploring the computational thinking effects in pre-university education. Computers in Human Behavior, Volume 80, 407-411, ISSN 0747-5632, https://doi.org/10.1016/j.chb.2017.12.005.

[11] Francisco J. García-Peñalvo, Angela Marie Rees, Jenny Hughes, Ilkka Jormanainen, Tapani Toivonen, and Jens Vermeersch. 2016. A survey of resources for introducing coding into schools. In Proceedings of the Fourth International Conference on Technological Ecosystems for Enhancing Multiculturality (TEEM '16). Association for Computing Machinery, New York, NY, USA, 19–26. DOI:https://doi.org/10.1145/3012430.3012491

[12] Ting-Chia Hsu, Shao-Chen Chang, Yu-Ting Hung. 2018. How to learn and how to teach computational thinking: Suggestions based on a review of the literature, Computers & Education, Volume 126, 296-310, ISSN 0360-1315, https://doi.org/10.1016/j.compedu.2018.07.004.

[13] Ilkka Jormanainen. 2018. On computer science major students' motivation in a practically oriented robotics course. In Proceedings of the 18th Koli Calling International Conference on Computing Education Research (Koli Calling '18).

Association for Computing Machinery, New York, NY, USA, Article 29, 1–2.

DOI:https://doi.org/10.1145/3279720.3279749

[14] Fatima Kaloti-Hallak, Michal Armoni, and Mordechai (Moti) Ben-Ari. 2015.

Students' Attitudes and Motivation During Robotics Activities. In Proceedings of the Workshop in Primary and Secondary Computing Education (WiPSCE '15).

Association for Computing Machinery, New York, NY, USA, 102–110.

DOI:https://doi.org/10.1145/2818314.2818317

[15] Edward McAuley, Terry Duncan, and Vance V. Tammen. 1989. Psychometric Properties of the Intrinsic Motivation Inventory in a Competitive Sport Setting:

A Confirmatory Factor Analysis. Research Quarterly for Exercise and Sport 60, 1 (1989), 48–58. https://doi.org/10.1080/02701367.1989.10607413 arXiv:https://doi.org/10.1080/02701367.1989.10607413 PMID: 2489825.

[16] Richard M. Ryan. 1982. Control and information in the intrapersonal sphere: An extension of cognitive evaluation theory. Journal of Personality and Social Psychology 43, 3 (1982), 450 – 461.

[17] Richard M. Ryan, and Edward L. Deci. 2000. Intrinsic and extrinsic motivations:

Classic definitions and new directions. Contemporary educational psychology 25, 1 (2000), 54–67.

[18] Richard M. Ryan, and Edward L. Deci. 2000b. Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being.

American Psychologist 55, 1 (2000), 68 – 78.

[19] Annastasia Shipepe, Ilkka Jormanainen, Marcus Duveskog, and Erkki Sutinen.

2020. Screams of joy yield creative projects at the educational robotics workshop in Namibia. In International Conference on Advanced Learning Technologies and Technology-enhanced Learning (ICALT) pp. 103-15. Tartu, Estonia, IEEE Computer Society.

[20] Tapani Toivonen, Ilkka Jormanainen, and Markku Tukiainen M. 2018. An Open Robotics Environment Motivates Students to Learn the Key Concepts of Artificial Neural Networks and Reinforcement Learning. In Lepuschitz W., Merdan M., Koppensteiner G., Balogh R., Obdržálek D. (eds) Robotics in Education. RiE 2017. Advances in Intelligent Systems and Computing, vol 630.

Springer, Cham. https://doi.org/10.1007/978-3-319-62875-2_29

[21] Igor M. Verner and David J. Ahlgren. 2004. Robot contest as a laboratory for experiential engineering education. J. Educ. Resour. Comput. 4, 2 (June 2004), 2–

es. DOI:https://doi.org/10.1145/1071620.1071622

[22] Jeannette M. Wing. 2006. Computational thinking. Commun. ACM 49, 3 (March 2006), 33–35. DOI: https://doi.org/10.1145/1118178.1118215