Eye-movement triggered animation for enforcing MLD children learners’ concentration on math question reading


Keyuan Zhou, Yen-Sung Chen



Introduction

When dealing with an unfamiliar math problem, it is important to read the question description carefully, in order to understand the question completely and conceive the way quickly to find correct answer. For young children learners who often make mistakes in math learning, sometimes themselves and their parents just blame these mistakes on carelessness lightly and make no effort to correct this bad habit. Actually, a majority of them do not fully understand the true meaning of questions so as to make mistakes in their later process of problem solving. And the misunderstanding of questions partly could be attributed to their distractions of the question reading. Especially for children who have learning disability (LD), it is even more difficult for them to sustain their attention on the given task (Richards et al., 1990). In order to solve this problem of distraction during the question reading, we propose to use eye-triggered animation to help to enforce children’s concentration during this process. This solution may improve their later problem solving process and math studying.

Overall, our project aims at helping children learners to concentrate on the question description accurately and efficiently if any distraction happens during their question reading process. In our project, if a children learner is distracted, that is, his/her eye fixation point moves out of the question description area for a longer time than a preset threshold, an animation would appear and redirect his/her attention to the question description. The eye-movement triggered animation is that an arrow starts from the current fixation point of children’s eyes to the question texts where he/she should pay attention. We want to use this animation to redirect children’s attention to the right place in this way. After the experiment, we made an evaluation and discussion to this method from accuracy, efficiency and user experience aspects.


Related Works

Concentration is important in learning new knowledge and solving problems. One approach to increase focus and productivity would be blocking distractions from the workspace (Mark et al., 2017). However, compared with adults, children have more limited memory and attention with encoding and task performance. According to Gagne (1985), using animation is an important way to gain students’ attention as well as provide clear instruction. Mayer and Anderson (1992) used experiments to demonstrate that animation combining visual and verbal information is effective to support the students’ learning process. Among the animations, Leutner et al. (2007) used experiment to examine that the text-highlighting strategy could help learners to form a better understanding and acquire more knowledge from the material. What’s more, according to Yeari, Oudega, and van den Broek (2017), text highlighting animation was demonstrated to decrease the amount of time the readers spent on the text. Besides the above methods, gesture instructions like pointing fingers are also shown to be able to help students with math learning disabilities (Hord et al., 2016).

Eye-tracking technology is often used to evaluate the effect of different learning methods as well as to enforce the learning process. For example, recorded data from eye-tracking could be used to analyze students’ emotional state (Porta et al., 2012), which could be a part of e-learning system. Not only an evaluation tool, eye-tracking technology could be used as a assisting tool. Sibert et al. (2000) created a eye gaze triggered reading assistant to automatically read the sentences that people focus on. It may potentially benefit children with dyslexia as well as adults with learning disabilities. The effect of using highlight in digital reading interfaces to direct attention toward relevant passage within texts (Chi et al., 2007) is also confirmed by eye-tracking data.


Rationale

On the one hand, seeing from the related works about eye-tracking technology in learning, it is mainly used as an evaluation tool for researchers to analyze testers’ learning behaviors. In this way, it usually takes a relatively long time for testers to benefit from the research to help them learn better. Because the application is not a real-time system that could provide feedback instantly.

On the other hand, simply using animations like sentences highlighting or blinking to attract learners’ attention may cause potential distractions for them who are not accustomed to these effects. In practical usage, these animation effects may affect the process of recognizing and comprehending the sentence for learners even when they are reading the sentence carefully. What’s more, as is stated in the article of Fisher et al. (2014), too much attention redirection markers may have negative influence on children’ concentration.

So choosing the appropriate time to use the animation to attract learners’ attention and balancing the goodness (attracting learners’ attention) and badness (making it hard to recognize texts) of animation effects are very essential in real context. For the first issue, we adopted a spatial restriction that the animation will be triggered only if learners’ attention is distracted to irrelevant places. For the second issue, we tried to keep the highlighted place as compact and detailed as possible and adopted a time restriction that the animation strengthness will decrease smoothly if learners’ attention is drew back.


System Design

We built our system as a computer-assisted instruction (CAI) program that provide instructional content in the math solving process (Seo and Woo, 2010). The system connects a children learner with the content of math problem mainly by the interface of a web application and the eye-tracker as an input device.

In our project, an eye-tracker (Tobbi Eye Tracker 4C) was used to observe children learners’ eye movement and obtain their gaze position. This non-intrusive eye tracking technology is named Pupil Centre Corneal Reflection (PCCR) that use a light source to illuminate eyes and calculate the reflections (“How do Tobbi Eye Trackers work?”, 2018). In the official website of Tobbi, its core SDK is provided (“Tobbi Core SDK”, 2018) to help developers to obtain data such as gaze position, filtered gaze stream, head pose etc.. But in our project, instead of using the SDK directly, we utilize an open source work (“GazeApp”, 2018) from the Github that could connect to the eye-tracker and return the gaze position to us in JavaScript environment. In this way, we could save time and focus on the building of the web application and its user experience.

The instructional content is one example (Fig. 1) of the math problems from the book Conceptual Model-Based Problem Solving (Xin, 2012) that are suitable for 3-6 years old children learners. Children learners are supposed to read math question and solve the problem along with the voice instructions.

Due to the limited precision of the eye-tracking device, it is hard to detect which line the children learner is focusing on. So we modified the linewidth of the layout (Fig. 2) to make sure a relatively accurate detection of the eye gaze position. Beside, as the whole area consists of two parts: one is for question comprehending, another one is for problem solving, we separate them in a clearer way in the visual presentation.

When using this web application, children learners have two things to do: one is to read (not necessary to speak up) the question description along with the pace of a preset voice; another one is to solve the problem with the help from voice instruction. In any stage of the whole process, if their attention is distracted elsewhere, the text read by the preset voice will be highlighted and a red arrow will direct to this particular highlighted sentence (Fig. 3). And the highlight effects will disappear once they re-concentrate on the text that they should focus on. If their attention is not distracted, there will be no highlight animation at all. This rule suits for both question reading stage and problem solving stage.


Experiment


Task

Participants are required to complete the whole process from instruction listening, question reading to problem solving in our system. Their eye movements data including gaze position, duration time etc. were recorded in the experiment.

Participant

Due to our aimed population and limited resources, we only recruited 5 children learners with math learning disability (MLD) from a local elementary school.

Hypothesis

  • H1: For children learners with MLD, eye-triggered animation could make their eye gaze route to approximate to the route of voice instruction.
  • H2: For children learners with MLD, eye-triggered animation could improve their correctness in math problem solving.

Discussion

To verify the usefulness of highlight and arrow animation, we also acquired data from the same children doing other math questions, but without any kind of assistance (within-subjects design). One key difference to be noted is that the sample rate for eye movement of the baseline experiment is 120 Hz, while in our experiment the sample rate is only 3 Hz. Therefore, we manually pick out one data point for every 40 samples in the baseline experiment to compare on an equal footing.

Compared with adults, it is more difficult to ask children to follow the experiment instructions since they do not obey orders well from strangers like us whom they have no trust in. For example, some participants tried to answer the question (and answered wrongly) before listening over the instructions. In the duration of our experiments, other accidents include: one participant told us she deliberately provided the wrong answer just for fun, and another participant refused to look at the screen the whole time. Therefore, among the total 5 participants, we have only 2 analyzable data sets (child A and child N).


Conclusion

For our first hypothesis (H1), we use the (In-range Points) / (Total Points) value as an metric to measure the degree of concentration in experiments. From Table 1 (baseline) and Table 2 (our approach), we can see child A’s ratio is 0.71 times smaller, while child N’s ratio is 2.09 times larger. If we calculate the average of A and N, the ratio is 1.07 times larger. (Chart 1).

We thus conclude that our approach slightly increases this ratio compared with the baseline experiment; in other words, helps children to direct attention to the desired working region.

Nonetheless, the result also shows that the correct rate between our experiment and the baseline do not have much discrepancy. This may be owing to the simplicity of the question structure, or the children have prior experiences to similar experiments. Hence, we cannot verify our second hypothesis (H2) that our approach improves the correctness of MLD children learner in math problem solving.




Reference

  1. Richards, Samuels, Turnure, & Ysseldyke. (1990). Sustained and Selective Attention in Children with Learning Disabilities. Journal of Learning Disabilities, 23(2), 129-136.
  2. Gagné, R. (1985). The conditions of learning (4th edition). New York: Holt, Rinehart and Winston.
  3. Mayer, R., Anderson, R., & Levin, Joel R. (1992). The Instructive Animation: Helping Students Build Connections Between Words and Pictures in Multimedia Learning. Journal of Educational Psychology, 84(4), 444-452.
  4. Leutner, Detlev. (2007). Self-regulated learning with a text-highlighting strategy: A training experiment. (Vol. 215).
  5. Yeari, M., Oudega, M., & Broek, P. (2017). The effect of highlighting on processing and memory of central and peripheral text information: Evidence from eye movements. Journal of Research in Reading, 40(4), 365-383.
  6. Porta, M., Ricotti, S., & Perez, C. (2012). Emotional e-learning through eye tracking. Global Engineering Education Conference (EDUCON), 2012 IEEE, 1-6.
  7. Sibert, J., Gokturk, M., & Lavine, R. (2000). The reading assistant: Eye gaze triggered auditory prompting for reading remediation. Proceedings of the 13th Annual ACM Symposium on User Interface Software and Technology, 101-107.
  8. Seo, You-Jin, & Woo, Honguk. (2010). The Identification, Implementation, and Evaluation of Critical User Interface Design Features of Computer-Assisted Instruction Programs in Mathematics for Students with Learning Disabilities. Computers & Education, 55(1), 363-377.
  9. Fisher, A., Godwin, K., & Seltman, H. (2014). Visual Environment, Attention Allocation, and Learning in Young Children. Psychological Science, 25(7), 1362-1370.
  10. Xin, Y. (2012). Conceptual Model-Based Problem Solving Teach Students with Learning Difficulties to Solve Math Problems (Career Development Series).
  11. Tobbi Core SDK. (2018, April 14). Retrieved from https://developer.tobii.com/tobii-core-sdk/.
  12. How do Tobbi Eye Trackers work?. (2018, April 14). Retrieved from https://www.tobiipro.com/learn-and-support/learn/eye-tracking-essentials/how-do-tobii-eye-trackers-work/.
  13. GazeApp. (2018, April 14). Retrieved from https://github.com/protagorastong/GazeApp.
  14. Mark, G., Iqbal, S., & Czerwinski, M. (2017). How Blocking Distractions Affects Workplace Focus and Productivity. UbiComp/ISWC’17 Adjunct, 928-934.
  15. Chi, E., Gumbrecht, M., & Hong, L. (2007). Visual Foraging of Highlighted Text: An Eye-Tracking Study. Human-Computer Interaction, Part III, 589-598.
  16. Hord, C., Marita, S., Walsh, J., Tomaro, T., Gordon, K., & Saldanha, R. (2016) Teacher and Student Use of Gesture and Access to Secondary Mathematics for Students with Learning Disabilities: An Exploratory Study. Learning Disabilities: A Contemporary Journal 14(2), 189-206.