O n f e r e n c e

data collection instrument
used. In the 
abstract submitted for this conference, an eye-tracking study is described. The main 
advantage of eye-tracking is that it is an online, real-time measure that allows 
researchers to capture learners’ eye
-
movements “during the uninterrupted processing of 
the input” (Ro
berts & Siyanova-Chanturia, 2013, p. 213). By doing so, it enables us to 
pinpoint learners’ focal attention on screen and study new types of research questions, 
which largely evolve around the relation between attention, noticing, and learning. Yet, 
there are also a number of limitations. These limitations are, for instance, situated at the 
level of data collection and directly affect the number of participants involved in the study 
(Winke, 2013). For instance, eye-calibration problems or glasses hinder accurate eye-
movement registration. Another challenge is related to the fact that eye-tracking does 
not, by itself, shed light on learners’ cognitive processes. In other words, we know what 
learners are focusing on, but eye-tracking does not reveal whether or how learners are 
actively engaging with the input. That is why other types of instruments, such as 
stimulated recall sessions, may need to be added in order to complement eye-movement 
results. Yet, if we want to know how learners use with captions (Vanderplank, 2013), 
eye-tracking is probably the most informative technique.
Finally, challenges are also at the level of 
data analysis and data reporting 
and 
concern the choice of appropriate statistical data analysis techniques. If we want to take 
into account the complexity of the construct and add elements that could not be 
controlled for in the design (e.g. a target word’s frequency of occurrence, target word 
length), analyses of (co-)variance may not be sufficient, even though they can be 
considered the most frequently used techniques in studies focusing on second language 
acquisition (Plonsky, 2013). Methods such as 
generalised estimating equations
(e.g. 
Hardin & Hilbe, 2003) and 
mixed-effects modelling
(Baayen, 2008) may present a series 
of benefits because they allow to take into account independent variables as well as a 
series of learner- or item-related variables (e.g. Sonbul & Schmitt, 2013) that cannot be 
included in, for instance, ANOVA analyses. However, it should be taken into account that 
the complexity of multi-level techniques is not only challenging for the researcher but it 
may also hamper the readers’ ease of reading and understanding. 
In this paper, I have chosen to focus on four methodological challenges. Yet, while some 
of the aspects may often be perceived and reported as study limitations, they also 
challenge us to continuously improve research designs and go beyond easily available 
options and solutions. In addition, these challenges present interesting ideas for new, 
perhaps methodologically-oriented, research questions and studies.

---

-261- 
2014 CALL Conference 
LINGUAPOLIS
www.antwerpcall.be