O n f e r e n c e

Methodology 
The present paper reports on the evaluation of a sample of utterances which were used 
in a CALL platform which was being developed as part of the overall project. The CALL 
platform in question is 
Digichaint
, which was adapted from the Language Trap, a 
language learning game which was developed by the Knowledge and Data Engineering 
Group, Trinity College, Dublin by Neil Peirce and Prof. Vincent Wade (Peirce & Wade, 
2010). 
Digichaint
is aimed at 16-18 year old students with specific application to the Irish 
language Leaving Certificate (final State examination in second level schools in Ireland) 
syllabus in mind. It is designed specifically for the aural and oral components of the 
syllabus. 
Digichaint
is in the form of an interactive guided dialogue that allows students to 
progress through a virtual world using the original 
Language Trap 
graphics of a hotel and 
its surrounds. Learners are given the task of conversing with various characters that form 
part of the game. 
 
The subjects on whom the material was tested were a group of Irish language teachers 
(N=31). An Intelligibility and Clarity formula was devised which took account of teachers’ 
ability to correctly orthographically transcribe each of 20 sentences and also took account 
of their estimation of the ease with which they could do so. For the teacher group, 
intelligibility was measured in terms of the respondents’ ability to successfully transcribe 
each of the 20 sentences. These sentences were marked on a 1 to 5 scale with 5 
representing a fully correctly transcribed sentence. This methodology for assessing the 
intelligibility was used by a number of previous researchers (Pellegrini et al., 2012; 
Peterson, 2010). In addition to this, the respondents were asked to mark the ease with 
which they transcribed each sentence as they proceeded with 5 representing complete 
ease and 1 representing complete difficulty. In an ideal speaker-hearer situation the 
hearer could be expected to be able to transcribe the utterance of the speaker with 
complete ease thus gaining 5 points for 
intelligibility
and 5 points for 
clarity
. In the 
system used for the present study such a scenario would be marked as
. The 
extent to which sentences deviate below 5 represents a shortfall in their ease of 
intelligibility or 
clarity
. The overall mean Intelligibility and Clarity score for the 20 
utterances as scored by the 31 respondents may be represented as follows: 
5 - 
∑
N = total number of respondents 
= the mean performance score all of respondents 
= the mean opinion score of all respondents 
Teachers were shown a three minute video clip of the 
Digichaint
game before they were 
asked to partake in the transcription exercise in order to acclimatise them to the voices 
and to familiarise them with the nature of the game. The sentences were chosen so as 
include a representative selection of sound classes in the Irish sound system (long 
vowels; short vowels; fricatives and stops) as well as samples of elision in fixed common 
phrases. As the teachers provided an orthographic transcription for each sentence, they 
were asked to simultaneously rate the 
ease
with which they did so, on a scale of 1 to 5. 

---

-108- 
2014 CALL Conference 
LINGUAPOLIS
www.antwerpcall.be