O n f e r e n c e

Ryan Downey, Lisa A. Frumkes, Emily Nava 
Rosetta Stone, Arlington, United States of America 
[e-mail], [e-mail], [e-mail] 
Research Challenges in the For-Profit Sector 
Bio data 
Dr.
Ryan Downey
is Director of Assessment at Rosetta Stone, where he oversees the 
development of assessment instruments for evaluating learner skills and progress, and 
program outcomes.
 
Dr.
 Lisa A. Frumkes
is Director of Curriculum at Rosetta Stone. She has developed CALL 
materials in higher education institutions and as part of for-profit educational companies 
for over 20 years. 
 
Dr.
 Emily Nava
is Director of Learner Studies and Academic Research at Rosetta Stone, 
where she conducts research and studies in the areas of second language acquisition and 
phonetics. 
Abstract 
As in academia, responsible test publishers and curriculum developers often claim to 
employ research-based approaches in developing and implementing their products. 
Evidence of learner engagement may come from user-experience testing and in-product 
feedback. Evidence of pedagogical efficacy may come from qualitative feedback (e.g., 
learner self-report) or quantitative measures (e.g., improved test scores). Particularly for 
commercial entities, a tension exists between the need to attract learners (as paying 
customers) with promises of "optimal learning conditions" and the need to provide 
realistic suggestions on what learners' expectations should be.
This paper details some of the challenges experienced during a 15-month long project 
undertaken by Rosetta Stone to develop and validate the efficacy of a robust self-study 
and tutor-led online curriculum for English language learners (ELLs) interested in 
improving their workplace English skills. 
The learners were placed into curriculum courses leveled at the B1, B2, or C1 level on 
the Common European Framework (CEFR) scale before joining the study. Learners then 
completed at least one 30-minute e-learning lesson each week, and worked one-on-one 
with online instructors for 45 minutes each week. The e-learning sessions included both 
human-scored and computer-scored assessments; the instructor-led sessions included 
reading materials and activities upon which learners and instructors could base their 
interactions. 
We will share samples of our e-learning and live-instruction content, and discuss which 
skill areas are addressed by each, to give context for the study results. 
Quantitative (learner performance) and qualitative (exit interviews) data will be 
presented, in addition to results from periodic surveys and scores from pre- and post-
study assessments. 

---

-134- 
2014 CALL Conference 
LINGUAPOLIS
www.antwerpcall.be 
We will also share our reflections on the ability of a for-profit organization to collect, 
analyze, and use the results of research to influence the development of future products, 
while providing the best possible product.