Advantages and challenges in using mobile apps for field experiments: a systematic review and a case study

Conclusion
App-based experiments are the future as apps continue to dominate digital engagement. 
Apps not only can display web-based contents, but also can integrate smartphone sen-
sors. Unlike using laptops or tablets, people often carry their smartphones all day long. 
Using apps could be an effective and efficient way to reach, engage with, and track par-
ticipants. Equipped with smartphone sensors, apps provide new ways to study causal 
mechanisms with experiment big data. With the advantages brought by apps, social sci-
entists may answer new questions regarding the complex systems of communication and 
human interactions.
The systematic review indicates that although more and more scholars have started 
incorporating apps in field experiments, the majority of the studies have not fully lever-
aged apps’ potential advantages. Practical difficulties such as limited time and financial 
and human resources may prevent researchers from using apps as the field experiment 
platform. The details of the app development reported in the case study provide some 
practical guidance to researchers with no experience in using apps. While acknowledg-
ing the potential promises of apps for field experiments, researchers should also keep in 
mind the inherent constraints of technology-based research regarding generalizability, 
retention, and the design quality, and consider implementing multiple strategies to 
address the limitations.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this 
article.

---

194
 
Mobile Media & Communication 6(2)
References
Aharony, N., Pan, W., Ip, C., Khayal, I., & Pentland, A. (2011). Social fMRI: Investigating 
and shaping social mechanisms in the real world. 
Pervasive and Mobile Computing
, 
7
(6), 
643–659.
American Cancer Society. (2013). 
Cancer facts & figures 2013
. Retrieved from https://www.
cancer.org/content/dam/cancer-org/research/cancer-facts-and-statistics/annual-cancer-facts-
and-figures/2013/cancer-facts-and-figures-2013.pdf
Asgary, R., Sckell, B., Alcabes, A., Naderi, R., Adongo, P., & Ogedegbe, G. (2015). Perceptions, 
attitudes, and experience regarding mHealth among homeless persons in New York City shel-
ters. 
Journal of Health Communication
, 
20
(12), 1473–1480.
Banerjee, A., & Duflo, E. (2017). 
Handbook of economic field experiments
. Amsterdam, the 
Netherlands: North-Holland.
Ben-Zeev, D., Brenner, C. J., Begale, M., Duffecy, J., Mohr, D. C., & Mueser, K. T. (2014). 
Feasibility, acceptability, and preliminary efficacy of a smartphone intervention for schizo-
phrenia. 
Schizophrenia Bulletin
, 
40
(6), 1244–1253.
Centers for Disease Control and Prevention. (2011). 
National diabetes fact sheet, 2011
. Retrieved 
from https://www.cdc.gov/diabetes/pubs/pdf/methods11.pdf
Chaudhri, R., Brunette, W., Goel, M., Sodt, R., VanOrden, J., Falcone, M., & Borriello, G. 
(2012). Open data kit sensors: Mobile data collection with wired and wireless sensors. 
Proceedings of the 2nd ACM Symposium on Computing for Development 
(pp. 1–10). New 
York, NY: ACM.
Comscore. (2017). 
The 2016 U.S. mobile app report
. Retrieved from https://www.comscore.com/
Insights/Presentations-and-Whitepapers/2016/The-2016-US-Mobile-App-Report
Ferguson, T., Rowlands, A. V., Olds, T., & Maher, C. (2015). The validity of consumer-level, 
activity monitors in healthy adults worn in free-living conditions: A cross-sectional study.