Translating Cognitive Science to EdTech: Searching for Impact
I’m a post-doc in the Gureckis lab working on translating some of the ideas in cognitive science in educationally-relevant directions. I’ve always felt it is a shame that despite thousands of studies on learning and memory most cognitive psychology findings are not translated or absorbed into educational practice (I’m not alone in this). In this post, I will describe my own journey in the lab to bridge this gap. My next post will introduce in more detail the types of applications I am developing and discuss some of the design decisions that need closer scientific scrutiny. For now I am planning to just share our progress and the adventure of taking cognitive science out of the lab and into the real world! I’d be really interesting in hearing other people’s thoughts about translating cognitive science and education, so feel free to comment below!
Just after I arrived at NYU in January 2015, I told Todd that I had an ambition to enter the Global Learning XPRIZE (GLXP), with the goal of creating tablet software that would teach children basic literacy and numeracy skills. The competition, put forth by the same XPRIZE Foundation that kickstarted the private space industry in 2004, will award $1 million to up to five finalists who will be chosen this November after their software is tested by a panel of judges. The software from each finalist will then be loaded onto 400 tablets and distributed to 7- and 8-year-olds in Tanzanian villages (one per child, 10 per village) who have enrolled and been given math and literacy pre-tests. The kids will be given no guidance on how or when to use the tablets, and indeed may not have ever had any formal schooling. After 1.5 years of using the tablets (which will be charged daily via solar charging stations, but without internet), the children will take a post-test, and based on their improvement a single grand prize winner will be selected and awarded $10 million. Ambitious? Certainly. But possible? Perhaps.
The Global Learning XPrize represents the ultimate application that I believe cognitive scientists should be interested in, but often merely pay lip service to.
Creating a science-based entry to the GLXP has been my post-doc side project while I continued to design, implement, run, analyze, and write up more traditional cognitive psychology experiments. To me, the GLXP represents the ultimate application that I believe cognitive scientists should be interested in, but often merely pay lip service to. With my background in language acquisition, and our mutual interest in active, self-directed learning, I thought Todd and I would have as good a shot as anyone at designing, implementing, and improving automated tutoring software. At the very least, I wanted him to know that I had an eye towards (ambitious) applications of the scientific knowledge garnered in our research. With this in mind, Team egoTeach was born and officially registered, and along with ~130 other teams from around the globe.
egoTeach bakes in a lot of hard-earned wisdom gleaned from decades of cognitive psychology experiments on memory and learning.
I can expand on our specific difficulties in a later post, but for now I will illustrate with a simple example: imagine you wake up at 9am in a panic, suddenly remembering that you have a French vocabulary test at 10am that you forgot to study for. Given one hour, how should you distribute your study time over the, say, 60 words you expect to be on the quiz?
You could simply stare at each French-English word pair for one minute, but you recall that decades-old findings from memory studies indicate that it is better to study each item multiple times (even for an equal amount of total study time). So you decide to cycle through the list three times (or four? five? surely not 50?), studying each item for 20 seconds. However, looking at the first few words you realize you already know some of them quite well, and you get bored before the allocated study time is done: but should you trust this feeling? Judgements of learning in memory studies are correlated with later performance, but not perfectly. Nonetheless, active memory studies (e.g. this one) have shown that learners benefit from being able to control how long they study each item, so it may be better to move on when you feel you’re ready. You may not even need to study the easy ones as often; your time may be better spent on the hard ones. However, if your memory is poor or if there are too many items, maybe you should only try to learn a portion of them: but how many is too much? Another problem with going down the list repeatedly is that you will learn the sequence, much like you will learn to expect the next song on your favorite album as the last one finishes. Recall for individual items may depend in part on having the sequential context, so it would be better to randomize the order to some extent. Of course, that creates a logistical nightmare: unless you have study software or a flash card system that can aid you, keeping track of the how often each item should be seen–and when, based on its difficulty and your overall performance–is impossible. After all, you can’t spend your entire hour trying to think up, let alone implement such a system! For the French test problem, there are in fact a number of spaced repetition apps out there to help you study, but what about fundamental concepts essential to building literacy and numeracy in the first place?
As it turns out the more we know about the cognitive science, the harder it may be to identify the best application of that knowledge! This is the main challenge that we have been addressing in our GLXP submission. In particularly, Hugo Goulart de Lucena and I have been working on a series of games to teach the alphabet, spelling, numbers, counting, addition, and subtraction using more than just spaced repetitions of rote facts.
- AutoCognitiva, with engineers in Hong Kong and an educator in the US, presented a nicely-designed collection of ‘sandboxes’ that implement a constructivist style of education, allowing children to actively experiment (and fail) with, for example, phonic spelling (using a specially-designed phonics keyboard), subject-verb-object sentence construction (enforcing correct structure, but not meaning), and a number line using blocks to teach quantity and addition that I particularly liked. Although these sandboxes were far less game-like than our egoTeach apps, AutoCognitiva’s approach resonated most strongly with ours, given that we both focus on exploration and active learning to some extent. However, our games do have goals — even if the child has to explore to discover them (or maybe they will come up with their own goals!).
- RoboTutor, founded by Prof. Jack Mostow of CMU, comes from a decades-old literacy project of his that uses quite stilted speech to teach concepts and ask kids questions. It seemed to me a rote repetition form of teaching, but although I personally dislike this it may work. The math approach is to give narrated math problems, with robot-narrated hints when mistakes are made.
- GLEN, founded by Prof. Madhow from UC Santa Barbara, is mostly focused on teaching English–but will grudgingly do a Swahili translation to be eligible for the GLXP.
- Narrativas is a one-man show with a rather abstract and arcane timeline idea, and is admittedly at the idea stage but in search of a technical co-founder.
- John Harrison Project is run by Patrick, a self-described outsider with math and CS training who was intentionally vague about his approach, but tellingly gave us all an overview on state-of-the-art Swahili text-to-speech and speech recognition. Although actively-developed open source solutions such as PocketSphinx and Kaldi are making speech recognition easier than ever, implementing (and testing) grammar specifications for Swahili is a challenge. I assume the John Harrison project will be attempting to read children’s books aloud, perhaps prompting children to read along. Patrick also admitted he is more focused on the Adult Literacy X-prize.
- LearnLeapFly are a duo of Canadian math PhDs who gave a well-presented talk that said very little about what they are actually doing, although they claimed to soon be open-sourcing the development of their game engines and authoring tools from their 20-person team sometime — I look forward to seeing it!
- EduApps4All had the most heartfelt story, for the founder Isabelle has persevered while working towards the GLXP’s goal for 10 years–before it was a twinkle in XPRIZE’s eye! EduApps4All has a few games with happy, cartoony graphics and quite a bit of content. It looks as though the interactions are mostly limited to touching fixed buttons on the screen, which is rather different than our approach of making integrating the response dynamics with the stimuli.
- DeepOrigins didn’t show up in time for introductions, but I had a nice chat with two of them at lunchtime and was intrigued to hear that their approach uses Ayurvedic music to teach all of the content. I’m not sure what that means, exactly, but I’m very curious!
- Sleeping Dragons – Masako and Kenneth didn’t do a full presentation, but they mentioned they are linked with Tulane and seemed quiet and competent: I wish I’d had more of a chance to speak with them.
- Curriculum Content International, much like their name suggests, is a company that licenses content to large textbook publishers–and has done so for 30 years. The leader, Reginald Poe, admitted that they do not have the technical expertise to create the apps, but was looking to partner with a team that needed content.
- Dev4X is another NY-based team, founded by seasoned entrepreneurs Bodo and Michael, that is creating a content-agnostic platform with the hopes of attracting content companies. It looks like they will then try to sequence/weave the content together. Dev4X is notable for trying a radically open approach: anybody who wants to volunteer can be involved simply by joining their weekly Skype calls.
- Linguaculturalists are a family team (Michael and Michelle Hall, I believe) making what seems to be Farmville for education: you plant seeds that grow letters, which can be sold at the market for in-game money, or perhaps combined to build words. They are to be commended for actually publicly posting their code (and for actually having code!). I’m eager to see if their approach is successful: I do think such an ambitious game is exciting, but I think it will be quite difficult to effectively adapt a curriculum for this style of presentation…but I wish them luck! It would be truly exciting if it works.
- Library4All is a pre-existing organization that is entirely focused on the literacy side. They certainly have content but I did not manage to hear from them how they will teach the alphabet and spelling.
- Midnight Illusions is, I believe, the only team of professional app developers that was at the D.C. summit. Coming from outside Toronto and represented by Jason Bavington, they seem sharp, although we didn’t see their full presentation because they had already given it in Paris! I look forward to reviewing that video.
- Todo School has a UC Berkeley child development researcher involved, but I’m afraid we didn’t get any details (and their website is down).