Professor partners with student team on interactive speech app for children with autism

Who they are

Impaired verbal and non-verbal communication are among the biggest problems for children with severe autism. Roughly 50 percent of kids diagnosed with autism are functionally non-verbal and will not develop sufficient natural speech or writing abilities to meet their daily communication needs, says Oliver Wendt, assistant professor of speech, language, and hearing sciences, and educational studies. Over the last several years, Wendt has been studying the effects of electronic devices intended to help children with autism learn how to communicate.

Initially, Wendt looked at specialized communication devices that speak messages stored on symbol cards and allow children to align up to five cards to construct simple sentences. He wondered whether a more sophisticated solution could be employed once children mastered the simpler communication devices.  Such devices also can be pricey and cost up to several thousand dollars. Wendt collaborated with students in Purdue’s Engineering Projects in Community Service (EPICS) program to explore the possibility of using Apple’s iPad to create a less expensive option. They looked at existing applications and didn’t find any appropriate for beginning communicators with autism. So they decided to create their own.

What they did

SpeakAll! is a customizable application for the iPad, available for free on iTunes, that facilitates natural speech and language development. The top portion of the screen features pictures and symbols, and the bottom is a storyboarding strip where kids can drag and drop pictures to create sentences. The app also allows parents and teachers to add photos to the existing bank of symbols. Once a child creates a sentence using the images, they can push the “Speak All” icon to hear the sentence out loud.

Nick Schuetz, a senior in electrical and computer engineering who led the student group, says another feature lets parents and teachers add pre-recorded speech to any picture. The child could combine his mother’s voice and his father’s picture, for example, to say “I want dad home.” Additional features include a hidden lock button, a randomization option for the graphic symbols and different symbol selection options. For children with fine motor difficulties who cannot “drag and drop” symbols, a one-touch mode is available for selecting symbols with a simple tap.

How they did it

About 15 EPICS students from a variety of disciplines worked on the app over two years. Wendt says a turning point occurred when he started using the prototype in his lab for research with real participants and asked the engineering students to observe. “They could see how it is being put to use, how participants react, what issues the clinicians run into, what needs further improvement,” Wendt says. “This gave them a good feel for the particular needs of the target population and allowed them to tailor the app around the specific behavioral and learning characteristics of children with severe autism.”

After disclosing the project through the Purdue Research Foundation and its Office of Technology Commercialization, a requirement for intellectual property developed with University resources that also serves to protect Purdue’s and the developers’ rights to benefit from it, the next challenge was meeting the requirements for publishing the app on the iTunes store. The students had to set up a developer’s account through Apple’s Developer Portal and meet Apple’s quality requirements. Released in November 2011 on iTunes, the app was downloaded more than 4,000 times in its first six months on the market. Tracy Holdman, a special education teacher in the West Lafayette Community School Corporation, has two students who use the app on classroom iPads. She says SPEAKall! gives her students a voice and helps them become more independent.

Free advice

"I don’t have a child with autism, and I never thought I’d be doing something like this, so don’t be afraid to try something because it’s new or difficult," Schuetz says. "Just get your hands dirty and stick with it, because someone who puts in time will gain experience, and that experience will help take a project to the next level."

"These students were passionate about their product, disciplined and goal-oriented and invested lots of time," Wendt says. "They all wanted to get this out there to help the autism community, and I think this passion and motivation really contributed to their success."