A UI/UX For AAC Speech Impaired App
Video explaining how the Touch Voice app works for the speech impaired
Coming up on 10 years now, I designed and developed an app to help the speech impaired to communicate. I thought it might be interesting to review my approach to the design and where it has succeeded and also where it has fallen short.
First off, a disclaimer, I am not a medical professional nor a Speech Language Pathologist. What I am is a seasoned computer professional with decades of experience programming user interface designs and a learned development methodology. So with that understanding in mind, I am not providing medical advice here, and would never try to do so.
To the problem: A dear friend of mine lost the ability to speak because of a medical condition. I reasoned that I might be able to give him a communicative voice using an iPad much like Professor Stephen Hawking’s wheel chair at the time. Also understand that his medical condition was specific and he eventually recovered the use of his voice. My friend and some close family members informally consented to allowing me to try to design and code an app to allow my friend to communicate with an iPad.
My first attempt was what you might call “a shot in the dark”. It was crude, clumsy, and barely worked. What it did prove was it was possible and that much improvement was still needed. At the time, a decade ago, there was no available speech synthesis or API in iOS; that since has changed.
Since my friend was in a hospital care situation while recovering, I determined I needed to know more about his environment and his communicative needs in that environment to design an effective interface. So I spent an entire day in his hospital room during visiting hours taking notes. I observed things like lighting levels, bed, medical equipment, lavatory, eating, nurse rounds, other visitors, etc. The nurses thought it was a little funny at the time, but I was polite and explained myself. They got it, and even answered a few of my questions.
Now onto the coding. At the time another good friend of mine was doing the coding in iOS, while I focused my efforts entirely on the user interface design. By the time the improved design was ready to be tested again, my friend in the hospital had been moved to a recovering care facility in a different city.
I thought it was important to have my friend the coder (and business partner), and of course my friend the patient-tester both present at this second evaluation of the AAC (Augmentative and Alternative Communication) app. Since both friends knew me, but neither had met each other this would be a perfect test for how effective the app was at communication between persons that had never met before. Again, I took mental notes. The setting was an outdoor patio on a warm day.
The things I noticed this time were the volume and quality of the synthesized voice as well as how outside noises could drown out the understandability. Also something very noticeable, the patient-tester was having trouble accurately targeting the speech buttons as well as typing. This implied to me there might be other aspects of manual dexterity to consider when designing a speech impaired app. Another adjustment, or “tweaking of the design interface” was done to address some of this.
I began reading up on different medical conditions which might lead to speech impairment in my efforts to improve the design for those who may have manual dexterity challenges. I also learned that some speech impairment may have associated cognitive impairment. For example: someone may not understand the word “Cat” as written, but recognize one in “a picture of a cat”. For this I included a series of selective pictures centered around communicating the needs of someone in a care giving situation.
Finally, the app has gone through a few more re-designs to address distributions on different devices, screen sizes and platforms. I have since provided many hours of email “tech support only” for the app. I listen to feedback and judiciously make improvements. I know there are other apps out there now that address speech impairment needs with different AAC approaches and with various levels of success for each individual situation. I also know my app Touch Voice is effective in some medical situations for some individuals.
Ultimately, this has been a design and development journey helping others, for which I am indeed very proud.
