Project Objectives
This project proposes to apply a method to supplement (not replace) speech therapy
for specific disorders at the articulation level based on an integrated computer-based
system together with automatic speech recognition and distance learning. The key
features of this proposal are:
(A) therapy is based on real-time audio-visual feedback
of client's speech productions, while therapy sessions are designed for each speech
disorder separately and tailored specifically for each client;
(B) speech production
evaluation and interfacing to assistive technology is provided through automatic
speech recognition based on statistical models of the data collected in the
training/therapy phase;
(C) distance learning provides remote collaboration and
data collection for analysis and evaluation in diverse conditions.
These features correspond to the major system components:
- OPTACIA (optico-acoustic articulography) will visualise vocal tract configurations
and trajectories through a tailored acoustic-to-articulation kinematic mapping
in 2D or 3D. A therapist will be able to design a map, or select a predefined
one, to suit an individual client. OPTACIA will thus provide the client with
real-time visual feedback about her/his speech: articulator configuration will
correspond to map position and articulator movement will correspond to map
trajectory. It will be possible to re-use data collected during therapy sessions
to re-train the mapping.
- GRIFOS will be a speaker-dependent, small-vocabulary, automatic speech recognition
system. Training data will be gathered through OPTACIA sessions, supplementing
existing databases of material from similar client cases. In early sessions,
GRIFOS will serve to set appropriate thresholds in OPTACIA to control
acceptability of a client's speech productions and to analyse quantitatively
these productions in syllable and word context during speech therapy. In later
stages, it will primarily serve to evaluate client productions in continuous
speech, with training material taken from OPTACIA sessions. For clients with
severe articulation problems (e.g., dysarthria) unlikely to be fully resolved
by therapy, GRIFOS will help stabilise production by providing feedback to
increase production consistency rather than intelligibility per se. This will
allow reliable interfacing to synthetic speech output and other assistive
technology devices. Only speech synthesis (from already developed and available
systems) will be used to test this functionality in the present project.
- TELEMACHOS will apply distance-learning principles based on web database
technology to provide the system's remote tutoring and monitoring ability.
Telemedicine technology will enable wide application of the system and remote
data collection and evaluation. It will facilitate remote speech therapy
sessions as well as diagnostic and remedial information sharing.
The technological components necessary for this project are already developed to a
functional stage and will only need relatively minor further development to become
integrated into a comprehensive system. Efforts will focus on tailoring the application
of the technology to the needs of the speech disordered populations, and on assessing
the effects of the application on the lives of these persons and their carers, aiming
to maximise the gains from intervention and the facilitation of communication, both
for the short and long term.
The project will bring together research work relating to different languages and
develop common tools and analysis methodologies. In that context, the final
demonstrator will be a multilingual software system consisting of language independent
modules coupled with language-specific knowledge so that extensions to other languages
can be made in an effective way without making changes to the general structure of the system.
From the point of view of a person with a speech disorder:
- OPTACIA will help eliminate misarticulated patterns and replace them with
correct patterns (e.g., in the case of functional articulation disorders such as
lisping, lalling, etc., and craniofacial disorders), create new speech patterns
(e.g., in the case of severely hearing-impaired children), or improve the
consistency of speech patterns (e.g., in cases of severe dysarthria).
- GRIFOS will provide score-based reinforcement tailored to each case: from
utterance to syllable to word and finally to sentence context in the first
two example cases mentioned above, or toward consistency primarily at segment
level and secondarily at word level for the last case. The adaptive criterion
of acceptability will prevent client discouragement and allow gradual progress
according to individual ability and rate of improvement toward the personalised
goals. GRIFOS will also allow reliable interfacing to synthetic speech output
and other assistive technology devices.
- TELEMACHOS will allow continuation of high-quality speech therapy outside a
clinical setting. Accurate evaluation and feedback will be available at the
convenience of one's home environment, thus increasing rate of progress while
reducing the total cost of therapy. Maintenance of progress will be possible
after termination of on-site therapy sessions, while post-therapy evaluation
will be available with minimum expert supervision.
From the speech therapist's point of view:
- OPTACIA will provide an efficient audio-visual tool for teaching speech patterns.
The need to describe complex articulatory configurations and gesture dynamics
will be eased by the availability of visual trajectories and locations on a map
representing articulatory-quantal regions in real time. The therapist will be
able to visually guide formation of the required links between auditory processing,
internal representation and motor planning and execution, helping the client learn
correct speech patterns and eliminate faulty ones with repetitive rewarded efforts.
- On the grounds that automatic evaluation will be consistent with expert speech
pathologist assessment, GRIFOS will assist the therapist to evaluate clients'
progress objectively, untainted by subjective individual auditory ability and
free from habituation to client's particular misarticulated speech patterns.
This will allow precise evaluation of phonetic production in phrase context
as well as of production consistency.
- TELEMACHOS will give the therapist time to see more patients and to acquire
significant experience from the shared use of the OLP system. It will enable
remote therapy sessions for clients unable or unwilling to go to a clinic.
It will also allow remote assessment of clients' progress and modification of
therapy parameters with instructions and feedback.
All these aspects will jointly contribute to increase (or even provide) adequate
communication abilities and independence for the users of the system.
|
