Coming up with a way to keep talking even in situations that do not favour chatter
Communication is an essential part of social interaction and can be conducted through many different channels. A very common one is on the phone, but there are situations when a phone call, while urgent is not necessarily appropriate - in the silent section of a train, for instance. The Karlsruhe Institute of Technology (KIT) in Germany has attended to this issue and is working on a solution – Silent Speech Interfaces (SSI).
Professor Tanja Schultz’s team at KIT, including research assistant Michael Wand, started working on the SSI project in 2005 and so far, they are the only team to have made some sense of continuous silent speech. Making language available as a means of communication is the primary focus of the Silent Speech system, even when the spoken word can’t actually be spoken. This not only makes communication in public places more subtle and provides an option for when sensitive information needs to be expressed, but it can also be of great assistance to people with a speech impediment.
The Silent Speech Interface system consists of electrodes, an amplifier and recorder – roughly the size of an old school Walkman, and is linked to a laptop, which processes the information. When someone is speaking normally, the electrical potential that is running through the contracting muscle is measured.
"This activity is a physiological characteristic of the human body, so we only measure what is already there"
Electromyography (EMG) signals convey phonetic features, such as the position of the tongue or rounded lips when making a vowel sound. Afterwards the system identifies sounds and sound components, putting them together to form sensible words.
The first system prototypes were developed in 2005 and since then much has improved. The Silent Speech Interface has learned to not only distinguish between different words, but also between individual sounds. It has become generally more flexible and can even mould new and unknown words. Yet the project has not been all smooth sailing and during its course several challenges have been encountered in different sections.
“The experience made with ‘verdant’ speakers was that the EMG signal changed, this has a negative influence on the readings.” Luckily, this problem can be easily resolved with a little practice on the speaker’s part – practice makes perfect.
Besides being able to turn muscle movement into speech, the interface can be coupled with a translation system and work as an interpreter in multilingual conversations. The translation system currently used by Wand’s team was developed by another research group and is mainly used to prove the possibility of the concept, which it is doing quite successfully. The SSI can already convey the meaning of silently spoken words from languages such as English and German and is currently being tested on Japanese. In theory, however, it is possible to convey and translate any language.
In order to refine recording and processing the team is working with a new array of electrodes that are covered in many small measure points, which will make the distinction between individual muscles even more precise. Peeking into the future, Wand reveals some plans of making the SSI even more advanced, perhaps even capturing emotion, emphasis and the intonation of speech, with the overall objective being a more natural mimicking of communication.
There is no shortage of interested parties in silent speech technology and the possibility of making the system handy and mobile only amplifies this demand. The direction that will be taken, however, will have to coincide with the KIT team’s very clear vision.
“We want our computers to adapt to the human being, not the other way around. One shouldn’t have to bend over backwards in order to be able to use the system,” Wand explains.
This is a promising prospect, particularly for the various users that could benefit from the technology such as those travelling in special quiet compartments, open office call centres or the disabled. It may even convince one or two sceptics to keep quiet in the future.
Image: Nils Gräber, KIT