To work correctly, a piece of software like this should be able to “transcribe” all complexities inherent in human speech, such as voice rhythm, length of speech and intonation. The main types of speech recognition are “automatic speech recognition” (ASR), “computer speech recognition” or “speech to text” (STT). Voice recognition might be the same technology used for the biometric identification of specific users. Machine learning (ML) software can make measurements of spoken words through a set of numbers that represent the speech signal.
Key challenges in automating speech recognition
Since emotions are subjective, emotion detection is a challenging task. Mel-frequency cepstrum coefficient (MFCC) is the most popular representation of voice signals’ spectral properties since it considers human perception sensitivity concerning frequencies. Latency is a key challenge in speech recognition. ML would need to predict words correctly in real-time to complete logically whole sentences. Bi-directional recurrent neural networks are some deep learning models that benefit from having access to an entire sentence due to the added context. In limiting context in the model structure via access to a short amount of information following a specific word, the neural network might reduce the latency. Speech speed is a significant issue in the context of SR since the same word can be pronounced very quickly or in a slow, drawling way. There are three essential features in a speech:
Lexical features (the vocabulary used): it would require a transcript of the speech based on the text extraction from the speech Visual features (the expressions the speaker makes): it would require access to the video of the conversation Acoustic features (sound properties like pitch, tone, jitter etc.)
Of course, bias is a well-known problem in artificial intelligence (AI). Stanford researchers found out that automated speech recognition systems made twice as many errors when interpreting words spoken by African Americans compared to when interpreting the same words spoken by whites.
As always, data is the fuel
Feeding sound waves into a computer is the first step in speech recognition. Turning sounds into bits is the way to do it. For social engineering of speech recognition to be successful, the process should include the following steps:
Access to a reliable speech database or voice sample collection. Extracting effective features: it improves the learning capabilities of the algorithm as the number of features that characterize a dataset are being reduced. Using ML algorithms to create reliable classifiers: ML algorithms can learn from training samples to classify new observations.
Deep learning is finally applicable to speech recognition, as it is accurate enough to use in everyday environments. Remember that a voice recognition system should operate smoothly, or it might be non-viable. In reality, everyone who wants to construct a workable voice recognition system needs to use a lot of training data. According to Soniox founder and CEO Klemen Simonic, “Google and Facebook have more than 50,000 hours of transcribed audio. One has to invest millions — more like tens of millions — of dollars into collecting transcribed data. Only then [can one] train a speech recognition AI on the transcribed data.” Speech recognition projects from Facebook and Microsoft used labeled and transcribed speech data, the length of which was between 13,100 and 65,000 hours. Mozilla’s Common Voice — a public voice database — has some 9,000 hours of recordings.
Speech recognition software: For good and bad
Data is the new oil or gold. You’ve heard that, right? It is not surprising that Google Now! or Siri comes cheap or even free — the goal is to make people use them as much as possible so the speech data they submit, wittingly or not, is recorded forever. It could be re-used as training data for speech recognition algorithms. Namely, the usage of tons of personal speech data is what separates the world-class speech recognition system from more specific private speech recognition projects. In addition, the common presupposition is that one can get a pile of data, feed it to the ML algorithm and nurture a top-notch AI system. That may work sometimes, but not for speech. Machines come across all kinds of troubles when recognizing speech: background noises, echo, bad quality microphones, different accents and more. Soniox claimed that by using large quantities of unlabeled audio and text to teach its algorithms to distinguish speech with accents from background noises, its speech recognition system could accurately convert 24% of words compared to other speech-to-text systems. For speech recognition, cybercriminals could use Soniox as an example of how to use “unknown” data for which no predefined label exists to train their ML software to classify, process the data and learn its structure. Semi-supervised learning is a technique where partially labeled data can be fed to ML to produce state-of-the-art results in speech recognition. However, for fine-tuning of a particular model in a specific dataset to be initiated, social engineers would need an actual transcribed audio dataset.
Automated speech recognition security and privacy implications
Brett McDowell, executive director of the FIDO (Fast IDentity Online) Alliance, stated that “voice recognition is vulnerable to a presentation attack; where the adversary records a sample of the targeted user’s physical characteristics and uses that to produce an imposter copy or ‘spoof’ of that user’s biometrics. We leave fingerprints on most everything we touch, and both our images and voices are easily recorded without our knowledge or permission.” An investigation by Raphael Satter for the Associated Press revealed how businesses, banks and even governmental institutions are quietly rolling out customer voiceprints. As the investigator explains, the term’s name derives from the fact that every person’s voice is unique in a way similar to their fingertips. Security researchers from Zhejiang University in China invented a way to activate voice recognition systems without speaking a word simply by employing high frequencies inaudible to humans but registerable on electronic microphones. You could prevent the attack if you turn off wake phases or restrict some functions on your devices. Hello Barbie is a smart doll that utilizes the combined power of ML and AI to produce logical conversations between her and her clients. Security researchers warned that Hello Barbie could act as a surveillance device by listening to a family’s conversations. This is possible due to natural language processing and advanced analytics that make sense of 8,000 lines of dialogue pre-recorded via a microphone on the doll’s necklace and stored on the company ToyTalk’s servers.
The ills of speech recognition
Scholars see a great future ahead of speech recognition technologies in communications, health, security, tourism and more. For example, speech recognition programs are beneficial to people with visual and hearing disabilities; however, we will probably see whether the pros will outweigh the cons.
Sources
4 Security Vulnerabilities That Affected Voice Recognition Technology, HackRead Automatic Speech Emotion Recognition Using Machine Learning, IntechOpen Deep Learning for Speech Recognition, Medium Hackers send silent commands to speech recognition systems with ultrasound, TechCrunch Machine Learning is Fun Part 6, Medium Speech Emotion Recognition (SER) through Machine Learning, Analytics Insight Speech Recognition in Machine Learning, What After College Speech Recognition-Problems-Security, Speech Recognition Information Soniox taps unsupervised learning to build speech recognition systems, VentureBeat Automated speech recognition is more likely to misinterpret black speakers, Stanford University Top 9 Machine Learning Applications in Real World, DataFlair Vocal theft on the horizon, CSO
