Non capisco - I don’t understand……
How many times have we used an AI translation tool and have discovered that the translation did not make sense or was misunderstood?
Whether using the AI translator to translate from English to German on my blog or using Google translate for English to Italian or Italian into English, I have seen many instances of incorrect translations. Why is it so difficult for AI Language translation to come up with the right match?
Even though AI language translation tools have been incredibly useful helping with communicating across different languages; they face some challenges which we will explore further.
AI language models utilize natural language processing — NLP tasks — and natural language understanding (NLU) to excel in comprehending and interpreting human language. Unlike earlier chatbots and automated systems that relied on rigid scripts and keyword matching, these models can better “understand” the context, sentiment, and intent. And this semantic understanding goes a long way to empower customer-support chatbots, virtual assistants, and search engines to do their best work.
AI language models have limitations in understanding complex language and may make mistakes when interpreting questions with a lot of context. This can make them less reliable for providing accurate information and help.
An embarrassing mistake happened when Google Translate was used to translate a NYC agency website into Chinese. Instead of translating "CPC" as it should have, it turned it into the Chinese Communist Party, causing confusion. It also translated "Contact DHS" as "Contact Department of Homeland Security" instead of "Department of Homeless Services." City officials fixed the translations, but it was seen as funny by many. However, immigrant rights advocates highlighted the deeper issue: immigrant communities struggle to access accurate information.
The most common issues relating to AI Translation models are as follows:
Language differences: English is the most common language spoken but reaching a multicultural audience, other language must be supported
Training data: The abilities of an NLP system depend on the training data provided to it. The best AI model must spend a significant amount of time reading, listening to, and utilizing a language
Development time: Consider the development time for an NLP system. To be sufficiently trained, an AI model must typically review millions of data points; requiring a distributed deep learning model and multiple GPUs working in coordination
Phrasing ambiguities: With some languages there may not be a clear concise meaning to be found in a strict analysis of the words; hence the model requiring more context or help to clarify. For example: in English, you’d say “I’m 30 years old.” In French, you would say «J’ai 30 ans» which translates to “I have 30 years” in English.
Misspellings: For a machine, misspellings can be hard to identify. You’ll need to use an NLP tool with capabilities to recognize common misspellings of words, in order to avoid mistranslations.
Innate biases: In some cases, NLP tools can carry the biases of their programmers, as well as biases within the data sets used to train them. Depending on the application, an NLP could exploit and/or reinforce certain societal biases, or may provide a better experience to certain types of users over others.
Words with multiple meanings: No language is perfect, and most languages have words that have multiple meanings or sound the same but have different meanings such as in French: une amande — an almond and une amende — a fine or au — contraction of à and le or aux — contraction of à and les or eau — water. There are plenty more!
Phrases with multiple intentions (double entendres): Some phrases and questions actually have multiple intentions, so your NLP system can’t oversimplify the situation by interpreting only one of those intentions. Your AI model needs to be able to distinguish these intentions separately.
How to overcome some of these challenges
Developing NLP models for multiple languages requires training in various languages to understand similarities and differences in language structure. This helps address linguistic diversity, and researchers strive to make NLP more inclusive by improving representation of underrepresented languages.
Many languages don't have much online content, so it's hard to train NLP models well. This is especially true for languages spoken by smaller groups or with unique writing systems. But researchers are looking into ways to help, like zero-shot and few-shot learning methods. These techniques allow models to work okay in languages with very little training data by using what's known about similar languages. Using more data and getting help from lots of people are also important for getting more multilingual training data.
In places where people speak multiple languages, they often switch between languages during a conversation or text (code-switching). This can be tricky for regular NLP models, which are usually trained to work with just one language. However, models like multilingual BERT and GPT-3 are good at handling code-switching by understanding the context around the mixed-language text. There's ongoing research to improve how we detect and understand code-switching.
Multilingual NLP models may unintentionally keep unfair ideas alive in their training data, which can hurt languages or communities that don't get as much representation. It's really important to be fair and fix these biases. Scientists are working on ways to make NLP models less biased, like using a wider range of training data, adjusting models to better notice discrimination, and creating ways to measure fairness. There are also rules and checks to keep things fair.
Training big multilanguage models needs a lot of computer power and energy, which can be a problem for many researchers and groups. People are working to make pre-trained models and training processes easier to use. They're also making smaller models with the same abilities to use fewer resources. There are cloud services and options to use pre-trained models, making it simpler for more people to use Multilingual NLP.
There are number of options for using AI Tools for Translation: Copy.ai, Google Translate, ChatGPT, BingMicrosoft Translator, DeepL, Smartling, Wordley, Unbabel, Amazon Translate. In some cases, use a couple of those in tandem to validate outputs.
Overall, the continuous evolution and improvement of multilingual models are opening up new possibilities for cross-cultural communication, language accessibility, and global connectivity. As these models continue to advance, the potential for innovation in multilingual natural language processing will undoubtedly lead to a more interconnected and inclusive digital landscape.
