Read the Report

Wait, Who is Timothy McVeigh?

Internews commissioned Localization Lab to analyze community policy guidelines on Facebook and Youtube in four languages: Amharic, Arabic, Bengali, and Hindi. Globally, the analysis found that community policies on both platforms contain systemic errors, and English-language references were not applicable and lacked human review for usability and comprehension. The report finds shortcomings in all four languages and highlights a need from both Meta and Google to improve efforts to effectively communicate with the growing number of users in non-Anglophone countries.

Key findings from the report:

1. The texts showed regular, often systemic mistranslation of terms and translations that are not recognizable to speakers of those languages, even when the translated terms may be technically accurate.
2. The lack of localized examples leads to confusion and insensitivity to audiences outside of the United States, to the exclusion of speakers from other Anglophone countries.
3. Facebook’s Amharic translation was assessed as unusable for most Amharic speakers. YouTube removed their Amharic translation before an assessment was possible, possibly due to recognition of quality issues. Translations for other Ethiopian languages, such as Oromo and Tigrinya, are currently unavailable.
4. The review found that while Facebook and YouTube’s Arabic language translations are mostly readable, they have numerous errors in the translation of key terms and the contextualization of the policies.
5. The review found that both Facebook and YouTube’s Bengali policy translations are of limited quality and usability to most Bengali speakers because they do not reflect spoken or written language and they omit key terms.
6. The review found that Facebook’s Hindi policy translations are of limited quality and usability to most Hindi speakers, while YouTube’s policies were readable and mostly coherent, although they would be strengthened by using colloquial rather than “Hinglish” terms.
7. In languages that have significant regional variation, the translations used language that is dialectically specific and not universally recognizable, which limits the accessibility of the translation.
8. The translations appeared to rely on machine translation which may be a factor in their limited readability. The use of machine translation without adequate human review is not acceptable practice.
9. The translations showed numerous errors of cultural and social sensitivity, such as gendered language, slurs, and offensive mistranslations, which were more frequent in the Facebook translations.
10. The translations frequently omitted key terms and entire phrases that resulted in incomplete translations, an issue that was particularly noticeable across YouTube’s translated policies.
11. The translated policies regularly use terms, such as acronyms, without sufficient contextualization and that are often recognizable exclusively to an American audience.
12. The issues of cultural sensitivity and adaptation are preventable if platforms co-translate policies and conduct reviews with end-user communities. The fundamental issue of translation usability is the contextualization of key concepts into culturally relevant and identifiable terms.