TAUS Review#4 - The Innovation Issue - July 2015

TAUS RE VIEW of language business and technology

The Innovation Issue Reviews of Language Business and Technology in Asia and Africa.

Columns by Nicholas Ostler, Lane Greene and Luigi Muzii and guest author John Moran.

PLUS an interesting use case from Intel.

July 2015 - No. IV

1

y mrofni lliw I An artificial intelligence approach touotranslation oitalsnart otwill hcaorevolutionize rppa ecnegilletn i laicďŹ business. itra nA gnikcehc ret fa your .kcots ruo

To realize a society in which everyone can interact freely across language barriers with the use of machine translation technology, and thereby contribute to invigoration and innovation in businesses. https://miraitranslate.com/en/

2

Magazine with a Mission How do we communicate in an ever more globalizing world? Will we all learn to speak the same language? A lingua franca, English, Chinese, Spanish? Or will we rely on translators to help us bridge the language divides?

Language business and technology are core to the world economy and to the prevailing trend of globalization of business and governance. And yet, the language sector, its actors and innovations do not get much visibility in the media. Since 2005 TAUS has published numerous articles on translation automation and language business innovation on its web site. Now we are bundling them in TAUS Review, an online quarterly magazine.

TAUS Review is a magazine with a mission. We believe that a vibrant language and translation industry helps the world communicate better, become more prosperous and more peaceful.

Communicating across hundreds – if not thousands – of languages requires adoption of technology. In the age of the Internet of Things and the internet of you, translation – in every language – becomes embedded in every app, on every screen, on every web site, in every thing.

In TAUS Review reporters and columnists worldwide monitor how machines and humans work together to help the world communicate better. We tell the stories about the successes and the excitements, but also about the frustrations, the failures and shortcomings of technologies and innovative models. We are conscious of the pressure on the profession, but convinced that language and translation technologies lead to greater opportunities.

TAUS Review follows a simple and straightforward structure. In every issue we publish reports from four different continents – Africa, Americas, Asia and Europe – on new technologies, use cases and developments in language business and technology from these regions. In every issue we also publish perspectives from four different ‘personas’ – researcher, journalist, translator and language – by well-known writers from the language sector. This is complemented by features and conversations that are different in each issue.

The knowledge we share in TAUS Review is part of the ‘shared commons’ that TAUS develops as a foundation for the global language and translation market to lift itself to a high-tech sector.

TAUS is a think tank and resource center for the global translation industry, offering access to best practices, shared translation data, metrics and tools for quality evaluation, training and research.

Colophon Editorial contributions and feedback can be sent to: TAUS Review is a free online magazine, published

General: editor@taus.net

four times per year. TAUS members and non-

Continental reviews of language business and

members may distribute the magazine through

technology:

their web sites and online media. Please write to

•

Africa review: africa@taus.net

editor@taus.net for the embed code. TAUS Review

•

Americas review: americas@taus.net

currently has about 7,000 readers globally.

•

Asia review: asia@taus.net

•

Europe review: europe@taus.net

Publisher & managing editor: Jaap van der Meer

Persona’s perspectives of language business and

Editor & publication manager, distribution and

technology:

advertisements: Anne-Maj van der Meer

•

Translator: translator@taus.net

•

Research: research@taus.net

Enquiries about distribution and advertisements:

•

Language: language@taus.net

review@taus.net

•

Journalist: journalist@taus.net

3

Content

Leader

Features

5. Leader by Jaap van der Meer

30. Unmoderated Remote Usability Testing of Machine Translated Content by Will Burgett

Reviews of language business & technologies 8. In Asia by Mike Tian-Jian Jiang 12. In Africa by Amlaku Eshetie

38. Contributors 40. Directory of Distributors 43. Industry Agenda

Columns 16. The Journalist’s Perspective by Lane Greene 18. The Language Perspective by Nicholas Ostler 21. The Translator’s Perspective by John Moran 25. The Research Perspective by Luigi Muzii

4

Leader

by Jaap van der Meer

No Government, No Innovation The

translation industry is quickly becoming a high-

tech industry.

So we say… But fear not. Translator’s jobs are not going away. Although Google Translate may be considered a big innovation, the company keeps hunting for more and better human translators who can produce the most readable and best localizations

and its economical application in the next millennium. But, as I wrote in my recent blog article “The Brains but not the Guts”, very often, at the end of a project, researchers join the companies that have the guts to make it work.

of its products.

“With all respect to the companies innovating in language today”, says Lane Greene in his column in this issue of TAUS Review, “no-one has achieved anything like that roll-up…” He refers to Uber and “Uberization”, a neologism that has become synonym for disruption of traditional and fragmented industries.

Nothing of the same scale is taking place yet in the global translation industry. Mergers and acquisitions in our sector aim at increasing market share, but up to now they have not lead to real innovation.

Where then is the real big innovation in the translation industry coming from, we asked our columnists. From insiders or invaders?

Where is the real big innovation in the translation industry coming from? Insiders or invaders?

Luigi Muzii refers to the conclusion that Mariana Mazzucato arrived at in her book “The Entrepreneurial State”: no government, no innovation. We’d like to think perhaps that the companies we admire bring us great inventions, but in fact the research underlying them is often funded by governments. One example for all: the ideas and mechanics behind Google Translate go back to, among others, Verbmobil, a research project funded by the German Federal Ministry of Education, Science, Research and Technology until the year 2000.

The project was aimed at giving Germany a top international position in language technology

What the global translation industry needs now is ambitious government-funded research programs that exceed national borders. In analogue to the Human Genome Project, in the past we coined the idea of a Human Language Project in various TAUS conferences and articles. The Human Genome Project – after thirteen years of research funded by European and US governments – delivered the first documented human DNA in 2003 and since then the project ignited a revolution in human science. The original funds amounted to 2.7 Billion US Dollars. According to some reports, the benefits of the Human Genome Project now already hover around 1 Trillion US Dollars, or $178 for every public dollar spent. The discoveries delivered by the Human Genome Project have incubated a company like Illumina. Illumina’s machines for sequencing human DNA potentially rewrite the way we manage healthcare in the world. Disruptive innovation in optima forma.

In the meantime, the translation industry is just spinning its wheels.

Localization project managers are burnt out on repetitive jobs and highly educated translators are exploited to produce more for less. The sector prides itself as an enabler for globalization, but it is just scratching the surface. Ninety percent of the content and ninety percent of

What the global translation industry needs now is ambitious governmentfunded research programs that exceed national borders.

5

Leader

by Jaap van der Meer

the languages remain untouched, because….

Well, because of the lack of innovation and automation.

Although making its strides into the sector, machine translation is still lacking the confidence that it will deliver consistent quality.

Breakthroughs that require more than the bravery of even the most daring commercial players are needed. It seems that governments everywhere now realize the crucial roles they have to play in overcoming the traditional trade barriers. Japan is injecting the industry with funding for the collection of translation data.

With the Olympic Games coming to Tokyo in 2020, Japan wants to surprise the world with a population happily conversing in the languages of the tourists and business people visiting the country. (See TAUS articles on translation innovation in Japan)

Machine translation is still lacking the confidence that it will deliver consistent quality.

In Europe, the focus is on the Digital Single Market: the EC’s Connecting European Facility (CEF) is a major infrastructural funding program covering all things – like broadband internet, roads and bridges as well as automatic translation – that will help to truly open the internal European market. Bringing down the barriers within Europe (language being one of these barriers) could contribute an additional EUR 415 Billion to the European economy. And the “One Belt – One Road” program in China is another example of big thinking. Like the CEF program, the “Silk Road Economic Belt” program in China is aimed at removing physical, financial, cultural and language barriers.

There is no doubt that these major government funding programs will drip down in innovations in translation. The most ambitious governments are likely to stimulate the biggest innovations.

With the 40 Billion US Dollars that General

6

Secretary Xi Jinping announced in November 2014 to invest in the Silk Road Economic Belt it looks like China may soon be driving more innovation in the translation industry than other economies.

Our dream of the Human Language Project, the disruptive i n n o v a t i o n , that would fundamentally change the way we communicate on this planet, the type of innovation that opens trillion dollar plus economic opportunities and rewrite the rules in human evolution, requires that all governments collaborate. Because this is not very likely to happen we keep our feet on the ground and invite both industry insiders and invaders to come and present their best examples of innovation at the TAUS Annual Conference in San Jose, CA in October. See the Call for the TAUS Innovation Excellence Awards.

The type of innovation that opens trillion dollar plus economic opportunities and rewrite the rules in human evolution, requires that all governments collaborate.

Send your comments or questions to review@taus.net

A thousand different workflows. One Solution.

The Business Management System for the Translation Industry

www.plunet.com

7

Review of language business & technologies in Asia

by Mike Tian-Jian Jiang

Translation innovations that are leading the change and may help us surviving the change Innovation is uneasy when it comes to differentiation.

There are many different kinds of innovations, ranging

from

improved

customer

satisfaction

to

invented market demands, and in terms of the recent trend, the

“disruptive”

kind is the most welcomed

innovation.

By decorating an adjective “disruptive”, however, doesn’t really weigh clearer degrees on the spectrum of innovation, hence it is common to transform the question to the degree of inefficiency: how efficient is the target market1? Here we adapt this viewpoint for the market of translation, and then try to identify certain disruptive innovations that might be game-changers, in particular two factors of inefficiencies: human-resource and software ability.

Inefficient supply-demand match making Crowd-sourcing translation services has been quite popular in the recent couple of years. On one hand, crowd translation indeed increased supply, and potentially decreased the price. On the other hand, the information asymmetry between buyers and sellers remains almost the same.

Customers may still find it difficult to locate the best translator who will yield the expected outcome, and vice versa.

What if a match making can be done systematically and semiautomatically?

While it is a general question to most businesses, translation can be even tougher when the work of conduct requires further “translation”. Typical tactics to tackle this problem include categorizing source texts, relying on translators’ reputations, assuring quality by post-editing and/or proofreading, etc.

1 https://medium.com/@moonstorming/whyinefficiency-is-needed-for-an-innovation-to-be-disruptive8994845b09d9

8

What if a match making can be done systematically and semi-automatically? That is one of the questions DuoLingo is trying to answer.

Luis von Ahn, the creator of reCAPTCHA, started DuoLingo. According to the very first sentence of its Wikipedia page, DuoLingo is “a free language-learning and crowdsourced text translation platform.” To comprehend how language-learning and crowd translation align, it might be easier when we first understand how reCAPTCHA works. Like the original CAPTCHA, reCAPTCHA asks web surfers to type words shown in images, to prevent bots accessing restricted web pages.

Unlike CAPTCHA, r e C A P T C H A presents two sets of word images.

One of which the answer is known in the system and another is not yet “translated”.

Optical Character Recognition (OCR) doesn’t work on images like this. The idea is to utilize the wasted human effort on entering CAPTCHA, so the entered

reCAPTCHA eases the pain of normal validation that is usually laborintensive.

Review of language business & technologies in Asia

by Mike Tian-Jian Jiang

words can actually be the answer of failed OCR attempts. When it comes to the efficiency of digitalizing paper-based archives, reCAPTCHA eases the pain of normal validation that is usually labor-intensive. One may ask that if the hard-to-OCR text image didn’t come with an answer, how can reCAPTCHA be trustworthy?

The trick is that if the known image got the correct result, it is usually true for the unknown one. Furthermore, the same unknown image can be presented to different users, so a voting system could help as well. By replacing every appearance of image, word/text, and OCR on the above description with source language, target language, and translation, it will start making sense to remodel reCAPTCHA into DuoLingo2.

One obvious problem remodeling The demand with reCAPTCHA into a of translation translation service is is currently that a bilingual web from news surfer is as much sites like an urban legend as a monolingual one.

CNN and BuzzFeed. So DuoLingo began as a free languagelearning platform with gamification traits, with successfully significant amount of learners. The top learners were asked to translate some new texts and vote on other people’s translations. New texts could then be used as new learning materials.

As DuoLingo revealed, the demand of translation is currently from news sites like CNN and BuzzFeed. Despite the fact that this business model is still in its early stage, it will be not surprising that eventually texts in different genres and domains will join this learning-translation feedback.

Recently DuoLingo launched a language certification service and, in my opinion, this complimentary business model will strengthen the learning-translation feedback and build a 2 http://www.businessinsider.com/luis-von-ahncreator-of-duolingo-recaptcha-2014-3

healthy cycle by inverting the supply-demand dependency every once in a while, so buyers and sellers will both know what they can get.

For example, BuzzFeed first earned some extra bucks by selling their translated news articles to DuoLingo as learning materials, and later DuoLingo may sell more translated news articles back to BuzzFeed with quality no worse than the learning materials, while learners may get certified and then motivated to join the translation industry.

Inefficient corpus annotation for machine translation model training If we swap reCAPTCHA’s image, text, and OCR elements with “machine” translation terms, would it be as promising as DuoLingo? One One major major issue lies in issue lies in the preparation of the preparation learning materials, of learning or annotated materials. corpus in terms of machine translation.

For DuoLingo, it is reasonable to leave the alignment problem (as in determining which source phrases correspond to which target

9

Review of language business & technologies in Asia

by Mike Tian-Jian Jiang

phrase) to learners. For common machine translation training, however, it is required to include alignment as one of the corpus curation tasks. We want machine translation models to remember minimal phrases, and their translations, to fluently generate full translations from them no matter whether the exact source sentence is seen or not, just like what qualified translators can do. One may notice how similar this task sounds as in reCAPTCHA-to-OCR model training scenario.

The unfortunate difference here between OCR and machine translation is that phrase alignment is not always straightforward.

One component from a source phrase could be missing in its target phrase, not to mention that not every language in the world comes with whitespace delimited word boundaries.

Typical machine translation modeling nowadays involves a module of automatic alignment, such

as GIZA, which requires carefully handcrafted training data. Something that needs careful handcrafting often implies inefficiency, and an even worse situation of alignment is that it has to be done by experts, and the market of this kind of experts is limited, so either reCAPTCHA or DuoLingo won’t help much.

What if machine translation modeling can get rid of the burden of alignment?

Thanks to the advance of deep learning, there might be a solution.

A series of researches led by Yoshua Bengio has shown that recurrent neural network, also known as deep learning (at least one major type of it), may be a potentially good alternative.

Instead of aligning source-target phrases strictly, this socalled neural machine translation performs “soft” alignment3.

Neural machine translation attempts to imitate how human translators do their jobs.

If we try our best to describe what a soft alignment is, we could say that neural machine translation attempts to imitate how human translators do their jobs: searching for the best fit translation for a phrase in their minds, without trying to align every phrase in a strict one-to-one fashion. As one may see in the figure of the demo site, while old-fashioned one-to-one alignment is in black lines, neural machine translation can also take grey-scaled lines into account.

It could be somewhat confusing that more alignment options seems to be more complex.

At first glance, it is. However, neural machine translation’s soft alignment doesn’t need human experts to annotate those multiple

Typical machine translation modeling nowadays involves a module of automatic alignment.

10

3

http://lisa.iro.umontreal.ca/mt-demo

choices in grey-scaled lines. Neural machine translation doesn’t ask for a single standard answer of alignment. It just learns to weigh its options. If we rephrase this nice feature in a translation industry’s way, one may say that the only thing neural machine translation asks for is translation memory.

If they are so magical, how come they haven’t taken over the world?

Both DuoLingo and neural machine translation have their weaknesses. DuoLingo doesn’t really care about every perspective of translation quality, it only cares about macro factors, such as consistency between learning material and translation demand. Acquiring learning materials of non-alphabetical languages to bootstrap the whole cycle is not a trivial task.

machine translation, too. All the fine-grained specifications one may encounter in translation business will be still there for quite a while.

That being said, disruptive innovation probably shouldn’t be seen as a savior of the industry. It is more like a shiny new armor or sword to help common mortals to survive in the constantly changing environment.

Send your comments or questions to asia@taus.net

A similar obstacle can take place in neural

Machine Translation System

Boost the value of your TMs! Lower costs

Shorter delivery time Increased productivity

Powered by MorphoLogic Localisation

@globalesemt www.globalese-mt.com

11

Review of language business & technologies in Africa

by Amlaku Eshetie

Ethiopian languages and the need for translation within languages Ethiopia

is one of the many linguistically diverse

nations in

Africa. It

houses over

85

vernaculars

of which many do not have a lot of speakers or have developed writing systems.

The most dominant three languages are Amharic, Oromo (Afaan Oromo), and Tigrigna. These languages correspond to the three large and dominant ethnic groups

This

– Amhara, Oromo

would

show

that

and

Tigre,

language

respectively.

hegemony

often

reflects or depicts socio-political hegemony.

Amharic, the descendant of Ge’ez, has been the language of everything for a long time – administration, trade, education, communication, etc. The entire nation was learning Amharic at school, work and in life.

It is only recently, since the formulation of the current constitution - 1994/95, that it was limited to serving as a federal working language and a language for the whole nation. Since then, many of the other vernaculars got the status of a working language and/or language of education in their respective regions.

The constitution is now 20 years old, and in those 20 years (nearly one generation) many local languages have developed writing systems, produced literature, and accumulated knowledge, which keeps each language community self-contained and less interrelated to the others in many respects. Children are taught in their mother tongue to the end of their elementary education, and in some regions, to the end of junior secondary education. The administrative language and the language of communication in almost all regions is the respective regional vernacular.

In contrast to this, there are people living and working in regions where the local language might not be their mother tongue. Hence, they either do not speak the language or do not want to speak that language at their workplace, at court, or anywhere else the situation demands they speak that language.

12

In addition to this, people travel from region to region to visit friends and family, for work or they take residence in another region. So they need to learn the culture and everything else about the other language community; the regional governments and businesses need to exchange information, values, experiences and results or decisions.

Therefore, all these needs and evolving problems entail that the gap between and within language communities in Ethiopia is getting bigger and bigger.

In order to bridge this gap, we may need a political and long-run decision or policy. However, I am arguing that translation is the solution at hand.

The gap between and within language communities in Ethiopia is getting bigger and bigger.

Tourists, investors, researchers, media, etc. use translators of various local languages at various occasions. For example, the local media translates news from other regions when they report it; tour operators hire a local translator

Review of language business & technologies in Africa

by Amlaku Eshetie

of technical and organizational support for the industry to emerge are the most significant limitations.

While the latter two are the same for almost all of the local languages, a lack of skilled and experienced translators is a major problem especially in the rare or less developed local languages. A good example is my experience with the Kunama language. A few of my clients (for Amharic, Afaan Oromo,Tigrigna) also asked me to provide them with Kunama translations.

when they take their tourists to a tourist destination in another region; researchers use an amateur or professional translator to get their survey questionnaires or data collection tools translated into the required vernacular; investors likewise get their working documents, contracts, manuals, etc. translated into the language of the region they operate in.

Thus in a country such as Ethiopia where large and many regions exist with their own independent and functional languages, translation is vital to bridge the gap and to allow the various l a n g u a g e communities to stay connected.

A lack of skilled and experienced translators is a major problem especially in the rare or less developed local languages.

Challenges to the remedy As mentioned in my previous articles, there are several limitations to the translation industry in order for it to serve as a bridge for the different language communities. A lack of skilled and experienced translators, the society’s attitude towards translation as a profession, and lack

I tried to use the available networks which clients use to find Kunama people, but none of them brought results.

Finally, one of the clients was able to find a translator of Kunama who lives in the USA, but when I contacted her, she seemed to be overqualified or too busy to work for me.

The attitude of the society is also a hindrance for translation to emerge as a profession and business.

The attitude of the society is also a hindrance for translation to emerge as a profession and business. This in turn contributes to the non-availability of translators: when it is not considered as a profession, clients do not pay well for translators and translators are not attracted to consider translation as a business or profession. As a result, the business or the profession is non-existent or not well developed in Ethiopia.

Remedy to the challenges I believe the solutions are not unreachable. The first thing to drive will be the market itself. As investment flow increases, business grows and the need for interaction or information exchange among cultures and language communities develops, demand for translation services starts to pull the translators. Another salvation for the translation industry in Ethiopia may be an intrusion of industry gurus. If an established

13

Review of language business & technologies in Africa

by Amlaku Eshetie

company enters and sets the standard and the system for the market available in the country along with the global competition, then it may not take long to revive and become a booming business.

Both approaches worked together in a few African countries as far as I have seen. If anyone from South Africa, Egypt, Kenya or anywhere else reads this and wants to back up on this by commenting, I would truly appreciate that.

It may not take long to revive and become a booming business.

14

Send your comments or questions to africa@taus.net

why risk it Your data has never been so important, so why risk its loss?

Secure Localization Management • • • • •

Secure integration Secure translation management system Capita IL3 data center hosting Secure machine translation solutions ISO 9001/27001 certified www.capitatranslationinterpreting.com E: enquiries@capita-ti.com | T: +44 (0)845 367 7000

15

The Journalist’s Perspective

by Lane Greene

Insiders or outsiders “If I’d asked said ‘a faster

people what they wanted, they’d have horse’.”

The

Henry Ford is a staple of tech-themed conference talks, the kind where “disruptive” is on every tongue. It’s intended, with more than a bit of smugness, to show just how head-in-sand, inside-the-box and, well, not “disruptive” some other people are. A faster horse, indeed. quote from

Ford himself, then, might have been surprised that, a century later, the industry he built has delivered – what, exactly? Nothing more than faster cars with internal-combustion engines.

Slow progress in electric and hydrogenpowered cars, and a century of dreaming of personal flight, hyperspace or teleportation (think The Jetsons, Star Wars or Star Trek) later, the majority of rich-world people wake up in the morning and turn the key in a petrolfueled car to go to work.

What if the best technology has already arrived?

People like their petrol cars. It is hardly the only technology that persists against apparently long odds. Despite the fact that everyone in the rich world has a smartphone with a big, sharp, contrasty screen in their pocket at all times, most people still prefer to read longer things improbably printed on dried sheets of pulped tree, a technology little improved since the Han Dynasty.

What if the best technology has already arrived?

So the question of where innovation in the language industry will come from is an open one. Will innovation be gradual, or sudden? Will it come from a company that already exists today, or from one yet to be founded? Will it come from someone who dreams in Portuguese, Dutch and Persian, or someone who dreams in Perl, Ruby and C++?

(Or, as the title of a recent New York Times Magazine article put it, “is translation an art problem or a math problem?”) Finally, will the

16

disruption come in translation itself—or in the business models of its providers?

This is the view of an outsider. In a recent article in The Economist, I tried to tackle the same subject, but by the time I had explained the shape of the language industry today, for the benefit of a nonspecialist audience, I hardly had space left to speculate about tomorrow.

Herewith, some more detail.

Will innovation be gradual or sudden? On the language technologies themselves, especially machine translation, the improvement of late has been gradual—and looks as though it may even be plateauing.

In my last column, I reported that MacDuff Hughes, the head of engineering at Google Translate, confessed that he did not expect more data to improve his engine’s performance. At this point, the next jump forward in quality looks likely to be more sudden

Will the disruption come in translation itself—or in the business models of its providers?

The Journalist’s Perspective

by Lane Greene

than gradual, as the next leap in innovation appears—whatever form that will take (and whenever it will happen).

Is a currently existing company in a position to bring that big innovation? Or, in today’s required cliché, I must ask and answer: who could be the Uber of translation? That company does not yet seem to exist.

The reason Uber has become the tech-journalism cliché that it has become is the speed with which it rolled up its space. With all respect to the companies innovating in language today, no one has achieved anything like that roll-up— in a fragmented industry that could sorely use some consolidation. Companies like Microsoft and Google have huge resources, of course, but it can be hard to be groundbreaking inside big companies with legacy products to think about. Perhaps the next big leap will come from an outsider, perhaps another clever graduate student in the mold of Sergey and Larry, building server racks out of Legos and building the world’s next great company in their spare time.

Will the breakthrough come from more traditional language experts or the mathheads?

Statistics-driven engines currently dominate machine translation. At a recent MT hackathon, Gideon Lewis-Kraus reports in the Times Magazine article mentioned above, “nobody betrayed any interest in language as such… they were there for the math.” One participant told him “It’s surprising how little it helps to know another language.” If, as MacDuff Hughes said, more data cannot help much, then the math-types are left trying to crunch it better. They may yet succeed gradually. But some marriage of math and language is more likely to produce a jump in quality. Having divorced the linguists in the past, the mathheads may now find themselves making up with them, and having a go at a second marriage, for the sake of the kids—their products.

Where the technological breakthroughs will come from is not clear. But another thing seems much clearer: the language-services industry is

ripe for disruption in its business model. As an outsider, I have the luxury of saying what many friends in the industry have told me privately away from the conference stage: the people in this industry are brilliant (not to mention fun), but much of the work is dull. The core personnel of the traditional language-services provider, the project managers, burn out and turn over quickly. (And no need to take my anonymous bar sources’ word for it; the industry c o n s u l t a n t Common Sense Advisory says the same.) Where the rubber meets the road, with translators themselves, rates are squeezed and times are tight for the mass of non-elite translators.

Will the breakthrough come from more traditional language experts or the math-heads?

The business that could automate and streamline the most boring parts of the job will make everyone in the chain happier.

Translators love French, not file formats. Clients want speed, flexibility and transparency, not the bad news that a job will take months and that this simply can’t be helped. And the good news here is that unlike fully automated, high-quality machine translation, no huge intellectual breakthrough is required. The companies of today can certainly survive by innovating their own business models—a faster horse, a faster car—and make a lot of money if they delight customers by doing so. If they don’t, a disruption will eventually come from outside. It’s only a matter of time.

Send your comments or questions to journalist@taus.net

17

The Language Perspective

by Nicholas Ostler

New Lamps for Old Offering

to make sense of someone else’s language

is an old profession.

A word for interpreter, still current in Turkish (tercüman = archaic English dragoman), is first attested in Assyrian of 1800 BC (targumannum); while its synonym in Sumerian emebal goes back a further 500 years. (The first known Assyrian interpreter worked with Hurrian, in what is now south-eastern Turkey, while the Sumerian one dealt in the language of Meluhha, to the south, possibly the Indus Valley1. So to innovate in this business will be a testing ambition.

But success is most likely to be achieved if the focus is not on means, but on ends.

Machine translation, as developed and applied, has mainly been for decoding use: that is to say, the translator is trying to gain knowledge from a foreign source. This is one reason why poor-quality text is sometimes acceptable as an output: at least it will be a much less foreign text than the input, and possibly accessible to the user’s common sense. Encoding uses – which attempt to make the knowledge in a familiar text accessible to foreigners – are much less tractable: why should the target audience, who did not choose to have the text translated, bother with an imperfect product? As a result, the audience best served is the one that uses the regnant lingua franca, in our own era overwhelmingly English, though the same argument would also favour translation into other big-beast language communities, who can plausibly see the world accessible in their language as world enough. Above all, the availability of such translation spares them the massive effort of conscious language learning.

Machine translation, as developed and applied, has mainly been for decoding use.

1 I.J. Gelb, The word for Dragoman in the Ancient New East, Glotta 2:1 (1968)

18

Language learning is another field fit for technical support (from conventional grammars, dictionaries and tutors through to CALL). But the effort is only repaid if the student foresees massive use of the new skill: this means that once again the regnant lingua franca will be the preferred goal, providing lots of hitherto inaccesible texts (and people!). Speakers of minority languages will aim to widen their scope by joining the second-language learners of this (already vast) language community.

So both these inter-lingual technologies – translation and learning – tend to favour the regnant lingua franca as their target language.

The incidence and level of costs is different but in both cases, the implicit aim is to make the “foreign language problem” go away: wider communication is to be achieved, largely by adding to the effective reach of the language that is already dominant. To the languages that have speakers and writers worth attention, more

The audience best served is the one that uses the regnant lingua franca.

The Language Perspective

by Nicholas Ostler

shall be given. “Foreign” – i.e. minority, less dominant – language communities gain little or nothing from use of these technologies.

But suppose the aim were not to by-pass the foreign languages, seen as obstacles to communication, but to empower them, as distinctive approaches to knowledge, hallowed by traditions that are open-ended, and in no way yet exhausted. Is it possible to give them aspects of the power or investment that are already being devoted to the lingua franca, and (to a lesser extent) to the other big beasts in the linguistic jungle?

Suppose the aim were not to bypass the foreign languages, seen as obstacles to communication, but to empower them.

This is not pure idealism. One area of language technology where this is happening is the IDN (International Domain Names) project of ICANN (the Internet Corporation for Assignment of Names and Numbers) which regulates internet addresses globally2.

More generally, the Unicode project has long laboured to define a system of compatible and distinct codes for all the characters of all the writing systems of the world (even past and present). The result is a rich and evolving system which transcends the narrow space of codes once laid out for ASCII, and then arbitrarily – and incompatibly – extended almost at random for other charactersets of the world’s languages.

The results bid fair to widen the effective use on digital platforms of the major non-English languages of the world. ICANN – in setting out the subset of Unicode scripts fit for use in toplevel domain labels – is explicitly committed to include all characters in use for sufficiently vital languages as measured by the Ethnologue’s EGIDS scale. The criterion chosen is grade 4: “in vigorous use, with standardization and literature being sustained through a widespread

2 <https://www.icann.org/resources/pages/idn2012-02-25-en>

system of institutionally supported education”.

As a result, the writing will be included for 572 (8.2%) of the world’s languages, but as written by 82.6% of the world’s population3.

This is one formal way in which investment is being made in new language directions. Less formally, but with immediate human content, is the range of new social media projects (blogs, Facebook, Tweets, corpora) in indigenous languages which are indexed and accessible at Kevin Scannell’s site4.

This empowering process can be promoted by application of computer-aided languagelearning (CALL), but only if it is applied imaginatively, in combination with direct human content, which might involve a mix of conferences, festivals, summer schools, as well as conventional language course teaching.

Machine Translation, and other software aimed at directly converting a stream in one language to another, seems less promising, just because it is focused on converting the cognitive content of a message or text, and so replaces, rather than builds, the cognitive skills that underlie command of a language.

Marginally, the product of such encoding might be useful after the event, if the original text 3 4

<https://www.ethnologue.com/statistics/status> <http://borel.slu.edu/nlp.html>

19

The Language Perspective

by Nicholas Ostler

were a familiar one: it could then be used for rapid reading, much as one might improve (to an extent) one’s familiarity in reading Latin by scanning a volume of Harrius Potter or Asterix in Latin translation.

The key is to facilitate, not replace, the development of skill – e.g., to provide a quick check on reading a passage in strange script, to show the structure of words or sentences, to look up new words.

The key is to facilitate, not replace, the development of skill.

The result would be to give a more penetrating sense of the language to which the learner aspires. As Ken Hale exclaimed, on first understanding the perversities of Tjiliwiri, an Australian language of initiates, and the joy in mastering a ritual system: “You certainly have something here!”1

1 A note on a Walbiri tradition of antonymy, in Semantics, ed. Steinberg & Jakobovits, Cambridge UP, 1975.

Send your comments or questions to language@taus.net

20

The Translator’s Perspective

by John Moran

Using translator activity data in CAT tools to better understand translation automation Most

translation agencies do not care how fast a

translator works, as long as they meet the deadline and the quality of the translated text is satisfactory.

Most translators who are paid by the word do care.

Certainly, it can take hours to chase down a single term, but a freelancer will soon starve if this happens too frequently. Most translators and project managers plan projects on the basis of between two and three thousand words per day.

The

number of

hours required to translate those words varies more than most people think.

In other words, for most unassisted translation, word price and quality are traditionally the only two metrics that count. However, linguistic technology is slowly changing this view.

Agency owners who were previously sceptical of the claims of MT vendors and organisations like TAUS are now trying to figure out how to manage post-editing projects to reduce translation costs. Some - mostly corporate end buyers have been asking for discounts for MT post-editing for some time.

U n f o r t u n a t e l y, these discounts are often granted without proper regard for a translator’s hourly earnings – disgruntled former post-editors are not hard to find at translator conferences. It is no exaggeration to say that the translation industry is still struggling to find its feet when it comes to establishing fair compensation models that take the volatility of MT into account.

Word price and quality are traditionally the only two metrics that count.

However, as Jost has pointed out in this column in the past, post-editing of fully machine translated sentences is far from the only game in town. In Transpiral (my own agency), a number of the freelance translators we work with use various versions of Dragon (an Automatic Speech Recognition application

by Nuance Communications) to dictate their translations into English and German. We find the technology has no noticeable impact on price, as the translator normally buys Dragon himself. However, we do support its use as it leads to faster and more terminologically consistent translation. This is because fewer linguists need to work on large documents with short turnaround times. Also, translators who translate fast and well using dictation tend to be experienced to the point where they do not need to do much terminology research. To put it another way – they know their stuff.

These quality upsides are great, but they have to be balanced against the cost of careful bilingual review. Constant vigilance is required to prevent sometimes hard to stop errors we call “dictos”. For instance in the previous sentence, “stop” should have been “spot”. The two words sound like each other and they look similar.

This (faked) dicto changed the meaning of the sentence, but it is still

The translation industry is still struggling to find its feet when it comes to establishing fair compensation models.

21

The Translator’s Perspective

by John Moran

plausible, so a monolingual review of the target sentence does not suffice. This means that when dictation is involved – even for trusted translators – a reviewer needs to check both source and target sentences for an unintended change in meaning. Another category for TAUS’s Dynamic Quality Framework, perhaps?

tests could be carried out. Our software is an analytics platform to test a range of measures that can impact on translator productivity.

Though it is based on OmegaT, a free opensource CAT tool we could adapt to our needs, we hope to collaborate with other proprietary CAT tools on sophisticated time reports.

Finally, interesting approaches to interactive or sub-segment level MT are starting to appear.

Translators who manually add terms to an autosuggest dictionary or termbase over the years have long reported very real productivity gains. Our success here has been mixed.

Terminology extraction is manually intensive and doesn’t yield enough proposals.

This means terms are hard to bootstrap for freelancers at the LSP level to the point where any negotiation regarding a discount would be reasonable. We still do it on large projects as a consistency measure, but not on smaller ones. MT-based approaches like the MT Autosuggest plugin in Trados Studio, and statistically generated frequent terms like the Muse function in MemoQ, seem popular with some translators on some content types but not with others. It is the first thing some translators switch off. On the whole, agencies like my own that mainly employ freelance translators with in-house review don’t really know very much about what goes on in most offline or desktop-based CAT tools. This knowledge gap has been the focus of my research in the ADAPT Centre for the past five years.

Currently we can measure the impact of an MT, automatic speech recognition (using Dragon) and soon, we hope, a promising new approach to interactive MT where proposals are provided automatically at a sub-sentential level. Linguistic technology aside, time data can also help with vendor management by helping PMs to match translators with content that suits their skills and experience. The package, which can be licensed through the ADAPT Centre, comes with an interoperability suite to support older versions of Trados and a range of TMS systems, including SDL TMS and SDL WorldServer. Unsurprisingly, it also works particularly well with GlobalSight, the open-source system published by Welocalize, our development partner on the iOmegaT Translator Productivity testbench.

Finally, interesting approaches to interactive or sub-segment level MT are starting to appear.

The analysis of User Activity Data to get a better understanding of the process of translation in CAT tools is a young field. Projects like MateCAT and CasmaCAT and our own collaboration with Welocalize on iOmegaT have resulted in (we hope) some interesting publications where relatively large-scale translation productivity

22

However, we are not alone in terms of translation speed reports. MemoQ has one and Wordfast Pro will soon have one too. For translators who keep their CAT tool versions up-to-date, Trados Studio 2014 will soon have a plugin called Studio Time Tracker that measures translation speed. These reports are

The Translator’s Perspective

by John Moran

private to the translator and can only be shared with the translator’s consent.

This may be a good thing for translators as tracking translation speed in words per hour can be dangerous to their interests. Some LSPs already use translation speed data to negotiate discounts, even when no MT is in play. One agency from New Zealand, with its own web-based CAT tool, even claims this as a selling point. Though time reports are not common across most systems with webbased CAT tools, economically the publishers behind them are less beholden to the needs of freelance translators as they are not their paying clients. This means they cannot vote against unpopular features with their wallets (though they can refuse to work in them). For example, a free web-based CAT tool developed under a European FP7 project called MateCAT has a time report with (as yet) no opt-out mechanism for translators.

The legalities of tracking working speed vary from country to country. In Germany during the late 90s I worked for a consulting company called Cap Gemini Ernst and Young for a number of years. It was never easy to measure employee productivity in large companies, as the permission of works councils was required.

I understand the same is true in France, but not in my home country of Ireland. For freelancers it seems it is perfectly legal to record working

John Moran John Moran graduated with a degree in computer science, linguistics and German in 1997. He is a former translator,

lecturer

in

translation,

software engineer and is a co-owner of

time.

It is in fact quite easy to measure the impact of MT on a translator’s working speed. You simply remove the MT from random segments, as we do in iOmegaT. A similar approach is taken in TAUS’s DQF platform. We call this SLAB testing (short for Segment Level A/B). Case A is translation from scratch and case B is translation using the MT. The approach is similar to one used in IBM’s internal TM/2 CAT tool. Once it has been established that translators are faster with MT, SLAB testing is not required and word price discount can be safely negotiated. It is important to keep an eye out for a drop in MT quality using at least intermittent speed reports. This can happen when a client introduces a new product feature or a technical writer retires.

It is in fact quite easy to measure the impact of MT on a translator’s working speed.

Is time reporting intrusive? I don’t feel it is – but of course, I have skin in the game. In my defence, I would say that working with hundreds of translators over nearly 20 years I have noticed that very few translators take more than a passing interest in their productivity.

The data I see from my own agency and in our published productivity tests with iOmegaT suggest that much can be done to improve it. My hope is that translators will begin to realise this and take a more empirical approach to analysing how they translate, as well as a more open-minded approach to technologies like dictation software and variations on the theme of MT.

Transpiral, an LSP in Dublin specialized in German technical translation and Irish Gaelic. He is a translation technology consultant and research fellow in ADAPT - a large language technology research centre in Trinity College Dublin - where he is working on a Ph.D. in computer science. His research interest is the analysis of user activity data in CAT tools.

Send your comments or questions to translator@taus.net

23

Join us on July 30 for conversation, a potluck dinner and pool competition

RSVP Here

24

The Research Perspective

by Luigi Muzii

Waiting for Landmarks In The Ten Faces

Innovation, to support his view of the inconsistency of user-driven innovation, author Tom Kelley quoted Henry Ford who said, “If I had asked my customers what they wanted, they’d have said a faster horse.” of

An Amelioration Orgy This could help explain why true innovations most often come from outsiders and why Evgeny Morozov’s target in To Save Everything Click Here is an “amelioration orgy” that he associates with Silicon Valley, whose favorite slogan, in the past few years, has quietly changed — he says — from ‘Innovate or Die’ to ‘Ameliorate or Die.’ Then again, in The Entrepreneurial State, Mariana Mazzucato outlines a number of case studies in different fields to show that most of today’s technologies (from the Internet to GPS, from touchscreens to NLP applications ) are heirs of earlier publicly funded projects. Many other of the most significant innovations of the last 50+ years (from photocopying to laser printing, from the personal computer to the WIMP paradigm and the GUI, from Ethernet

Go and face the future

Come with me and introduce me

to ubiquitous computing) were publicly funded too and came from a single place in a fairly short time lapse.

Innovation in Translation: Navel-gazing Google launched Google Translate, an SMT engine, in October 2007 after years of using SYSTRAN’s RbMT engine developed with massive investments from the U S D O D and the EC. More or less overtly, GT was born to support the company’s core business, online advertising, and unintentionally disrupted one of the last craftsman businesses. Today, like it or not, GT has become the benchmark for machine translation and translation tout court, but Google remains a major translation buyer.

The lack of innovation in the translation business comes from pulverization.

The translation community has been thriving for decades now on technology and basks in the belief of operating in a high-tech business, but most players are actually oldfashioned. The lack of innovation in the translation business

25

The Research Perspective

by Luigi Muzii

comes from pulverization, a disinclination to collaboration, and a highly conservative nature of players. Even the regular mergers and acquisitions have never reduced pulverization nor produced any real innovation. They aim at complementing customers, offerings, and extend market penetration, but very seldom at acquiring greater financial strength to fund innovation.

The last real innovation in translation were translation memories, a quarter of a century ago, as a development of the machine-aided translation effort following the (in)famous ALPAC report.

Even TMS are a very peculiar abridged application of project management software, afar from workflow management systems, which remain extraneous to the translation business, although they could be a leap forward.

More than the invisible hand, it is realistically predictable that the longa manus of some technological outsider will bring unintended benefits resulting from any innovations.

Quality is a perfect example of navel-gazing in translation, especially in the academic field. It is a most debated topic and yet the translation community is still at the I-knowit-when-I-see-it stage. However, it is almost ludicrous to figure a judge making a landmark case by sentencing about the quality of translation.

The last real innovation in translation were translation memories, a quarter of a century ago

Today, the value of an industry depends more on its technology, financials, and the image projected than on the value of its products.

Despite technology, automatic evaluation of translation quality is yet to come, while there is no innovation in yet another and another quality standard. Should these new efforts see

26

the light, they would already be obsolete, and not raise the overall quality level, as the other standards. Unmistakably, they are meant to be inclusive, comprehensive, exhaustive, and universal, and are presented as simple, easy and quick to apply.

Technology and Life Te c h n o l o g y has always scared people, even more so now that also whitecollar jobs (that seemed away from any technological threat) could be lost to machines.

Big data, advanced analytics, artificial intelligence and robotics will increasingly replace human workforces

In Jeremy Rifkinâ&#x20AC;&#x2122;s view, big data, advanced analytics, artificial intelligence and robotics will increasingly replace human workforces across all sectors of the market economy as an IoT infrastructure is developed and the marginal cost of labor drops to near zero.

A 2013 study of Oxford Martin School estimated that 47% of US jobs are vulnerable to computerization. Two years later, another study envisioned that the digital economy might cause secular stagnation.

There is nothing more slippery than forecasts, and yet this seems to be a mere anticipation of reality.

In 2004, Frank Levy and Richard Murnane excluded that a driverless car could ever be possible. A decade later, Google cars have logged nearly 700,000 autonomous miles (1.1 million km), driver assistance systems have gone to mass production, and automatic parking, collision avoidance and cruise control systems are no longer a frill.

In The Glass Cage, Nicholas Carr envisions

The Research Perspective

by Luigi Muzii

a future in which the human factor declines and echoes Tesla Motors’ Elon Musk warning, “With artificial intelligence, we are summoning the demon.” By the way, Tesla Model S is the first mass-production vehicle equipped with autopilot.

Carr also recalls Katherine Hayles observing in her 2012 book How We Think that when her computer goes down or her Internet connection fails, she feels lost, disoriented, unable to work; in fact, she feels as if her hands have been amputated. In Hybrid Reality, Ayesha and Parag Khanna wrote that our relationship w i t h technology is already beyond the purely instrumental level to enter the realm of life. In other words, we are the technology we use, and

We are the technology we use, and the increasing, unknowledgeable intimacy with technology dictates and possibly biases expectations.

the increasing, unknowledgeable intimacy with technology dictates and possibly biases expectations.

Datafication and Deep Learning The brain’s computing speed is the equivalent of 1 kHz. My laptop computer runs at 1.90 GHz. Latest processors run at 3.5 GHz (1 GHz = 1 million kHz). Nonetheless, to accomplish the same number of a brain’s binary operations per second, a computer should be as large as a room, consuming the same power as a small town, while a human brain requires only 40 watts out of the 1,000 the whole body consumes.

Tech giants are pouring big money into deep learning.

Innervation makes the difference, enabling parallel rather than serial brute-force computing. In addition, thousands of years of (continuing) evolution have enabled the development of heuristics, while computers are still stuck to algorithms.

That’s why tech giants are pouring big money into deep learning to allow computers to operate following a model from example inputs and make predictions, with heavy-duty statistical analysis of huge amount of data.

So while IBM’s Watson was already set up to analyze health records, looking for medical insights, last year, Narrative Science launched Quill Engage, a free Google Analytics application to deliver plain-English, narrativestyle reports, which Forbes is already using to cover basic financial stories.

Epistemic Asymmetry The translation business is affected by a classic information asymmetry, while translation business players are affected by epistemic — and possibly non-cooperative — asymmetry.

Epistemic asymmetry makes many in the translation business feel threatened by

27

The Research Perspective

by Luigi Muzii

machine translation, as if it were about to wipe them out.

Some of these people claim for re-imagining technology, while translation technology disrupted the translation business precisely when business players started using it.

Not surprisingly, according to a One Hour Translation survey of May 2015, more than 80% of consumers can tell the difference between a professional human translation and a machine translation output.

Another silent and yet fast-pacing disruption is on its way, coming from web-based platforms, many of which are, again, from technology companies rather than translation service companies.

The most advanced platforms are designed as virtual market places.

The translation community is lagging behind and a foreseeable future is not in machine translation, but in aggregators.

However, due to the many facets of translation, a real disintermediation is far from being achieved.

From workflow customization to staffing, we have a long road ahead.

28

The translation community is lagging behind and a foreseeable future is not in machine translation, but in aggregators, similarly to what happened in tourism. Again, these will come from translation customers who are eager to find a way to reduce information asymmetry, increase speed, and easily set prerequisites to anticipate quality.

Maybe automation enhancements will eventually involve workflow and quality, with customers being able to buy or lease platforms including full workflow customization and automatic profiling, sampling, and quality evaluation.

Automatic quality evaluation could already be at hand, through detection of bad samples for reworking and ranking-based assessment with a combination of content profiling via previously collected requirements and inspection, and with linguistic evaluation revolving around a combination of correlation and dependence, precision and recall and edit distance.

Send your comments or questions to research@taus.net

memoQ.com

â&#x20AC;&#x153;memoQ is a very intuitive,

user-friendly, easy to learn, and easy to integrate translation tool. In addition, the level of support provided by their team is excellent.

â&#x20AC;?

Birgitte Bohnstedt FLSmidth A/S

29

Unmoderated Remote Usability Testing of Machine Translated Content

by Will Burgett

ABSTRACT While we contend not

appropriate

solutions,

Intel

machine

for

all

translation

of

today’s

(MT)

is

translation

MT to translate 2007. Intel IT recently completed a project to assess the users’ acceptance of the MT content published on the support website.

Intel IT along with our TAUS colleagues have driven and enabled a sweeping adoption of MT as a reasonable strategy for companies, governments, and organizations to reach their global users. Amidst this rapid growth of MT solutions, very little user experience research has been done on unedited MT content. has promoted using

knowledgebase content since

The barriers of cost and complex global logistics have kept internationally scaled usability tests for multi-

MT comparisons out of reach, but Intel IT recently completed a study using Unmoderated Remote Usability Testing (URUT) software, and found URUT not only overcomes the barriers of language

cost and global resourcing but it supported the hypothesis that there is a high-degree of acceptance of

MT

content among users.

These

studies have

shown that there is an acceptance among technical and non-technical customers alike for the un-edited

MT

content that

Intel’s

solution provides.

This

tolerance was striking in that it shows acceptance of un-edited some

URUT

MT

sourced content even from within

traditionally software

effective

solution

resistant

offers to

geographies.

organizations

conduct

focused,

a

The cost-

in-depth,

statistically significant, multilingual studies of their machine-translated websites without the expense of moderators, recruiters, and complex usability labs.

The Problem December 2014 was the seventh anniversary of the launching of Intel’s first machine translation (MT) system. The language was Latin-American Spanish and it was integrated into the platform that published content to the Intel Customer Support website. Intel IT designed the workflow to automatically detect any additions and changes to the English content every twenty-four hours, queue them for translation, and publish the unedited output to the Spanish website. As a result of this deployment, ninety-five percent of the Spanish

30

was translated by machine and five percent by humans (new products, safety, and legal, content).

Since the launch of Latin-American Spanish, Intel has developed and integrated many more machine translation languages into the Intel Customer Support website: Brazilian Portuguese, Simplified Chinese, Traditional Chinese, Korean, Russian, French, and Italian.

Eventually many of these MT language engines were also integrated into the Intel Support Community Forum to give customers the ability to translate, in real time, forum postings from other customers or Intel support agents.

Intel and other organizations both inside and outside of the TAUS community have fueled and driven tremendous growth in unedited machine translated content. These investments have been made based on a financial principle that MT is the only cost-effective solution to bring massive amounts of content to a global audience. Yet very little usability testing and customer experience studies had been conducted and published. A few well-done usability studies on machine translated content using eye-tracking techniques published by CNGL (for example, Doherty and O’Brien, 2013) exist, but considering the growing dependency on machine translation to deliver all sorts of content, Intel decided it is now time to better understand the MT and user experience.

If we could demonstrate acceptance of MT content, we could build trust in previously r e s i s t a n t regions and parts of the industry, which could allow us to quicken the pace of MT deployment. But how to conduct statistically rich studies in many languages, with globally dispersed users? With funding for translation

If we could demonstrate acceptance of MT content, we could build trust in previously resistant regions and parts of the industry.

Unmoderated Remote Usability Testing of Machine Translated Content

by Will Burgett

already in short supply, how to secure the budget for resources, facilities, and equipment to do traditional lab usability studies on a global scale? To solve the problem the Intel team sought out some innovation to address the needs.

The History Of Machine Translation at Intel Corporation Intelâ&#x20AC;&#x2122;s impetus for adopting machine translation was based on a number of factors: 1. Volume of Content. Intel Customer Support had over 8,000 English source files in its content management system.

These documents were published to the website and then periodically updated as information changed.

2. Languages. To support its major global markets, Intel had to deliver translated

Over time using machine translation had a huge impact on the depth and breadth of translation, cost savings and avoidance, and the speed of translation. These improvements were easily measured. For example, as a result of translating the entire site into Spanish, visits to that website quadrupled in less than six months (from about 22,000 per month to 90,000 per month).

An interesting challenge was how to measure the customer experience of unedited machine translated content. First, a disclaimer in the

content in at least ten key market languages.

3. Time. Typically it took about ten business days for a complete localization cycle (1200-1500 files per cycle), even with the use of workflow automation technology.

That cycle time meant that the translated content was at least ten days out of sync with the English content.

4. Budget. Because of the huge volume of content, we had depleted the budgets for human translation. For even the most important languages, there were only enough funds to translate about 15% of the English into any given target language.

5. User experience: With only 15-20% of the English content translated, users would have a poor experience using a patchwork of English and translated content.

target language was placed on every page to inform the user that this is automatic translation: â&#x20AC;&#x153;This information is a combination of a translation by human and machine translation by computer of the original content for your convenience. Updates and changes are translated and made available in 24 hours. This content is provided as general information only and should not be considered as complete or accurate.â&#x20AC;?

31

Unmoderated Remote Usability Testing of Machine Translated Content

by Will Burgett

To empower the user, translated documents include a link to the corresponding English page if the user is not satisfied with the translation.

Also from the very beginning we developed and deployed a survey widget that appears on every page. The survey asks one question: “Did you find this information helpful? Yes or No.” The survey widget also asks the user if they have any additional feedback and provides an area for them to write their comments.

Over the next few years, the data from our survey widget showed that the level of customer satisfaction from our machine-translated content was getting very close to English levels. After more than six years of collecting data, there were very few comments about the quality of translation. However, a short survey with comments was too one-dimensional to be useful in drawing conclusions about the usability of our machine translated content.

We needed much more dialog with customers and better usability data to clearly see user experience in more than one dimension.

The problem is that a typical moderated u s a b i l i t y- t e s t i n g approach requires the use of labs, resources to develop and conduct the tests, and expensive recruiting of typical users to take several hours out of their day to come to the lab to do the usability studies. On top of that, the expense and physical location of the usability lab limited the number of participants and the languages.

After more than six years of collecting data, there were very few comments about the quality of translation.

So how could we overcome all of these limitations to gain a deeper and more dimensional picture of the user’s experience of unedited machine translated content?

32

The Solution Unmoderated Remote Usability Testing (URUT) is an ideal solution for global websites. URUT technology automatically presents tasks to participants and tracks their interaction, including navigation path, page scrolling, and click location. The data collected measure the important usability dimensions of: • Effectiveness • Efficiency • Subjective satisfaction URUT deals with the following issues that online user researchers typically must solve: 1. Collecting large samples of data to quantify usability and user experience. URUT generates larger user samples than inlab moderated usability testing, which are limited by cost and space.

2. Users can participate in their natural context, at home or at work, or on the road using their own computers or mobile devices.

3. Organizations with a global target audience can test in many countries at the same time without a large budget.

4. URUT allows you to conduct benchmarks among different competing sites and see who is performing well and offers the best usability and user experience.

5. We want to know users’ real goals and why they do what they do. Web analytics tools give us lots of data about what happened on a site at an aggregate level, but we still don’t know why it happened or how it relates to different user goals. The flexibility and depth of many of the mature URUT tools in the market can analyze the user experience at a much deeper level.

URUT software can conduct both usability and survey- based studies, including: • Web Voice of the Customer (VOC) • Intercept surveys • Using feedback tab • Mobile apps • Web remote usability testing • Mobile usability testing • Card sorting

Unmoderated Remote Usability Testing of Machine Translated Content

by Will Burgett

• • • •

Tree testing Screenshot click test Screenshot timeout testing Online surveys

The types of data that can be collected using some types of URUT software include: • Usability • Survey responses • Information architecture • Behavioral data • Video replay • Video feedback for mobile There are a number of companies that offer different URUT software and services and Intel selected one of the better known providers.

The selected URUT application supported localization in many languages and included all the study types that Intel was interested in. However, the actual study or test contents that we created for each study (survey questions, usability tasks, etc.) had to be translated into the target languages.

The actual study or test contents that we created for each study had to be translated into the target languages.

The Study The project team defined a number of goals and objectives they wanted to achieve with their initial usability test of unedited machine translated content on the customer support website: 1. True intent of users 2. Evaluate and quantify the user experience 3. Establish a benchmark 4. Collect data that help improve machine translation and the user experience The first set of user experience studies included a task-based study of procedures that the user had to perform on the customer-support website. The first tests were conducted using

English as the control language, Latin-American Spanish and Simplified Chinese as the target languages.

This was followed by a usability survey that was deployed in five languages including Latin-American Spanish, Brazilian Portuguese, Russian, Simplified Chinese and Korean. The study used an intercept method to invite users to take the study (or decline). A goal of 200 responses was set for each language, and a window of one month was given to collect the responses.

A goal of 200 responses was set for each language.

The focus area of the test: • Were users able to accomplish their sitevisit goal • Level of user satisfaction • What errors or actors prevented users from solving their problem • Quality and usability of content • How machine translated impacted user experience The Results The following graphs show the results of the true intent study with the five target languages using English as the control.

33

Unmoderated Remote Usability Testing of Machine Translated Content

by Will Burgett

1. What is your primary language?

The majority of the users were on the site to download software or drivers. There were also significant numbers of users who were using the site to research product specifications or troubleshoot issues.

4. What was your overall satisfaction with the site?

The primary language for 25% of the users visiting the English site was not English. It would have been interesting to add more questions for these users that investigated how their experience may have been different than the native English speaker.

2. The language I used most on this site was

For Russian and Brazilian-Portuguese, more than 60% rated their satisfaction level as 4 or 5; there was a similar result for Latin-American Spanish. Korean scored the lowest level of 4 and 5, but 50% of respondents did score it a 3.

5. At any point during your visit did you follow instructions for completing a task and did you successfully complete your task?

A significant number of respondents said that they used both English and their primary language (p+eng) while on the site.

3. What best describes the purpose of your visit?

Only the people who answered “Yes” for the first question are shown in the second graph.

Korean had the highest number of users who couldn’t complete the task successfully. That was not a surprising result since Korean tends

34

Unmoderated Remote Usability Testing of Machine Translated Content

by Will Burgett

to be a very difficult language for statistical machine translation.

8. You answered that you used more than one language during your visit.

Can you provide more details?

6. What best describes the reason you were not able to complete the task?

The sample for this question was much smaller.

Although Russian had the highest number of “quality of language is not good”, it still had the highest overall score. Russian, because of its complex morphology and syntax, is also one of the more difficult languages for statisticalbased MT. There wasn’t any data or feedback that gave us any ideas on why the “content wasn’t accurate or incorrect.” In future studies it will be interesting to investigate if the information is missing or incomplete, and determine whether or not that was due to the quality of the machine translation.

9. How well do you understand English?

7. What best describes your opinion of the language on this site?

Spanish and Portuguese had the highest number of users responding that “I am fluent” in English.

10. What statement best describes your opinion of machine translation?

Russian, Latin-American Spanish, and Brazilian scored the highest, Korean the lowest. Users of the English survey were not asked this question.

35

Unmoderated Remote Usability Testing of Machine Translated Content

by Will Burgett

About 13% of the respondents said they were not familiar with the term “machine translation.”

If English is excluded, this ratio drops to 10%. The responses to this question supported our hypothesis that users with a primary language other than English are much more comfortable with MT. These users were twice as likely to say that they thought MT was “useful sometimes” or that they use it “all the time”. Similarly, users with English as a primary language were much more likely to say that they had never heard of MT.

Users with English as a primary language were much more likely to say that they had never heard of MT.

Perhaps a follow-up question may have shown that they are more familiar with service names like “Google Translate” or “Microsoft Translator.” This fact well illustrates the benefit of moderation to enable follow up discussion.

11. English survey: listed as “Other”

Primary

languages

One of the very interesting findings of this true-intent study was that 51 out of the 200 participants taking the English survey stated their primary language was not English.

Conclusions Customers come to www.Intel.com and www.

36

support.intel.com for a multitude of reasons.

This study focused on technical and nontechnical users who primarily came to the Intel Customer Support site to accomplish the following tasks: • Download drivers or software • Search for product specifications • Troubleshoot problems Overall we found the use of unedited machinetranslated content did not impact the user experience in Latin-American Spanish, Brazilian-Portuguese, Russian, and Simplified Chinese. Only the quality of machine translated Korean showed an impact on the usability and user experience of the Korean site. Looking at how well m a c h i n e translated sites scored compared to the English reference site demonstrated that MT content a c t u a l l y increased the user’s level of satisfaction. This true intent study showed that the majority of the users visiting the five target language sites found machine translated content to be useful.

The majority of the users visiting the five target language sites found machine translated content to be useful.

These initial studies also demonstrated the potential power and versatility of URUT technology to: • Enable both large and small organizations to do focused, in-depth tests and studies of websites and web applications on a global scale.

• Simultaneously conduct statistically significant tests and studies across many languages.

• Conduct usability testing at a much lower cost without the need for labs or moderators.

Caveat: While Intel IT does conduct unmoderated remote usability testing on a variety of topics, one in-house user-experience

Unmoderated Remote Usability Testing of Machine Translated Content

by Will Burgett

expert stated a preference for moderation to enable more in-depth follow up.

The results of this study do corroborate that moderation would have enabled follow up with users who said they didn’t understand the term “machine translation” to see if they were instead familiar with “Google Translate” or “Microsoft Translator”. And with a moderator we could have elicited feedback to better understand users’ ideas on why the content was viewed as incorrect.

However, for this first study on acceptance of MT content, the expense of moderation was not needed to help us learn what we really needed to know: Intel’s investments in deploying machine translated content for support.intel. com are not at the expense of the international users’ experience.

Will Burgett Will

Burgett

years

of

Electro

has

over

experience

Scientific

30 with

Industries

(ESI), The Writing Company, and Intel Corporation. Will has

managed

publications,

localization, human factors engineering, product development, innovation programs, and is a PMI certified Project Management Professional. He was Chairman of the Board for TAUS Data Association and a member of the TAUS Advisory Board. Will

Send your comments or questions to review@taus.net

retired from Intel Corporation in January 2015 and is currently the Communications Manager for the nonprofit Translators without Borders, and he also operates his own rare and out-of-print book business in Oregon.

Julie Chang

Ryan Martin

Julie Chang has over 15 years of experience as a Localization Program

Ryan Martin is an application

Manager at Intel Corporation. She

developer at Intel Corporation

leads

and

automation

develops

translation

(2001-present). Over the past

strategy

for

7 years, Ryan has designed

Intel

Customer Support group. She has

and

deployed

multiple

also managed and executed multiple Machine Translation

enterprise MT solutions across Intel. His interests

implementations

Translation

include translation automation, natural language

technology into various platforms across Intel organizations.

processing, computational linguistics, and data

Julie lives in Portland, Oregon with her husband, daughter,

analysis. Ryan lives in Portland, Oregon with his

and two dogs.

wife and two kids.

and

integrated

Machine

37

Contributors

Reviews Mike Tian-Jian Jiang Mike was the core developer of GOING (Natural Input Method, http://iasl.iis.

Andrew Joscelyne

sinica.edu.tw/goingime.htm), one of the most famous intelligent Chinese

Andrew Joscelyne has been reporting

phonetic

on language technology in Europe for

He was also one of the core committers of OpenVanilla,

well over 20 years now. He also been a market watcher

one of the most active text input method and processing

for European Commission support programs devoted

platform. He has over 12, 10, and 8 years experiences

to mapping language technology progress and needs.

on C++, Java, and C#, respectively. Also familiar with

Andrew has been especially interested in the changing

Lucene and Lemur/Indri. His most important skill set is

translation industry, and began working with TAUS from

natural language processing, especially for Chinese word

its beginnings as a part of the communication team.

segmentation based on pattern generation/matching,

Today he sees language technologies (and languages

n-gram statistical language modeling with SRILM, and

themselves) as a collection of silos â&#x20AC;&#x201C; translation, spoken

conditional random fields with CRF++ or Wapiti.

interaction, text analytics, semantics, NLP and so on.

Specialties: Natural Language Processing, especially for

Tomorrow, these will converge and interpenetrate,

pattern analysis and statistical language modeling.

releasing new energies and possibilities for human

Information Retrieval, especially for tuning Lucene and

communication.

Lemur/Indri. Text Entry (Input Method).

Brian McConnell

Amlaku Eshetie

Brian

McConnell

is

the

Head

of

input

method products.

Amlaku earned a BA degree in

Localization for Insightly, the leading

Foreign

small

for

(English & French) in 1997, and

Google Apps. He is also the publisher

an MA in Teaching English as a

of Translation Reports, a buyers guide

Foreign Language (TEFL) in 2005,

for translation and localization technology and services,

both at Addis Ababa University, Ethiopia. He had been a

as well as a frequent contributor to TAUS Review.

teacher of English at various levels until he switched to

Specialties: Telecommunications system and software

translation and localisation in 2009. Currently, Amlaku

design with emphasis on IVR, wireless and multi-modal

is the founder and manager of KHAABBA International

communications. Translation and localization technology.

Training and Language Services at which he has been

business

CRM

service

Languages

&

Literature

able to create a big base of clients for services, such as localisation, translation, editing & proofreading, interpretation, voiceovers, copy writing.

38

Contributors

Perspectives Jost Zetzsche Jost Zetzsche is a certified Englishto-German technical translator, a translation technology consultant, and a widely published author on various aspects of translation. Originally from Hamburg, Germany, he earned a Ph.D. in the field of Chinese translation history and linguistics. His computer guide for translators, A Translator’s Tool Box for the 21st Century, is now in its eleventh edition and his technical newsletter for translators goes out to more than 10,000 translation professionals. In 2012, Penguin published his co-authored Found in Translation, a book about translation and interpretation for the general public. His Twitter handle is @jeromobot.

Luigi Muzii Luigi Muzii has been working in the language industry for more than 30 years as a translator, localizer, technical

writer,

author,

trainer,

university teacher of terminology and localization, and consultant. He has authored books on technical writing and translation quality systems, and is a regular speaker at conferences.

Nicholas Ostler

Lane Greene

Nicholas Ostler is author of three

Lane Greene is a business and

books on language history, Empires

finance

of the Word (2005), Ad Infinitum (on

Economist based in Berlin, and

Latin - 2007), and The Last Lingua

he also writes frequently about

Franca (2010). He is also Chairman

language for the newspaper and

of the Foundation for Endangered Languages, a global

online. His book on the politics of language around

charitable organization registered in England and Wales.

the world, You Are What You Speak, was published by

A research associate at the School of Oriental and African

Random House in Spring 2011. He contributed a chapter

Studies, University of London, he has also been a visiting

on culture to the Economist book “Megachange”, and his

professor at Hitotsubashi University in Tokyo, and L.N.

writing has also appeared in many other publications. He

Gumilev University in Astana, Kazakhstan. He holds an

is an outside advisor to Freedom House, and from 2005

M.A. from Oxford University in Latin, Greek, philosophy

to 2009 was an adjunct assistant professor in the Center

and economics, and a 1979 Ph.D. in linguistics from

for Global Affairs at New York University.

correspondent

for

The

M.I.T. He is an academician in the Russian Academy of Linguistics.

39

Directory of Distributors Appen Appen is an award-winning, global leader in language, search and social technology. Appen helps leading technology companies expand into new global markets.

BrauerTraining Training a new generation of translators & interpreters for the Digital Age using a web-based platform + cafeteriastyle modular workshops.

Capita TI Capita TI offers translation and interpreting services in more than 150 languages to ensure that your marketing messages are heard - in any language.

Cloudwords Cloudwords accelerates content globalization at scale, dramatically reducing the cost, complexity and turnaround time required for localization.

Concorde Concorde is the largest LSP in the Netherlands. We believe in the empowering benefits of technology in multilingual services.

CPSL Multilingual language provider for global strategies: translation, localization, interpreting, transcription, voice over & subtitling.

Crestec Europe B.V. We provide complete technical documentation services in any language and format in a wide range of subjects.

Whatever your needs are, we have the solution for you!

Global Textware Expertise in many disciplines. From small quick turnaround jobs to complex translation. All you need to communicate about in any language.

HCR HCR works in conjunction with language partners to deploy software products and linguistic services globally in core industries such as IT, Automotive and more.

Hunnect Ltd. Hunnect Ltd. is an MLV with innovative thinking and a clear approach to translation automation and training post-editors. www.hunnect.hu Iconic Translation Machines Machine Translation with Subject Matter Expertise. We help companies adopt MT technology.

iDisc Established in 1987, iDISC is an ISO-9001 and EN-15038 certified language and software company based in Spain, Argentina, Mexico and Brazil.

40

InterTranslations Intertranslations LLC is based in Athens, London and Nicosia offering translation and localization services in all languages.

IOLAR Founded in 1998, Iolar employs 40 highly-skilled linguists and engineers specialised in translation of highly demanding documentation and software localisation.

Jensen Localisation Localization services for the IT, Health Care, Tourism and Automotive industries in European languages (mostly Nordic, Dutch and Spanish).

KantanMT.com KantanMT.com is a leading SaaS based statistical machine translation platform that enables users to develop and manage customized MT engines in the cloud.

Kawamura International Based in Tokyo, KI provides language services to companies around the world including MT and PE solutions to accelerate global business growth.

KHAABBA International Training and Language Services KHAABBA is an LSP company for African languages based in Ethiopia.

Larsen Globalization Ltd Larsen Globalization Ltd is a recruitment company dedicated to the localization industry since 2000 with offices in Europe, the US and Japan.

Lingo24 Lingo24 delivers a range of professional language services, using technologies to help our clients & linguists work more effectively.

Linguistic Systems LSI provides foreign language translation services in over 115 languages and unlimited subject matter. Contact us at 877-654-5006 or www.linguist.com Lionbridge Lionbridge is the largest translation company and #1 localization provider in marketing services in the world, ensuring global success for over 800 leading brands MateCat MateCat is a free web CAT tool for LSPs and translators.

Use it to translate your projects or to outsource to over 120,000 professional translators in one click.

Memsource Cloud An API-enabled translation platform that includes vendor management, translatio memory, integrated machine translation, and a translatorâ&#x20AC;&#x2122;s workbench.

Directory of Distributors Mirai Translate Mirai Translate will custom-build a translation A.I. which make innovation happen for your business and create an exciting “MIRAI (future)”.

Moravia Flexible thinking. Reliable delivery. Under this motto, Moravia delivers multilingual language services for the world’s brand leaders.

Morningside Translation We’re a leading translation services company partnering with the Am Law 100 and Fortune 500 companies around the globe.

MorphoLogic Localisation MorphoLogic Localisation is the developer of Globalese, an SMT system that helps increase translation productivity, decrease costs and shorten delivery times.

Pactera Pactera is a leading Globalization Services provider, partnering with our clients to offer localization, in-market solutions and speech recognition services.

Plunet Plunet GmbH develops and markets the business and translation management solution Plunet BusinessManager for professional LSPs and translation departments.

Rockant Consulting & Training We provide consulting, training and managed services that transform your career from “localization guy/girl,” to a strategic adviser to management.

Safaba Translation Solutions, Inc. A technology leader providing automated translation solutions that deliver superior quality and simplify the path to global presence unlike any other solution.

SeproTec SeproTec is a 25 years experience Multilingual Service Provider ranked among the Top 40 Language Service Companies in the world.

Sovee Sovee is a premier provider of translation and video solutions. The Sovee Smart Engine “learns” translation preferences in 6800 languages. sQuid sQuid help companies integrate and exploit translation technologies in their workflows and maximize their use of their language data.

STP Nordic Translation STP is a technology-focused Regional Language Vendor specialising in English, French, German and the Nordic languages. See www.stptrans.com.

SYSTRAN SYSTRAN is the market historic provider of language translation softwaresolutions for global corporations, public agencies and LSPs tauyou language technology Machine translation and natural language processing solutions for the translation industry text&form text&form is an LSP with expertise in software & multimedia localization, technical translation, terminology management and SAP consulting.

Tilde Tilde develops custom MT systems and online terminology services, with special expertise in the Nordic, Baltic, Russian, and CEE languages.

TraductaNET Traductanet is a linguistic service company specialising in translation, software and website localisation, terminology management and interpreting.

Trusted Translations Internationally recognized leader in multilingual translation & interpretation services. Committed to providing clients with the highest quality service.

UTH International UTH International is an innovative professional provider of globalization solutions and industry information, serving customers with advanced technologies.

Welocalize Welocalize offers innovative translation & localization solutions helping global brands grow & reach audiences around the world.

Win & Winnow Provider of translation, multimedia and desktop publishing services founded in 2004. We are one of the top ten language services providers in Latin America.

XTRF XTRF is a platform for project management, quoting, invoicing, sales and quality management, integrated with CAT, accounting and CRM tools.

41

Get your insights, tools, metrics, data, benchmarking, contacts and knowledge from a neutral and independent industry organization.

Join TAUS!

TAUS is a think tank and resource center for the global translation industry.

Open to all translation buyers and providers, from individual translators, language service providers & buyers to governments and NGOs.

taus.net

42

Industry Agenda

Upcoming TAUS Events

Upcoming TAUS Webinars

TAUS Roundtable 6 October, 2015 Washington, DC (USA)

TAUS Translation Technology Showcase QuickTerm & Terminotix 2 September, 2015, 5 PM CET

TAUS Annual Conference 12-13 October, 2015 San Jose, CA (USA)

TAUS Translation Quality Webinar Crowdshaping Translation Quality 16 September, 2015, 5 PM CET

TAUS QE Summit San Jose 14 October, 2015 San Jose, CA (USA) hosted by eBay

Ephemeral translations 18 November, 2015, 5 PM CET Translation Automation Users Call Translation as a Utility 25 September, 2015, 5 PM CET

TAUS HAUS

Industry Events

TAUS Office Amsterdam Keizersgracht 74 30 July, 2015, 6 PM CET

Brand2Global 30 Sept - 1 Oct, 2015 London (UK) LocWorld Silicon Valley 14-16 October, 2015 Santa Clara, CA (USA) MT Summit XV 30 Oct - 3 Nov, 2015 Miami, FL (USA)

Do you want to have your event listed here? Write to editor@taus.net for information.

43

Innovation and continuous improvement is embedded in our corporate culture • A leader in translation and interpretation services and your guarantee for quality at the most competitive rates

• Flexible solutions to accommodate your needs

• A committed and unparalleled customer service

Toll Free (US) 1-877-255-0717 www.trustedtranslations.com

sales@trustedtranslations.com

44