AI Responses: Peril, Fear, Alarming

Just one week after its release, ChatGPT had one million registered users. This resulted in a big stir in the media. Headlines threw around words like ‘peril’, ‘fear’, ‘controversy’ and ‘alarming’ in response to ChatGPT’s capabilities. Much of these words came in response to fears that AI will displace us. Krugman points out that ‘dislocation’ of workers by technology has been ‘a feature of modern societies for at least two centuries’. xi He warns that ‘some of us may find ourselves either unemployed or earning far less than we expected, given our expensive educations’, as a result of recent advances in AI.xii Others are more optimistic, Pattisall highlights that AI is not quite at its peak yet and that AI will always need supervision.xiii In this way, AI is a tool that can support human creativity, rather than hinder it. It is humans who need to work out how to involve AI in creative processes to use ChatGPT in an optimal manner.

ChatGPT has also caused concerns in the education sector. On 5th January 2023, investor Marc Andreessen tweeted, ‘The war on children intensifies’, accompanied by a screenshot of a newspaper headline stating, ‘NYC education department blocks ChatGPT on school devices, networks’.xiv In response, Elon Musk tweeted, ‘It’s a new world. Goodbye homework!’xv The education sector has been thrown off balance by ChatGPT and its uncanny ability to construct arguments and make conclusions all in fairly natural sounding prose. It has been found that ChatGPT would be able to achieve a B grade on a Wharton MBA Exam.xvi Researchers have also discovered that ChatGPT was ‘comfortably within the passing range’ of the US medical licensing examination.xvii As such, ChatGPT presses us to reconsider the very meaning of education. Writing for the Financial Times, Camilla Cavendish notes that her own children ‘sit national exams which feel horribly similar’ to the ones she took at school and that their exams now come with the ‘new horror of mark schemes which must also be learnt to score points by parroting the correct key words’.xviii In light of this outdated system, Cavendish emphasises that we need to ensure we teach children how to use generative AI, rather than push them away from it. In her eyes, the legacy of ChatGPT is prompting a ‘wholesale rethink’ of our education system. She is indeed right. A rehaul of the curriculum has been needed since the invention of Google. If AI can take on the basics of any area of study, then perhaps students need not spend so long painfully memorising them, after all technology puts all the answers at our fingertips. Instead, children need to learn skills of critical thinking, analysis, and real-world application from an earlier age. Being able to use knowledge stored in online spaces is the only real skill that will be needed in the future, and curriculums need to begin adapting to that reality now. I asked Chatty about this issue, and he provided a similar response to my own opinion:

Jieun: How should education systems respond to the existence of ChatGPT?

Chatty: Education systems should consider incorporating the use of language models like ChatGPT to enhance language learning and improve writing skills. They can also be used to assist with research and composition of written work. Additionally, educators should make sure to teach students about the limitations and potential biases of these models, and how to critically evaluate the information they generate.

The most worrying issue making headlines about AI is not its potential impact on our day to day lives, but its potential to spread false information (even unknowingly). AI learns through data collected on the internet, but not all data on the internet is true. As such, ChatGPT has the potential to generate incorrect information without giving its users any possibility to find an alternative. ChatGPT lists the following limitations on its landing page:

‘May occasionally generate incorrect information’

‘May occasionally produce harmful instructions or biased content’

‘Limited knowledge of world and events after 2021’

Thus, although ChatGPT seems amazing at first glance, it is still a work in progress. It has great potential to spread false information. Gary Marcus, a Professor at New York University, warns that generative AI could be used to manipulate elections, as it would be easy to write thousands of pieces of misinformation at one time, which is how false news gains traction.xix

Developments in AI: Deep learning and the transformer model

AI can only do what it has learnt through deep learning in a pool of big data. Even just a few years ago, discussions about the potential of AI centred around the lack of available data. Now, the collection and storage of vast amounts of data has become easier than ever, and so the capabilities of AI have expanded. It is not difficult to understand how big data is generated; every action performed on a screen is recorded somewhere. In fact, it is rarer for one’s data to not be recorded, unless you are in a remote location without any internet access. While you may wish to take a break from the constant collection of data, it is very difficult to avoid. Throughout the course of a day, we constantly give away our data just by reading the news or watching a video online. This happens on such a vast scale in the population that the amount of data generated is almost incomprehensible.

ChatGPT works via a transformer model. The transformer model, first introduced in 2017 by Vaswani et al., is a deep learning model that analyses a sequence of data as input and produces another sequence as output. This allows for tasks such as translating text from one language to another using an attention mechanism that tracks the relationships between words with long-range dependencies. This is a more efficient method than previous models such as recurrent neural networks (RNN) and long short-term memory (LSTM) networks, which process data word-by-word instead of in sequences. Other models similar to transformers include convolutional neural networks (CNN), but they are only effective when working with shorter sentences. Notable transformer models include BERT, GPT-2, and GPT-3, each with increasing numbers of parameters. An upcoming model, GPT-4, is set to surpass all previous models with 100 trillion parameters. These models, while highly advanced, still lack the ability to understand and use language in the same way that humans do. They are also known for their high computational and financial costs. For example, it is estimated that the financial cost of training a GPT-3 model would be around $10 million, while the amount of computing power needed for training is around a quadrillion neural network computations per second (Heaven, 2021). Additionally, increases in parameters in these models also result in longer training periods: it would take 342 years to finish training a GPT-3 model if just one Nvidia V100 GPU is used (Bansal, 2021, p. 172). Furthermore, when using a BERT model, a 100,000-word-long text would result in a mind-boggling 10 billion word pairs, which requires astronomical computing power to process (Rothman, 2021, p. 159).

The Human Brain is Still More Cost Efficient than AI

The large amount of data and computation required for these models also leads to significant financial costs for the companies that use them. The cost-inefficiency of these AI models stands in stark contrast to the human brain, which is highly adapted for costefficiency (Eysenck & Keane, 2020, p. 14). The human brain can carry out 100 trillion complex operations per second, with each operation only costing around 2 x 10-15 joules of energy. This makes it minimally “a factor of a billion more efficient than our present technology” and “a factor of 10 million more efficient than the best digital technology that we can imagine” (Fiorini, 2020, p. 334). The cost-efficiency of the human brain can also be seen in how we process and use language efficiently as humans, that is to say in the pragmatic usage of language. Humans are able to understand and use language in context and they can adapt their language to different situations and audiences, which AI models currently struggle to do. It is important to note that the goal of AI is not to replace human intelligence, but to augment and enhance it.

AI Varieties

Aside from ChatGPT, there are a few other publicly available AI chatbots. One is You.com, which comprises YouCode, YouWrite, and YouImagine, for generating code, writing, and images respectively. YouWrite is particularly interesting because it allows you to add the context of your writing – email, essay, paragraph, social media post, title, blog, or essay – and it allows you to choose a tone – neutral, friendly, professional, witty, or persuasive. Another platform is Perplexity, which is more similar to ChatGPT, in the sense that you can ask it anything in one search box. Interestingly, however, Perplex gives you the source of its information, which could help to battle the spread of false information. The challenge with AI is not providing information, but providing quality information.

Other recent breakthroughs in AI technology include DeepMind’s AlphaCode and LG’s Tilda. AlphaCode has coding capabilities at a similar level to human software developers. DeepMind entered AlphaCode into 10 programming competitions, and on average, it ranked in the top 50% against human competitors. According to DeepMind, AlphaCode can ‘solve new problems that require a combination of critical thinking, logic, algorithms, coding, and natural language understanding’. Tilda is a digital fashion designer that operates on ExaOne software, which has learned from a 600 billion corpus of data and 250 million high-resolution images combined with text. At New York Fashion Week 2022, Tilda presented 200 outfits with South Korean designer Park Yoonhee. The designs were based on 3000 patterns and images that Tilda generated. Thus, AI is gradually expanding its net into all areas of human life.

Limitation 1: Linguistic Diversity

The abundance of available data has allowed AI to become increasingly human-like. The question arises: will we arrive at a time when we can no longer distinguish between humans and AI? Digital humans are still limited in their likeness to humans for two reasons. The first is linguistic diversity. Many AI programmes function well in English, but they are limited in other languages, particularly non-European languages. In the case of East Asia, China, Japan, and the Korean peninsula tend to use different platforms than the rest of the world. China’s main search engine is Baidu, whilst South Korea’s is Naver. Japan’s most popular social media app is LINE, whilst China’s is WeChat. As such, their data is stored locally, and ChatGPT likely has little access to mining this data. Korea has its own generative pretrained transformer, called KoGPT, which is trained on a Korean data set from which ChatGPT would likely never have learned. As such, there is a lack of linguistic diversity in AI. I asked Chatty about this:

Jieun: How many languages can AI process?

Chatty: Artificial intelligence (AI) has the ability to process and understand multiple languages, but the number of languages it can process depends on the specific AI model and the data it has been trained on. Some AI models are trained on a large dataset of text in multiple languages, and can understand and respond to multiple languages. However, the quality of the AI’s understanding and response may vary depending on the language and the complexity of the task.

Currently, many AI models are focused on processing and understanding a limited number of languages, such as English, Chinese, Spanish, and a few other languages. For example, GPT-3, one of the most advanced AI language models, can process and understand more than 50 languages, but it is more proficient in some languages than others.

As AI technology continues to advance, it is likely that AI models will become better at understanding and processing a wider range of languages. However, it’s important to note that the development of AI models for a specific language requires a large amount of data and resources, so it may take a while before all the languages are covered.

Jieun: What about Asian Languages?

Chatty: Asian languages, including but not limited to Chinese, Japanese, Korean, Thai, Hindi, Urdu, Bengali, etc, can be processed by AI models, but the quality of the AI’s understanding and response may vary depending on the language and the complexity of the task. Some Asian languages, like Chinese and Japanese, have a different writing system than many other languages, which can make them more challenging for AI models to understand. Additionally, many Asian languages have a large number of characters and variations, which can also make them more difficult for AI models to process.

Currently, there are some AI models that are specifically designed to process and understand Asian languages, such as BERT-wwm-ext-chinese and GPT-3-Chinese. These models have been trained on large datasets of text in the specific Asian languages, which allows them to understand and respond to those languages with a high level of accuracy. However, it is still a challenging task to train AI models for many Asian languages, particularly those that are less widely spoken, due to the lack of resources and data.

As AI technology continues to advance and more data becomes available, it is likely that AI models will become better at understanding and processing a wider range of Asian languages.

Jieun: Do you think in the future you can process Asian languages well?

Chatty: As a language model, I don’t have the ability to think or have personal opinions, but it is likely that AI models will continue to improve their ability to process and understand Asian languages in the future. With the increasing use of AI in various fields, the demand for AI models that can process and understand Asian languages will also increase. This, in turn, will drive more research and development in this area, and more resources will be allocated to improve the performance of AI models in processing Asian languages.

The advancements in natural language processing (NLP) and machine learning (ML) techniques, as well as the availability of more data in Asian languages, will also contribute to the improvement of AI models’ ability to process Asian languages. Additionally, more collaboration between researchers, industries, and governments can help to overcome the challenges of training AI models for Asian languages and increase the availability of resources.

It’s worth noting that, even though AI models will continue to improve in processing Asian languages, it may still take some time before all Asian languages are covered, as developing AI models for specific languages requires a large amount of data and resources.

Jieun: You know that there is AI in Asia available to locals?

Chatty: Yes, it is true that the development and use of AI technology is rapidly increasing in Asia. Many countries in Asia, such as China and South Korea, have invested heavily in AI research and development and have become global leaders in the field.

In these countries, AI technology is being applied to a wide range of industries, such as healthcare, finance, transportation, and manufacturing. Additionally, there are a lot of startups and companies that are focused on developing AI-based products and services for the local market.

In terms of natural language processing, many companies and researchers in Asia have developed AI models for Asian languages, such as Chinese, Japanese, and Korean, that are specifically designed to understand and respond to the unique characteristics of these languages, such as the large number of characters and variations.

It’s worth noting that the development of AI models for Asian languages is an ongoing process and the quality of the AI’s understanding and response may vary depending on the language and the complexity of the task. But, it’s clear that the AI in Asia is available to locals and it’s expected that it will continue to improve and become more widely used in various fields.

In our ever globalising and ever diverse world, lack of diversity is a serious limitation to AI chatbots. ChatGPT cannot serve us well if it is missing data from whole continents. ChatGPT is banned in China, and it seems that China will end up using a different generative AI platform, as it has done with social media.xx This is a serious challenge that AI creators need to think about. How can better data sets be constructed so that AI bots, like ChatGPT, can provide a range of information from diverse sources that support the diverse world in which we live?

Limitation 2: Pragmatic Diversity

Human language is founded on the basis Zipf’s ‘principle of least effort’. I’ll ask Chatty to explain it:

Jieun: Explain Zipf’s principle of least effort

Chatty: Zipf’s principle of least effort is a principle proposed by George Zipf, a linguist and philosopher, which states that people will tend to use the easiest and most efficient means of communication available to them. According to this principle, people will naturally gravitate towards using the most common words and structures in a language because they require the least effort to produce and understand. This principle is closely related to Zipf’s law, which states that the frequency of words in a language follows a power-law distribution, with a small number of high-frequency words and a large number of low-frequency words. Together, these two principles suggest that people will tend to use the most common words and structures in a language because they are the most efficient means of communication.

When we speak, we aim to create meaning efficiently, finding the optimal words. Our memories are limited. It is impossible to know all the words in our languages, and even more impossible to use them all regularly. AI does not function on the same ‘least effort’ principle, there are no limitations to its memory or ability to use all the words that it knows. For this reason, AI language is far from human language. Nonetheless, human language practices are not only about being efficient and economic. The mystery of human language lies in the fact that we do not always take the most efficient route. For example, why do we use fillers like ‘um’ in our spoken language? Linguists label these as disfluencies. Humans have numerous spontaneous, unpredictable, and inefficient expressions, and most of them are based on emotion. Humans do not always use the most efficient language because they want to express emotion and attitudes in what they are saying. In other words, we use pragmatics to encode meaning into our interactions. Pragmatics are versatile and change fluidly and instinctively according to the context of our interactions.xxi Social interaction is different in every culture from a macro perspective, and also different to each individual human being from a micro perspective. Asian languages in particular have very complex pragmatics based on interpersonal relations and societal hierarchies.xxii These languages are not well understood in comparison to English and other European languages, and therefore AI translators, like Google Translate, and generative AI, like ChatGPT, have trouble using those languages. Our use of pragmatics takes into account so many variables and varies from language to language and culture to culture, that AI does not yet have the capacity to produce language with natural use of pragmatics. That is not to say it is impossible, however. When I was studying for my PhD, it was believed that AI would never be able to understand humour, but now we have enough data for AI to learn how humans make humour. Thus, AI can likely acquire an understanding of pragmatics through enough deep learning, but for now AI is pragmatically limited. I asked Chatty about this issue:

Jieun: Tell me about pragmatics

Chatty: Pragmatics is the branch of linguistics that studies the ways in which language is used in context to convey meaning. It examines how people use language in different social situations and how they interpret the meaning of what is said or written. This includes the study of non-literal language, such as irony, metaphor and sarcasm, as well as the social rules that govern language use. Pragmatics also looks at the factors that influence language choice, such as the speaker’s intent, the listener’s background and the context of the conversation. It is a multidisciplinary field of study that draws on psychology, sociology, anthropology, and other fields to understand how people use language in different contexts.

Jieun: Are you able to use pragmatics?

Chatty: As a language model, I am able to understand and use pragmatics to a certain extent. I am able to take into account the context of a conversation and use language appropriately based on the context. For example, I can use politeness strategies, such as hedging or indirectness, to avoid being too direct or confrontational. I can also understand and respond to different speech acts, such as requests or questions, and can take into account presuppositions and implicatures in order to understand the intended meaning of a text.

However, my understanding of pragmatics is limited by the data that I’ve been trained on. The ability to use pragmatics is based on the patterns of usage that I’ve seen in the data, so it’s possible that I may not always use pragmatics in the way that a human would in certain contexts.

Time to Catch Up

Whilst AI developers work furiously to improve their AI models, we have some time to take stock and consider the most serious questions about AI. What humans can do in a split-second takes AI engines a long time to work out through deep learning. AI must be not only considered from the point of linguistics, but also sociology, child development, law, ethics, and more. The corporations responsible for creating these AI models may largely be motivated by commercial desires. Before AI dominates our lives, we need to be asking questions about the effects of AI and what role it should play. When AI is integrated into our lives, it will already be too late.