{"id":671,"date":"2023-02-28T23:01:51","date_gmt":"2023-03-01T07:01:51","guid":{"rendered":"https:\/\/linguamonium.com\/?p=671"},"modified":"2024-02-14T09:48:45","modified_gmt":"2024-02-14T17:48:45","slug":"ais-babbling-towers-chatgpt-and-large-language-models","status":"publish","type":"post","link":"https:\/\/linguamonium.com\/?p=671","title":{"rendered":"AI\u2019s babbling towers: ChatGPT and large language models"},"content":{"rendered":"<p>Okay. I didn\u2019t want to write a post on ChatGPT, because its public release has been like shrapnel through the meat of the internet \u2013 but at the end of the day this IS a language blog, and I would be remiss for not doing due diligence.<\/p>\n<p>If you\u2019ve been under a rock for the past few months, ChatGPT is a new chatbot that answers user questions, generating natural and conversational (\u201chuman\u201d) sounding text responses. Beyond the standard chat applications, it can be used to write fiction or non-fiction from a prompt, generate computer code, and translate between languages, among other things. (If you\u2019d like to try it out for yourself, follow the link <a href=\"https:\/\/openai.com\/blog\/chatgpt\/\" target=\"_blank\" rel=\"noopener\">here<\/a>.)<\/p>\n<p>I\u2019ll go over some basics first. Then we\u2019ll explore several major concerns about the technology, along with more specific limitations. I won\u2019t be discussing all the things ChatGPT does well, because (despite working in tech) I\u2019m a luddite, and prefer to gripe. Plus it\u2019s more entertaining to inspect the flaws.<\/p>\n<p>&nbsp;<\/p>\n<h2>How it works (the nutshell version)<\/h2>\n<p>ChatGPT stands for \u201cChat Generative Pre-Trained Transformer\u201d. OpenAI, the company that produced it, used their existing <strong>large language model<\/strong> <strong>(LLM) <\/strong>architecture (specifically, a model in the GPT-3.5 series), then fine-tuned it with both <strong>supervised<\/strong> and <strong>reinforcement learning <\/strong>(called <strong>Reinforcement Learning with Human Feedback<\/strong>, or<strong> RLHF<\/strong>). Broken down a bit:<\/p>\n<ul>\n<li><strong>Large language model:<\/strong> A machine-learning model that tries to predict the next word in a sentence by assigning probabilities to sequences of words. The model is trained on enormous amounts of text.\n<ul>\n<li>GPT-3 has 175 billion parameters, and was trained on 570 gigabytes of text.<\/li>\n<\/ul>\n<\/li>\n<li><strong>Supervised learning: <\/strong>Uses labeled datasets and correct answers to teach algorithms how to classify data or predict a particular outcome. In the case of ChatGPT, human trainers created and labeled two-sided conversations, which were then mixed into the existing dataset.<\/li>\n<li><strong>Reinforcement learning: <\/strong>A dynamic process that trains algorithms via a system of reward and punishment. The algorithm learns without human intervention by maximizing its rewards (for correct answers) and minimizing its punishments (for incorrect answers). With ChatGPT, humans ranked 2+ model responses by quality, and used these to create reward models for training.<\/li>\n<\/ul>\n<p>ChatGPT was trained on mountains of data scraped from the internet, along with text sources from public and research domains. It finished training at the beginning of 2022.<\/p>\n<p>&nbsp;<\/p>\n<h2>Large language models<\/h2>\n<p>A few qualities of large language models more generally:<\/p>\n<ul>\n<li>LLMs are a type of <a href=\"https:\/\/www.techopedia.com\/definition\/32902\/deep-neural-network\" target=\"_blank\" rel=\"noopener\">deep neural network<\/a> \u2013 a neural network that has more than two layers.<\/li>\n<li>The scale of these models changes their behavior dramatically \u2013 so that they\u2019re able to perform tasks they weren\u2019t explicitly trained on (called <strong>zero-shot learning<\/strong>).<\/li>\n<li>They\u2019re able to do things that range from searching, summarizing text, and engaging in chatbot dialog, to creating charts from text descriptions, translating between languages, generating code, and writing essays.<\/li>\n<\/ul>\n<p>Quite a few LLMs exist already<a href=\"#_ftn1\" name=\"_ftnref1\">[1]<\/a>, predating the grandiose hullabaloo surrounding ChatGPT. Examples include Pathways Language Model, or PaLM (540 billion parameters) and Language Models for Dialog Applications, or LaMDA (137 billion parameters) from Google Research; Megatron-Turing NLG (530 billion parameters) from Microsoft\/Nvidia; and BLOOM \u2013 BigScience Large Open-science Open-access Multilingual Language Model (176 billion parameters).<\/p>\n<p>&nbsp;<\/p>\n<h2>Major concerns<\/h2>\n<p>People are fired up about ChatGPT. Articles from every corner of the web extol its virtues, and also caution against its limitations and potential for harm. Below is a summary of the concerns I\u2019ve seen.<\/p>\n<ul>\n<li><strong>Gives inaccurate, often completely fabricated responses<\/strong>\n<ul>\n<li><strong><a href=\"https:\/\/en.wikipedia.org\/wiki\/Hallucination_(artificial_intelligence)\" target=\"_blank\" rel=\"noopener\">Hallucinates<\/a><\/strong> responses that, nonetheless, sound very plausible.<\/li>\n<li>Also, does not (yet) contain identifying sources for the information it provides, so that human users cannot vet the source material for reliability.<\/li>\n<\/ul>\n<\/li>\n<li><strong>Environmental impact<\/strong>\n<ul>\n<li>Training LLMs takes <em>enormous <\/em>environmental resources.<\/li>\n<li>A paper from 2019 estimated that \u201ctraining a neural architecture search based model with 213 million parameters\u201d would \u201cproduce more than five times the lifetime carbon emissions from the average car.\u201d<a href=\"#_ftn2\" name=\"_ftnref2\">[2]<\/a> Then compare those 213 million parameters with the 175 <em>billion <\/em>of ChatGPT.<\/li>\n<\/ul>\n<\/li>\n<li><strong>Exhibits political bias<\/strong>\n<ul>\n<li>From <a href=\"https:\/\/en.wikipedia.org\/wiki\/ChatGPT#Limitations\" target=\"_blank\" rel=\"noopener\">Wikipedia<\/a>: \u201cAccording to the BBC, as of December 2022 ChatGPT is not allowed to \u2018express political opinions or engage in political activism.\u2019 Yet, research suggests that ChatGPT exhibits a pro-environmental, left-libertarian orientation when prompted to take a stance on political statements from two established voting advice applications.\u201d<\/li>\n<li>I\u2019ve asked it political questions myself, and confirm that it has a clear left-wing bias.<\/li>\n<\/ul>\n<\/li>\n<li><strong>Facilitates cheating on school assignments<\/strong>\n<ul>\n<li>The fear is that this technology will hinder learning and critical thinking on a massive scale, unless safeguards are put in place. (Some plagiarism-detection software has already been released; I haven\u2019t looked into how well it works.)<\/li>\n<\/ul>\n<\/li>\n<li><strong>Potential for geopolitical consolidation of power<\/strong>\n<ul>\n<li>From <a href=\"https:\/\/english.elpais.com\/science-tech\/2023-01-31\/chatgpt-is-just-the-beginning-artificial-intelligence-is-ready-to-transform-the-world.html\" target=\"_blank\" rel=\"noopener\">El Pa\u00eds article<\/a> (about AI more generally): \u201cDisinformation campaigns, cyberattacks, data collection, facial recognition and the like are already a significant problem, and may develop even more sophisticated capabilities that can disrupt democracies and consolidate internal control in authoritarian regimes.\u201d<\/li>\n<\/ul>\n<\/li>\n<li><strong>Used underpaid laborers to build toxic content filters<\/strong>\n<ul>\n<li>Since most of ChatGPT\u2019s training data was collected from the internet, OpenAI wanted to build an \u201cAI-powered safety mechanism\u201d (basically a filter for toxic content).<a href=\"#_ftn3\" name=\"_ftnref3\">[3]<\/a> To accomplish this, they used outsourced Kenyan workers, paid less than $2 per hour, to label examples of violence, hate speech, sexual abuse, etc., that could then be used as training data for the AI safety system. Beyond the low pay, workers reported experiencing deep psychological harm from repeated exposure to such content\u2026which obviously raises some important ethical questions.<\/li>\n<\/ul>\n<\/li>\n<li><strong>Invasion into creative sectors<\/strong>\n<ul>\n<li>ChatGPT and other AI tools (e.g. image generation software) will and already are disrupting the artistic world \u2013 pursuits that until now were uniquely human, such as painting, creative writing, and music.<\/li>\n<\/ul>\n<\/li>\n<li><strong>Responses may be insulting \/ biased<\/strong>\n<ul>\n<li>I haven\u2019t personally experienced this one, and to be honest, it doesn\u2019t bother me much (I think a bit of <a href=\"https:\/\/medium.com\/stoicism-philosophy-as-a-way-of-life\/how-to-deal-with-insults-the-stoic-way-e305891eeef2\" target=\"_blank\" rel=\"noopener\">Stoic training<\/a> would do wonders for those sunk deep in our current hyper-sensitive culture), but as this is widely reported to be a serious negative, I\u2019m including it here.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<h2>Specific limitations<\/h2>\n<p>In the above section, I outlined some large-scale drawbacks of ChatGPT and this kind of AI technology generally. Next I\u2019d like to detail ChatGPT-specific limitations, that I\u2019ve confirmed or discovered by giving it various prompts myself.<\/p>\n<p>First, I will say that a couple aspects did really impress me \u2013 one, the speed with which the chatbot answers; two, the incredibly fluid language of the responses; and three, its \u201cmemory\u201d of past dialog.<\/p>\n<p>Now onto the fun stuff!<\/p>\n<ul>\n<li><strong>Confirmed: ChatGPT is super quick to hallucinate<\/strong>\n<ul>\n<li>I asked it about a current event, which it couldn\u2019t comment on because its last training data dates to September 2021. I asked the same question, but pretended the event happened in 2019. It quickly fabricated a believable account of the event.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-672\" src=\"https:\/\/i0.wp.com\/linguamonium.com\/wp-content\/uploads\/2023\/02\/toxic-chem-ohio.png?resize=550%2C655&#038;ssl=1\" alt=\"\" width=\"550\" height=\"655\" srcset=\"https:\/\/i0.wp.com\/linguamonium.com\/wp-content\/uploads\/2023\/02\/toxic-chem-ohio.png?resize=252%2C300&amp;ssl=1 252w, https:\/\/i0.wp.com\/linguamonium.com\/wp-content\/uploads\/2023\/02\/toxic-chem-ohio.png?w=703&amp;ssl=1 703w\" sizes=\"auto, (max-width: 550px) 100vw, 550px\" data-recalc-dims=\"1\" \/><\/p>\n<ul>\n<li><strong>Confirmed: ChatGPT may categorize terms and concepts completely wrong<\/strong><\/li>\n<\/ul>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-686\" src=\"https:\/\/i0.wp.com\/linguamonium.com\/wp-content\/uploads\/2023\/02\/Inkedpolysyndeton.jpg?resize=550%2C368&#038;ssl=1\" alt=\"\" width=\"550\" height=\"368\" srcset=\"https:\/\/i0.wp.com\/linguamonium.com\/wp-content\/uploads\/2023\/02\/Inkedpolysyndeton.jpg?resize=300%2C201&amp;ssl=1 300w, https:\/\/i0.wp.com\/linguamonium.com\/wp-content\/uploads\/2023\/02\/Inkedpolysyndeton.jpg?resize=768%2C514&amp;ssl=1 768w, https:\/\/i0.wp.com\/linguamonium.com\/wp-content\/uploads\/2023\/02\/Inkedpolysyndeton.jpg?w=855&amp;ssl=1 855w\" sizes=\"auto, (max-width: 550px) 100vw, 550px\" data-recalc-dims=\"1\" \/><\/p>\n<ul>\n<li style=\"list-style-type: none;\">\n<ul>\n<li>It categorized the clause <em>it was<\/em> as a <strong>conjunction<\/strong>, giving an incorrect example with that misclassification.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-688\" src=\"https:\/\/i0.wp.com\/linguamonium.com\/wp-content\/uploads\/2023\/02\/Inkedchatgpt_seedoils.jpg?resize=550%2C338&#038;ssl=1\" alt=\"\" width=\"550\" height=\"338\" srcset=\"https:\/\/i0.wp.com\/linguamonium.com\/wp-content\/uploads\/2023\/02\/Inkedchatgpt_seedoils.jpg?resize=300%2C184&amp;ssl=1 300w, https:\/\/i0.wp.com\/linguamonium.com\/wp-content\/uploads\/2023\/02\/Inkedchatgpt_seedoils.jpg?resize=768%2C472&amp;ssl=1 768w, https:\/\/i0.wp.com\/linguamonium.com\/wp-content\/uploads\/2023\/02\/Inkedchatgpt_seedoils.jpg?w=936&amp;ssl=1 936w\" sizes=\"auto, (max-width: 550px) 100vw, 550px\" data-recalc-dims=\"1\" \/><\/p>\n<ul>\n<li style=\"list-style-type: none;\">\n<ul>\n<li>It categorized \u201colive oil\u201d as a \u201cseed oil\u201d (an olive is a fruit, not a seed).<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>Below are features that voice assistants like Google Assistant and Alexa currently have, that ChatGPT does not have. (Although I\u2019ve heard that integrations between the chatbot and voice assistants are planned or already in-progress.)<\/p>\n<ul>\n<li><strong>Asking disambiguation questions<\/strong><\/li>\n<\/ul>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-675\" src=\"https:\/\/i0.wp.com\/linguamonium.com\/wp-content\/uploads\/2023\/02\/disambig_portland.png?resize=550%2C267&#038;ssl=1\" alt=\"\" width=\"550\" height=\"267\" srcset=\"https:\/\/i0.wp.com\/linguamonium.com\/wp-content\/uploads\/2023\/02\/disambig_portland.png?resize=300%2C146&amp;ssl=1 300w, https:\/\/i0.wp.com\/linguamonium.com\/wp-content\/uploads\/2023\/02\/disambig_portland.png?w=669&amp;ssl=1 669w\" sizes=\"auto, (max-width: 550px) 100vw, 550px\" data-recalc-dims=\"1\" \/><\/p>\n<ul>\n<li style=\"list-style-type: none;\">\n<ul>\n<li>To my question about Portland, I would prefer a follow-up question from ChatGPT, asking something like, \u201cDo you mean Portland, Oregon, or Portland, Maine?\u201d, instead of having to correct its first assumption.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<ul>\n<li><strong>Pretending to emote<\/strong><\/li>\n<\/ul>\n<ul>\n<li style=\"list-style-type: none;\">\n<ul>\n<li>Some people may view this as a positive, however (as they\u2019re loath to continue blurring the line between human and AI). Then there are the people pining for a reality like that in the movie <em><a href=\"https:\/\/www.imdb.com\/title\/tt1798709\/\" target=\"_blank\" rel=\"noopener\">Her<\/a><\/em>.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-676\" src=\"https:\/\/i0.wp.com\/linguamonium.com\/wp-content\/uploads\/2023\/02\/emotions.png?resize=550%2C316&#038;ssl=1\" alt=\"\" width=\"550\" height=\"316\" srcset=\"https:\/\/i0.wp.com\/linguamonium.com\/wp-content\/uploads\/2023\/02\/emotions.png?resize=300%2C172&amp;ssl=1 300w, https:\/\/i0.wp.com\/linguamonium.com\/wp-content\/uploads\/2023\/02\/emotions.png?resize=768%2C441&amp;ssl=1 768w, https:\/\/i0.wp.com\/linguamonium.com\/wp-content\/uploads\/2023\/02\/emotions.png?w=878&amp;ssl=1 878w\" sizes=\"auto, (max-width: 550px) 100vw, 550px\" data-recalc-dims=\"1\" \/><\/p>\n<ul>\n<li style=\"list-style-type: none;\">\n<ul>\n<li>For the record, when I asked the same question to Google Assistant, it responded with \u201cBeing helpful to you is the best part of my day \ud83d\ude0a\u201d, which admittedly is a bit indirect, but still more personable.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<ul>\n<li><strong>Making use of personal user context\/information to give the most relevant responses<\/strong>\n<ul>\n<li>ChatGPT can\u2019t recommend businesses \u201cnearby\u201d because it doesn\u2019t have your location data (unlike enabled search engines and voice assistants).<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-677\" src=\"https:\/\/i0.wp.com\/linguamonium.com\/wp-content\/uploads\/2023\/02\/restaurant_nearme.png?resize=550%2C123&#038;ssl=1\" alt=\"\" width=\"550\" height=\"123\" srcset=\"https:\/\/i0.wp.com\/linguamonium.com\/wp-content\/uploads\/2023\/02\/restaurant_nearme.png?resize=300%2C67&amp;ssl=1 300w, https:\/\/i0.wp.com\/linguamonium.com\/wp-content\/uploads\/2023\/02\/restaurant_nearme.png?w=661&amp;ssl=1 661w\" sizes=\"auto, (max-width: 550px) 100vw, 550px\" data-recalc-dims=\"1\" \/><\/p>\n<ul>\n<li style=\"list-style-type: none;\">\n<ul>\n<li>It does give restaurant recommendations within a particular location if your query contains the location name, such as \u201ccan you recommend a good Thai restaurant in San Jose?\u201d<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<ul>\n<li><strong>Controlling device behavior<\/strong>\n<ul>\n<li>By \u201cdevice behavior\u201d, I mean actions like setting an alarm on your phone, turning off the lights in your kitchen, or playing a song on your smart speaker. ChatGPT does not (yet) have the ability to remotely control any smart devices.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<h2>In closing<\/h2>\n<p>There is a personal side to the chatbot brouhaha for me, as I was recently let go in the <a href=\"https:\/\/www.cnbc.com\/2023\/01\/21\/google-employees-scramble-for-answers-after-layoffs-hit-long-tenured.html\" target=\"_blank\" rel=\"noopener\">giant wave of Google \/ tech layoffs<\/a>\u2026 And although it seems like the layoff decisions were complex, multifactored, and mainly algorithmic, I <em>was <\/em>working in a field (natural language understanding \u2013 or NLU \u2013 for Google Assistant) into which Google leadership is now incredibly keen to push LLM technology. And who needs linguists when you have LLMs?! (An answer: If you followed the <a href=\"https:\/\/www.theverge.com\/2023\/2\/8\/23590864\/google-ai-chatbot-bard-mistake-error-exoplanet-demo\" target=\"_blank\" rel=\"noopener\">recent Bard demo debacle<\/a>, which <a href=\"https:\/\/www.cnn.com\/2023\/02\/08\/tech\/google-ai-bard-demo-error\/index.html\" target=\"_blank\" rel=\"noopener\">cost Google $100 billion from a single-day drop in shares<\/a>, well\u2026 my linguist colleague found the training data source and we quickly spotted the ambiguous phrasing and context that contributed to Bard\u2019s mistake<a href=\"#_ftn4\" name=\"_ftnref1\">[4]<\/a>. Hmm\u2026)<\/p>\n<p>I have to end with the hackneyed but nevertheless true observation (repeated across many other recent articles on ChatGPT and AI) that, despite the host of current issues, it behooves us all to reconcile with this technology, and to improve it across every dimension \u2013 because it is very likely here to stay.<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>Note: The featured images at the top of the post were made with <a href=\"https:\/\/www.midjourney.com\/home\/?callbackUrl=%2Fapp%2F\" target=\"_blank\" rel=\"noopener\">Midjourney<\/a> (an image generator), using the prompt \u201ca babel-like explosion of words a tower a creepy beautiful woman \u2013 all in the style of Anne Bachelier\u201d. Bizarrely, I tried multiple prompts, with more explicit focus on \u201cwords\u201d and \u201cletters\u201d, but could not get a picture that rendered actual words of any kind.<\/p>\n<p><a href=\"#_ftnref1\" name=\"_ftn1\">[1]<\/a> See fuller list in <a href=\"https:\/\/en.wikipedia.org\/wiki\/Language_model#Notable_language_models\" target=\"_blank\" rel=\"noopener\">https:\/\/en.wikipedia.org\/wiki\/Language_model#Notable_language_models<\/a><\/p>\n<p><a href=\"#_ftnref2\" name=\"_ftn2\">[2]<\/a> <a href=\"https:\/\/towardsdatascience.com\/not-all-rainbows-and-sunshine-the-darker-side-of-chatgpt-75917472b9c\" target=\"_blank\" rel=\"noopener\">https:\/\/towardsdatascience.com\/not-all-rainbows-and-sunshine-the-darker-side-of-chatgpt-75917472b9c<\/a><\/p>\n<p><a href=\"#_ftnref3\" name=\"_ftn3\">[3]<\/a> <a href=\"https:\/\/time.com\/6247678\/openai-chatgpt-kenya-workers\/\" target=\"_blank\" rel=\"noopener\">https:\/\/time.com\/6247678\/openai-chatgpt-kenya-workers\/<\/a><\/p>\n<p><a href=\"#_ftnref4\" name=\"_ftn1\">[4]<\/a> Email me or otherwise get in touch if you\u2019d like to know the details!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Okay. I didn\u2019t want to write a post on ChatGPT, because its public release has been like shrapnel through the meat of the internet \u2013 but at the end of the day this IS a language blog, and I would be remiss for not doing due diligence. If you\u2019ve been under a rock for the&#8230;<\/p>\n","protected":false},"author":1,"featured_media":680,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_newsletter_tier_id":0,"footnotes":"","jetpack_publicize_message":"","jetpack_is_tweetstorm":false,"jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","enabled":false}}},"categories":[20,22],"tags":[38,205,202,203,204],"class_list":["post-671","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-nlu-nlp-ml-ai","category-polemical-topics","tag-ai","tag-chatbots","tag-chatgpt","tag-llms","tag-machine-learning"],"jetpack_publicize_connections":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v21.7 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>AI\u2019s babbling towers: ChatGPT and large language models - Linguamonium<\/title>\n<meta name=\"description\" content=\"A summary of how ChatGPT and LLMs work, a list of major concerns about the technology, and an analysis of specific limitations.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/linguamonium.com\/?p=671\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"AI\u2019s babbling towers: ChatGPT and large language models - Linguamonium\" \/>\n<meta property=\"og:description\" content=\"A summary of how ChatGPT and LLMs work, a list of major concerns about the technology, and an analysis of specific limitations.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/linguamonium.com\/?p=671\" \/>\n<meta property=\"og:site_name\" content=\"Linguamonium\" \/>\n<meta property=\"article:published_time\" content=\"2023-03-01T07:01:51+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-02-14T17:48:45+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/linguamonium.com\/wp-content\/uploads\/2023\/02\/a_babel-like_explosion_of_words_a_tower_a_creepy_beautiful_woman_anne_bachelier_.png\" \/>\n\t<meta property=\"og:image:width\" content=\"787\" \/>\n\t<meta property=\"og:image:height\" content=\"787\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"hannah\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@linguamonium\" \/>\n<meta name=\"twitter:site\" content=\"@linguamonium\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"hannah\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"9 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/linguamonium.com\/?p=671#article\",\"isPartOf\":{\"@id\":\"https:\/\/linguamonium.com\/?p=671\"},\"author\":{\"name\":\"hannah\",\"@id\":\"https:\/\/linguamonium.com\/#\/schema\/person\/f6a9c49248cb623f9a6061aaacff0238\"},\"headline\":\"AI\u2019s babbling towers: ChatGPT and large language models\",\"datePublished\":\"2023-03-01T07:01:51+00:00\",\"dateModified\":\"2024-02-14T17:48:45+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/linguamonium.com\/?p=671\"},\"wordCount\":1703,\"commentCount\":1,\"publisher\":{\"@id\":\"https:\/\/linguamonium.com\/#\/schema\/person\/f6a9c49248cb623f9a6061aaacff0238\"},\"keywords\":[\"AI\",\"chatbots\",\"chatgpt\",\"LLMs\",\"machine learning\"],\"articleSection\":[\"NLU\/NLP\/ML\/AI\",\"Polemical Topics\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/linguamonium.com\/?p=671#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/linguamonium.com\/?p=671\",\"url\":\"https:\/\/linguamonium.com\/?p=671\",\"name\":\"AI\u2019s babbling towers: ChatGPT and large language models - Linguamonium\",\"isPartOf\":{\"@id\":\"https:\/\/linguamonium.com\/#website\"},\"datePublished\":\"2023-03-01T07:01:51+00:00\",\"dateModified\":\"2024-02-14T17:48:45+00:00\",\"description\":\"A summary of how ChatGPT and LLMs work, a list of major concerns about the technology, and an analysis of specific limitations.\",\"breadcrumb\":{\"@id\":\"https:\/\/linguamonium.com\/?p=671#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/linguamonium.com\/?p=671\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/linguamonium.com\/?p=671#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/linguamonium.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"AI\u2019s babbling towers: ChatGPT and large language models\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/linguamonium.com\/#website\",\"url\":\"https:\/\/linguamonium.com\/\",\"name\":\"Linguamonium\",\"description\":\"A tumult of linguistic musings\",\"publisher\":{\"@id\":\"https:\/\/linguamonium.com\/#\/schema\/person\/f6a9c49248cb623f9a6061aaacff0238\"},\"alternateName\":\"Linguamo\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/linguamonium.com\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":[\"Person\",\"Organization\"],\"@id\":\"https:\/\/linguamonium.com\/#\/schema\/person\/f6a9c49248cb623f9a6061aaacff0238\",\"name\":\"hannah\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/linguamonium.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/linguamonium.com\/wp-content\/uploads\/2018\/06\/cropped-dummeh-babboh-2-1.png\",\"contentUrl\":\"https:\/\/linguamonium.com\/wp-content\/uploads\/2018\/06\/cropped-dummeh-babboh-2-1.png\",\"width\":512,\"height\":512,\"caption\":\"hannah\"},\"logo\":{\"@id\":\"https:\/\/linguamonium.com\/#\/schema\/person\/image\/\"},\"sameAs\":[\"https:\/\/linguamonium.com\",\"www.linkedin.com\/in\/hannah-vanbrunt\",\"https:\/\/twitter.com\/linguamonium\"],\"url\":\"https:\/\/linguamonium.com\/?author=1\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"AI\u2019s babbling towers: ChatGPT and large language models - Linguamonium","description":"A summary of how ChatGPT and LLMs work, a list of major concerns about the technology, and an analysis of specific limitations.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/linguamonium.com\/?p=671","og_locale":"en_US","og_type":"article","og_title":"AI\u2019s babbling towers: ChatGPT and large language models - Linguamonium","og_description":"A summary of how ChatGPT and LLMs work, a list of major concerns about the technology, and an analysis of specific limitations.","og_url":"https:\/\/linguamonium.com\/?p=671","og_site_name":"Linguamonium","article_published_time":"2023-03-01T07:01:51+00:00","article_modified_time":"2024-02-14T17:48:45+00:00","og_image":[{"width":787,"height":787,"url":"https:\/\/linguamonium.com\/wp-content\/uploads\/2023\/02\/a_babel-like_explosion_of_words_a_tower_a_creepy_beautiful_woman_anne_bachelier_.png","type":"image\/png"}],"author":"hannah","twitter_card":"summary_large_image","twitter_creator":"@linguamonium","twitter_site":"@linguamonium","twitter_misc":{"Written by":"hannah","Est. reading time":"9 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/linguamonium.com\/?p=671#article","isPartOf":{"@id":"https:\/\/linguamonium.com\/?p=671"},"author":{"name":"hannah","@id":"https:\/\/linguamonium.com\/#\/schema\/person\/f6a9c49248cb623f9a6061aaacff0238"},"headline":"AI\u2019s babbling towers: ChatGPT and large language models","datePublished":"2023-03-01T07:01:51+00:00","dateModified":"2024-02-14T17:48:45+00:00","mainEntityOfPage":{"@id":"https:\/\/linguamonium.com\/?p=671"},"wordCount":1703,"commentCount":1,"publisher":{"@id":"https:\/\/linguamonium.com\/#\/schema\/person\/f6a9c49248cb623f9a6061aaacff0238"},"keywords":["AI","chatbots","chatgpt","LLMs","machine learning"],"articleSection":["NLU\/NLP\/ML\/AI","Polemical Topics"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/linguamonium.com\/?p=671#respond"]}]},{"@type":"WebPage","@id":"https:\/\/linguamonium.com\/?p=671","url":"https:\/\/linguamonium.com\/?p=671","name":"AI\u2019s babbling towers: ChatGPT and large language models - Linguamonium","isPartOf":{"@id":"https:\/\/linguamonium.com\/#website"},"datePublished":"2023-03-01T07:01:51+00:00","dateModified":"2024-02-14T17:48:45+00:00","description":"A summary of how ChatGPT and LLMs work, a list of major concerns about the technology, and an analysis of specific limitations.","breadcrumb":{"@id":"https:\/\/linguamonium.com\/?p=671#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/linguamonium.com\/?p=671"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/linguamonium.com\/?p=671#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/linguamonium.com\/"},{"@type":"ListItem","position":2,"name":"AI\u2019s babbling towers: ChatGPT and large language models"}]},{"@type":"WebSite","@id":"https:\/\/linguamonium.com\/#website","url":"https:\/\/linguamonium.com\/","name":"Linguamonium","description":"A tumult of linguistic musings","publisher":{"@id":"https:\/\/linguamonium.com\/#\/schema\/person\/f6a9c49248cb623f9a6061aaacff0238"},"alternateName":"Linguamo","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/linguamonium.com\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":["Person","Organization"],"@id":"https:\/\/linguamonium.com\/#\/schema\/person\/f6a9c49248cb623f9a6061aaacff0238","name":"hannah","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/linguamonium.com\/#\/schema\/person\/image\/","url":"https:\/\/linguamonium.com\/wp-content\/uploads\/2018\/06\/cropped-dummeh-babboh-2-1.png","contentUrl":"https:\/\/linguamonium.com\/wp-content\/uploads\/2018\/06\/cropped-dummeh-babboh-2-1.png","width":512,"height":512,"caption":"hannah"},"logo":{"@id":"https:\/\/linguamonium.com\/#\/schema\/person\/image\/"},"sameAs":["https:\/\/linguamonium.com","www.linkedin.com\/in\/hannah-vanbrunt","https:\/\/twitter.com\/linguamonium"],"url":"https:\/\/linguamonium.com\/?author=1"}]}},"jetpack_featured_media_url":"https:\/\/i0.wp.com\/linguamonium.com\/wp-content\/uploads\/2023\/02\/a_babel-like_explosion_of_words_a_tower_a_creepy_beautiful_woman_anne_bachelier_.png?fit=787%2C787&ssl=1","jetpack_likes_enabled":true,"jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/linguamonium.com\/index.php?rest_route=\/wp\/v2\/posts\/671","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/linguamonium.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/linguamonium.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/linguamonium.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/linguamonium.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=671"}],"version-history":[{"count":8,"href":"https:\/\/linguamonium.com\/index.php?rest_route=\/wp\/v2\/posts\/671\/revisions"}],"predecessor-version":[{"id":689,"href":"https:\/\/linguamonium.com\/index.php?rest_route=\/wp\/v2\/posts\/671\/revisions\/689"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/linguamonium.com\/index.php?rest_route=\/wp\/v2\/media\/680"}],"wp:attachment":[{"href":"https:\/\/linguamonium.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=671"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/linguamonium.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=671"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/linguamonium.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=671"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}