StellarGlyph

Prompts are the text you provide to an AI tool to receive some output from it. A Human-AI interaction consists of the following 3 key components:

Prompt: The instruction or question you type into the AI.
Response: What the AI writes back after reading your prompt
Context: Any extra information you give the AI so it understands the situation (e.g., “I’m writing a blog for beginners”).

Getting your prompting honed can dramatically improve the quality of results you get back from an LLM. It could be a question, an instruction, a bit of background context, or all three at once. Everything you type is your prompt. The AI reads it, interprets it, and generates a response based on what you've written.

Prompt engineering is the act of refining prompts to get better results. Now, depending on what you are trying to achieve, "better" could mean different things. In some instances it could mean a concise response, in others a larger, more valid response may be desired.

To be able to write a good prompt, it's useful to understand some more about the Foundational models you are likely to use on a day to day basis (Gemini, ChatGPT, Claude, etc). Let's dive into some of the inherent limitations of this technology.

Near enough is not good enough

One of the more surprising things about AI models is that they are not actually looking up answers. There is no internal database being queried or a list of files containing this information. Instead, the model is effectively doing very sophisticated pattern matching: predicting, word by word, what a plausible and helpful response looks like based on everything it was trained on. This is called probabilistic generation.

The model is not certain about anything it says. It is making its best statistical guess at every step. Most of the time, those guesses are impressively good. But the mechanism itself has no concept of truth or accuracy. It has a concept of likelihood.

It's something like autocomplete on your phone, but trained on huge volumes of data and capable of writing essays. Your phone suggests "morning" after "good" not because it knows what time it is, but because that combination appears frequently. LLMs work on a similar principle, just at a different scale and complexity. This matters because it means AI output can sound completely authoritative while being partially or entirely wrong. The model does not flag uncertainty the way a cautious human would. It does not say "I'm not sure about this bit." It just continues generating text that sounds plausible.

Fake it till you make it

This probabilistic nature is what leads to one of the most talked-about quirks of AI: hallucination. This is the term used when a model produces something that is factually incorrect but presented with complete confidence.

You might ask an AI to list five academic papers on a topic, and it will return five papers with plausible-sounding titles, realistic author names, and credible journal references. Some of them may simply not exist. The model has not lied to you in any meaningful sense. It has simply generated text that fits the pattern of what a list of academic papers looks like. Hallucinations tend to occur more often when:

The topic is niche or obscure, where the training data was thinner
The requirement is for specific facts, figures, dates or citations
The model is pushed to fill gaps it does not have the inherent ability to answer

This is not a reason to avoid AI tools. It is a reason to treat their output the way you would treat a first draft from a very fast, very confident intern. This is why it is so important to review and check all work generated by AI.

Good prompting can reduce the likelihood of hallucination by giving the model more to work with, narrowing the space of plausible responses and keeping it closer to what you actually know to be true. But it cannot eliminate it entirely. A healthy habit of reviewing and verifying AI output is simply part of working with these tools responsibly.

All the gear, no idea

These models are trained on large volumes of textual information: books, articles, websites, code, academic papers, forum posts and much more. The scale is truly massive and almost impossible to picture. This is what makes them feel so capable. You can ask them about Roman history, tax law, bread recipes or satellite engineering and they can respond plausibly to all of it. This knowledge is vast, but it does mean that these models default to providing the average response that will be appropriate for most situations.

As these models have absorbed such a vast and varied range of text, they have learned to produce responses that are broadly acceptable across a huge number of possible situations. If the prompt is too general, the model does not know how it should modify or frame the topic to make it more appropriate for your use case. So it defaults to the middle ground: the kind of answer that would be roughly appropriate for most people asking that question. That middle ground leads to the model sounding more like a Wikipedia article than say an email from your business. It is technically correct, often comprehensive but probably too generic for your needs.

A prompt like "write me a marketing email" could have come from a charity, a law firm, a street food van, or a multinational. The model has no way to tell. So it produces something that vaguely fits all of them, which means it likely doesn't match the style, content or tone of the response you were initially after.

Amnesia, every time

Each conversation with an AI typically starts completely fresh. The model does not know your name, your industry, your tone of voice, what you tried last week, or what your customers respond to. Every single prompt is, from the model's perspective, the beginning of a conversation with a stranger. If you've watched the Adam Sandler and Drew Barrymore film 50 First Dates, you'll get an idea of what this looks like (note, the LLM is the Barrymore character but with sufficient prompting you could get it to sound like Sandler).

This lack of memory means:

Anything omitted from the prompt the model will fill in with assumptions drawn from its training data. Those assumptions are not personalised to you.
A model has no internal mechanism to keep track of previous conversations so you need to provide this history (or context)

The assumptions or guesses a model makes are drawn from statistical averages across millions of documents. Until very recently, models didn't ask clarifying questions the way a human colleague would. If you asked a colleague to write you a report and your brief was a bit thin, they would probably come back with a few questions before getting started. What is it for? Who is reading it? How long should it be? What tone are you after? Most AI tools, by default, will not do this. They will make their best guess and run with it. That guess is based on probability, not intuition or shared context. You can see where this could lead to problems. You may have experienced this already, the response seems on point until there's some text which either isn't true or isn't relevant to the rest of the response.

General knowledge is not the same as relevant knowledge

There is a meaningful difference between knowing a lot and knowing the right thing for a specific situation. A model that has read extensively about customer service can still produce a response that is completely wrong for your particular business, your particular customer or the specific issue.

Wide general knowledge is genuinely impressive. But it can also create a false sense of confidence in the output. The response may sound well-informed while being entirely misaligned with what you needed. This is why reviewing AI output, rather than just accepting it, always matters.

Additionally models are trained with data that is available up to a certain point. For example, asking Claude Sonnet 4.6 "What is the latest training data you have knowledge of?" gives the following:

My knowledge cuts off at the end of August 2025,
so anything that happened after that
point I won't have reliable information on.

Is there something specific you'd like me to check or verify?
I can search the web if you need current information

This issue has largely been reduced in 2026 because most (but not all) models have the ability to search the internet for information. However they do this using internal tools and perhaps they end up selecting and summarising a different set of websites or references compared to what you would have chosen.

Good Prompting

Prompting is how you bridge the gap between these inherent limitations which are part and parcel of the operation of a model. All of these issues we've discussed: the probabilistic nature of models, the defaults toward the average, the lack of personal context, the gap between general and specific knowledge may sound like a lot of limitations. But they can all be addressed with a well-constructed prompt. When you give the model your role, your goal, your audience, your tone, and your constraints, you are not just asking more clearly. You are compensating for the structural gaps that exist between a general-purpose model and your very specific situation. That is why prompting is less about learning tricks and more about understanding what the model is missing, and then supplying it.