What are LLMs and how to think from their perspective

agents-101 · 2026-05-21 · It all began here btw

LLMs are now everywhere. So much so that they are now synonymous to AI itself. AI is a whole bigger umbrella concept. I don't think we have all agreed to one sole description of what AI means tbh, but LLM sure is a small part of it, and also is well defined.

I am writing this blog with technical POV in mind. I might add some code blocks here and there. But as I might share this with people who have never written code ever, or are simply not interested in it, I will try to make this as explanatory as possible as compared to tutorial-ly. Full disclosure the Agents 101 series is going to be a technical one. So that part won't go anywhere.

As we go on to the posts where we talk about building AI Agents etc. or your own Agent harness, things will inevitably get more technical than exploratory.

So what is an LLM, really?

The google definition of LLM is more than enough to comprehend it:

Large Language Models (LLMs) are advanced artificial intelligence systems designed to understand, process, and generate human language.

The best way to imagine this, is by picturing how two humans talk to each other. Person A might say "hey!", and if you are Person B, you will most likely say "Hi there! How are you!"

how humans talk. Simple hi c- hello give and take convo — how humans talk. Simple hi - hello give and take convo

Why so?

You know the answer. It's because you are taught that. Since childhood you have been trained to talk a certain way, answer questions a certain way. As we grow we might learn to talk in more sophisticated manner, understand how to read between the lines, or simply gain more knowledge to answer questions more precisely

And that is exactly how LLMs talk! It is trained to say "Hello" back at you when you say "hi" to it. It is trained to say "Capital of France is Paris" when you ask for the capital of france because it learnt it. Over terabytes of data, where human conversations and texts were present, went into training these models

how humans talk. But instead of talking to another human, we talk to an LLM

Let's go a bit more granular

A language model has one job. Given an incomplete string of words, it must predict the next word. The words to say when someone says "hi", is most likely "Hello!", most likely followed by "how", most likely followed by "are", mostly like followed by "you", most likely followed by "doing?". If you have used ChatGPT or Claude or any other AI, you would see the LLM generating the response word by word, and this exactly why.

When we talk, we don't mentally prepare the entire paragraph before uttering the first word. We don't conceptualise the "Hello! How are you doing?" sentence in our mind before we speak, and definitely not those long monologue or answers we might give. We say one word at a time, and subconciously our brain stitches words together

And that is it. This is how you can think about how you speak to understand LLMs. They also predict the next words, after another, which finally ends up being the complete response.

We talk word by word, not entire sentences in one swing. And so do LLMs

Basically a machine, which takes your query, text whatever you throw at it, and it returns a response, as if a human wrote it.

I'm afraid I cannot go into technical depth of what makes an LLM. As ML is not my forte.

But if you really want to know it, you can enter this beautiful, deep blue rabbit hole of a universe of Machine Learning. Best place to start for now would be this blog titled "How Large Language Models work"

How are they different from us?

If you use LLMs daily (I'm sure you do), you must know that when you start one session with ChatGPT or Claude, and then after a long time, the response quality drops, or your queries are resolved, and you start a new session, the LLM has completely "forgotten" about what you did who you are etc.

Now, ChatGPT, and Claude.ai have a feature called "MEMORY" but that is an external hatch to preserve, well, memory. But why do we need an external hatch? Isn't supposed to mimic humans talking?

Turns out there are some important technical considerations

Remember when I said that LLMs are...

Basically a machine, which takes your query, text whatever you throw at it, and it returns a response, as if a human wrote it.

Well this machine ONLY takes, in your questions and gives you an answer. It does not have a brain to store memory. It does not possess the ability to remember

In technical terms, LLMs are stateless

What is statelessness you ask? Well google again was precise

Statelessness in software engineering refers to a design approach where a system processes each request independently, retaining no memory or session information from past interactions. It ensures high scalability, reliability, and simple load distribution across server instances.

Think from this POV, there are 8 billion people in the world. And if you used AI heavily you must have created more than 1 account just to use AI. For each user, We will need to maintain a separate LLM altogether, which will hold YOUR context ( ...foreshadowing ). But instead LLMs try to be simpler. I can ask a questions to the same LLM, the same piece of code running on some server in USA, as you, and get a response, which feels personal to me, without us each other knowing, or our queries interferring each other.

To summarise, the LLM only takes an input ( your query ... for now), and that is it, and gives a response out.

Welcome to the realm of "context"

But then a question can come in, how is an LLM able to hold a convo? How is it able to infer from some message I sent, in the same session.

Turns out, your current question is not just the only thing it takes. It takes the WHOLE conversation as an input.

The list of questions and answers, back and forth, between you and the LLM is the context

Context has 3 parts

System prompt -> this is the heart of the LLM. This is the instruction given to the LLM on how to behave, who it is, what is it's personality, what it can do, can't do. It is the agent's identity. If tomorrow ChatGPT or Claude.ai starts talking to you differently, it means the system prompt for the LLM is now changed. This is literally plain text information. LLMs can "comprehend" human text, and it also comprehends it to give responses how you like
User messages -> your queries, text, whatever
Assistant messages -> the LLMs responses

A chain of system prompt, user message, the LLM's response, user message again , and so on, is what essentially is "context"

To quote google

In Large Language Models (LLMs), context is the information the model considers when generating a response. It functions like a working memory, containing your current prompt, previous conversation history, and any background documents you've provided.

Everytime you chat with an LLM in a given "session", it recieves the entire chat history, the user <-> LLM back and forth entirely for generating the response.

If we don't send the previous chats too, it will "forget" (in quotes as for forgetting one must possess the ability to remember). It will hence treat your new query as a new session itself.

How to think from their perspective?

It is actually very simple. Just like how it is supposed to be. Simply understanding these points will help

The Agent does not know anything beyong it's own training data and systemt prompt. Hence do not assume it would infer anything on it's own, or make assumptions which seem natural to you
The only thing the LLM has context for is the current chat, so it will not remember any of your old conversations. You will have to setup a memory system or simply copy paste the important bits
LLMs cannot do beyond simply talking. You might say Wait but claude can write code and show me the output or surf the internet and tell me matching results. What is that? We will get there in the next section where we understand how LLMs can dothings

Once you know these, you will have good idea on how an LLM will respond. Bottom line - It knows no world beyond your system prompt, conversation and whatever is in it's training data ( which is unreliable. Always assume it does not know things, for a good mental model )

How to percieve an LLM

LLMs are a piece of code, which takes human readable text as input and returns something which one can consider a response written like a human
LLMs are stateless, and by the virtue of it, will not remember you. Each new session is talking to an LLM who is your stranger
LLMs live on context, which consists of system prompt holding the behaviour and additional knowledge for LLM, and conversation history between the user and LLMs

In the next section, we will use the OpenAI SDK to see how you can chat with an LLM too, but using python code. THis is a step closer to writing an AI Agent. and you have no idea how close we are too it. Trust me, it is not that deep to build "AI Agents" or "Agentic workflows".