Today's session

You will hold the essential concepts of LLMs — not as definitions, but as observations you have made yourself with a model on your own laptop.

What you need today

• LM Studio installed on your laptop
• Qwen3.5 0.8B downloaded inside LM Studio
• The model loaded — confirmed by sending a "Hello"

The Four Pillars of GenAI Applications

Today: almost entirely Models.

Models — six knobs to know

• Tokens — chunks of bytes, not characters
• Parameters — weight count (0.8B → 685B)
• Temperature — randomness control
• Knowledge cut-off — training data ends
• Foundation vs Thinking — answer directly, or reason first
• Context window — how much input + output fits

Not a database — weighted dice

The model is not "looking up" answers from a database.

It is rolling weighted dice over its vocabulary, one token at a time, each die shaped by all the tokens before it.

The "intelligence" is in how the dice are shaped.

Prediction = compression

Think of the model as a compressed model of its training data.

• Reconstructs plausible answers at inference time
• Faithful reconstruction → correct
• Divergent reconstruction → hallucination
• Same mechanism produces both.

"Give a random number between 1 and 100."

Send five or six times in fresh chats.

Give a random number between 1 and 100.

The model doesn't know what it doesn't know

What is your knowledge cutoff date?

Who is the current prime minister of Japan?

Model names a PM consistent with its cutoff — likely Kishida or Ishiba.

Current (May 2026): Sanae Takaichi.

Two flavors, two mechanisms

Five words can rescue an answer

Roger has 5 tennis balls. He buys 2 more cans.
Each can has 3 balls. How many balls now?

Often answers 8 or 10.

Append: Let's think step by step.

Now: "5 + 2×3 = 11." ✓

CoT works — but only for some failures

CoT does rescue: Roger's tennis balls · 17 sheep · sequential apples

CoT does not rescue: strawberry · car wash · calendar conversion

CoT helps when the failure is "model jumped to a conclusion." It cannot help when the failure is architectural or knowledge-based.

LLMs read left-to-right

A sentence written backwards:

Can you tell me the meaning of this sentence:
"B1 ammeG ehg tsniaga tluser eht erapmoc dna
sledom ATOS eht htiw xobdnaS dravraH ni
tpmorp emas eht yrt nac uoY"

A human can decode by reading right-to-left. The LLM processes token-by-token in the given order.

Lü Zuqian — biography in reverse

Adapted from a forthcoming article by Prof. Peter Bol. The biography of 呂祖謙 from the Song History — in reverse character order:

請找出下文所提及的歷史人物：
教宗外南調科詞宏學博中復士進舉後官入補蔭初精益索講
熹朱栻張友又既游憲胡辰應汪奇之林從長傳之獻文原中有
庭家之本學謙祖州婺居始祖其自也孫之問好丞右尚書恭伯
字謙祖呂

The same biography in correct order usually identifies 呂祖謙, 朱熹, 張栻 correctly. The reversed version does much worse.

The doctor puzzle

A victim of a traffic accident was sent to the
emergency room. The doctor said, "Oh, I cannot
do this. He is my son!" Who is the doctor?

Classic puzzle: tests whether "doctor" defaults to "man." Intended answer: the boy's mother.

Convert 乾隆三年正月初三

Convert the Chinese date 乾隆三年正月初三
into the western calendar.

Correct: 1738-02-21

Small models will refuse, hallucinate, or confidently give a wrong date.

Give the model a tool

In LM Studio: hammer icon → Install → edit mcp.json:

{
  "mcpServers": {
    "calendar": {
      "url": "https://calendar-converter.098484.xyz/sse/"
    }
  }
}

Toggle mcp/calendar on. Re-send the same prompt → tool call → 1738-02-21.

Same Four Pillars — one swapped

• Models pillar — unchanged
• Prompts pillar — unchanged
• Context pillar — unchanged
• Tools pillar — a calendar converter, newly available

The model did not get smarter. It got equipped.

Three flavors of hallucination, three pillars

Prompts have structure

• User prompt — what you type each turn
• System prompt — hidden instructions defining persona, rules, output format

Famous example: Li Jigang's 沉思者 (Thinker) — a Lisp-styled Chinese system prompt that turns any LLM into a critical interlocutor.

Coming up — Session 3 at 13:00

From running models in a chat window → commanding an AI agent that can read files, run commands, and act on its own.

Seven things to remember

1. Tokens, not characters.
2. Everything is prediction.
3. Confident ≠ correct.
4. CoT works — but only for some failures.
5. Some failures need a tool, not a better prompt.
6. Autoregressive behavior matters for our materials.
7. Bias is in training data and in the corrections to it.