35 Jahre Zurück in die Zukunft II: “Wo wir hinfahren, brauchen wir keine Straßen!”

Vor 35 Jahren reiste Marty McFly zurück in die Zukunft – ins Jahr 1955, und ins Jahr 2015. Wer damals Kind war, lernte ein neues Wort: Almanach. Von Peter Osteried (Zurück in die Zukunft, Film)

Vor 35 Jahren reiste Marty McFly zurück in die Zukunft - ins Jahr 1955, und ins Jahr 2015. Wer damals Kind war, lernte ein neues Wort: Almanach. Von Peter Osteried (Zurück in die Zukunft, Film)

Why AI language models choke on too much text

Compute costs scale with the square of the input size. That’s not great.

Large language models represent text using tokens, each of which is a few characters. Short words are represented by a single token (like "the" or "it"), whereas larger words may be represented by several tokens (GPT-4o represents "indivisible" with "ind," "iv," and "isible").

When OpenAI released ChatGPT two years ago, it had a memory—known as a context window—of just 8,192 tokens. That works out to roughly 6,000 words of text. This meant that if you fed it more than about 15 pages of text, it would “forget” information from the beginning of its context. This limited the size and complexity of tasks ChatGPT could handle.

Today’s LLMs are far more capable:

Read full article

Comments

Why AI language models choke on too much text

Compute costs scale with the square of the input size. That’s not great.

Large language models represent text using tokens, each of which is a few characters. Short words are represented by a single token (like "the" or "it"), whereas larger words may be represented by several tokens (GPT-4o represents "indivisible" with "ind," "iv," and "isible").

When OpenAI released ChatGPT two years ago, it had a memory—known as a context window—of just 8,192 tokens. That works out to roughly 6,000 words of text. This meant that if you fed it more than about 15 pages of text, it would “forget” information from the beginning of its context. This limited the size and complexity of tasks ChatGPT could handle.

Today’s LLMs are far more capable:

Read full article

Comments