{"id":659928,"date":"2026-05-09T14:16:24","date_gmt":"2026-05-09T14:16:24","guid":{"rendered":"https:\/\/www.newsbeep.com\/au\/659928\/"},"modified":"2026-05-09T14:16:24","modified_gmt":"2026-05-09T14:16:24","slug":"llm-from-scratch-is-a-hands-on-workshop-where-you-write-every-piece-of-an-ai-from-nothing","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/au\/659928\/","title":{"rendered":"LLM From Scratch is a hands-on workshop where you write every piece of an AI from nothing"},"content":{"rendered":"\n

Summary<\/p>\n

Free hands-on “LLM From Scratch” course that builds a tiny LLM from nothing to a working model.<\/p>\n

It comes in six parts: tokenization, transformer, training loop, generation, scaling experiments, and a poetry competition.<\/p>\n

Built for hobbyists – runs on a laptop; see the LLM From Scratch GitHub for details.<\/p>\n

Have you ever wondered how an LLM works? Perhaps you know some of the basics, such as how they use tokens to get the job done, and how their text generation is somewhat like the autocomplete feature on your phone; however, how would you go about making one from scratch? Where would you even start?<\/p>\n

If you want to learn how to make an LLM from scratch, then you’ll need to check out this DIY course, called, uh…”LLM From Scratch.” As one of the best examples of the phrase “does what it says on the tin,” LLM From Scratch starts you off with nothing and walks you through a miniature course to make your own AI.<\/p>\n


\n \"A<\/p>\n

<\/a><\/p>\n

Related<\/p>\n

\t\t
\n\t\t\t6 ways anyone can use LM Studio and a local LLM on their PC
\n\t\t<\/a><\/p>\n

Most people can find a use for a local LLM on their PC, and here’s how I use mine.<\/p>\n

LLM From Scratch takes you from nothing to creating the ideal AI poet<\/p>\n

It’s broken up into six parts<\/p>\n

\"Kate<\/p>\n

As spotted by Hackaday<\/a>, LLM From Scratch is a free course that takes you from absolutely nothing to creating your own AI. You’ll start off by learning about tokenization and how LLMs parse data, and then finish by training a model on poetry until you’re happy with it. You’re not going to be taking on ChatGPT or Gemini with it, but it will run on a laptop just fine, so it’s great for hobbyists.<\/p>\n

Here are all the courses you can take:<\/p>\n

Part<\/p>\n

What You’ll Write<\/p>\n

Concepts<\/p>\n

Part 1: Tokenization<\/a><\/p>\n

Character-level tokenizer<\/p>\n

Character encoding, vocabulary size, why BPE fails on small data<\/p>\n

Part 2: The Transformer<\/a><\/p>\n

Full GPT model architecture<\/p>\n

Embeddings, self-attention, layer norm, MLP blocks<\/p>\n

Part 3: The Training Loop<\/a><\/p>\n

Complete training pipeline<\/p>\n

Loss functions, AdamW, gradient clipping, LR scheduling<\/p>\n

Part 4: Text Generation<\/a><\/p>\n

Inference and sampling<\/p>\n

Temperature, top-k, autoregressive decoding<\/p>\n

Part 5: Putting It All Together<\/a><\/p>\n

Train on real data, experiment<\/p>\n

Loss curves, scaling experiments, next steps<\/p>\n

Part 6: Competition<\/a><\/p>\n

Train the best AI poet<\/p>\n

Find datasets, scale up, submit your best poem<\/p>\n

If you’d like to learn more, be sure to head over to the LLM From Scratch GitHub<\/a> page to read more about the inspiration and what to expect when undertaking this workshop.<\/p>\n


\n \"Close-up<\/p>\n

<\/a><\/p>\n

Related<\/p>\n

\t\t
\n\t\t\tAfter a year of self-hosting LLMs, I realized the real bottleneck isn\u2019t the GPU
\n\t\t<\/a><\/p>\n

Hardware is just the entry fee for local intelligence.<\/p>\n","protected":false},"excerpt":{"rendered":"Summary Free hands-on “LLM From Scratch” course that builds a tiny LLM from nothing to a working model.…\n","protected":false},"author":2,"featured_media":659929,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[256,254,255,64,63,105],"class_list":["post-659928","post","type-post","status-publish","format-standard","has-post-thumbnail","category-artificial-intelligence","tag-ai","tag-artificial-intelligence","tag-artificialintelligence","tag-au","tag-australia","tag-technology"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts\/659928","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/comments?post=659928"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts\/659928\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/media\/659929"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/media?parent=659928"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/categories?post=659928"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/tags?post=659928"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}