Byte Musings

 This page is my reading list of various articles on AI and Cloud and some random notes. # Tools, Libraries, Frameworks ##  Omnigent [Omnigent AI Harness](https://omnigent.ai/) Omnigent is being used by DataBricks. Its a harness layer, which allows to multiplex models, agents. [Coding agent from scratch](https://github.com/Camrahd/claud_code) Why does the first token from an LLM take longer than the rest? Every autoregressive LLM response has two distinct phases: 𝗣𝗿𝗲𝗳𝗶𝗹𝗹 The model processes your entire prompt in one parallel pass, building the KV cache for every input token. This is compute-heavy — attention scales with the square of sequence length, so longer prompts mean more work before anything comes out. 𝗗𝗲𝗰𝗼𝗱𝗲 Once the first token is generated, the model produces every subsequent token one at a time, reusing the cached K/V from prefill instead of reprocessing the whole prompt. This is memory-bound rather than compute-bound, so each token after the first is m...