Reading List

-

 This page is my reading list of various articles on AI and Cloud and some random notes.

# Tools, Libraries, Frameworks

## Omnigent

[Omnigent AI Harness](https://omnigent.ai/)

Omnigent is being used by DataBricks. Its a harness layer, which allows to multiplex models, agents.

[Coding agent from scratch](https://github.com/Camrahd/claud_code)

## Graphs and Context:

(https://adam-slack.medium.com/designing-context-graphs-around-how-organisations-work-part-1-79067ebbb27a)

(https://adam-slack.medium.com/copying-data-into-context-graphs-part-2-27aaa5710a95)

(https://adam-slack.medium.com/context-graphs-knowledge-graphs-and-agent-memory-part-0-948adfacf74b)

Why does the first token from an LLM take longer than the rest?
Every autoregressive LLM response has two distinct phases:
𝗣𝗿𝗲𝗳𝗶𝗹𝗹
The model processes your entire prompt in one parallel pass, building the KV cache for every input token. This is compute-heavy — attention scales with the square of sequence length, so longer prompts mean more work before anything comes out.
𝗗𝗲𝗰𝗼𝗱𝗲
Once the first token is generated, the model produces every subsequent token one at a time, reusing the cached K/V from prefill instead of reprocessing the whole prompt. This is memory-bound rather than compute-bound, so each token after the first is much cheaper.
The gap between sending your prompt and seeing that first token is called 𝗧𝗶𝗺𝗲 𝗧𝗼 𝗙𝗶𝗿𝘀𝘁 𝗧𝗼𝗸𝗲𝗻 (𝗧𝗧𝗙𝗧) — and it exists in every decoder-only model: GPT, Llama, Claude, Qwen, Mistral, all of them. Longer prompts and larger contexts directly translate into longer TTFT, even if decode speed stays constant.
Here’s the part people often miss: 𝗧𝗧𝗙𝗧 doesn’t go away with 𝘀𝗽𝗲𝗰𝘂𝗹𝗮𝘁𝗶𝘃𝗲 𝗱𝗲𝗰𝗼𝗱𝗶𝗻𝗴 either. Speculative decoding speeds up the decode phase — a small draft model proposes several tokens ahead, and the main model verifies them in one batched pass instead of generating strictly one token at a time. That reduces the time to get from token 2 onward. But the very first token still has to come from a full prefill pass over the prompt on the main model. TTFT is a prefill-phase cost; speculative decoding is a decode-phase optimization. They solve different problems.
Practical takeaway for anyone building or serving LLM applications: if you care about responsiveness, look at TTFT and inter-token latency separately. Prompt caching / prefix caching attacks TTFT (by skipping redundant prefill on repeated context). Speculative decoding, batching, and KV cache optimizations attack decode throughput. Conflating the two leads to optimizing the wrong bottleneck


Memory Management in ADK

Production Ready AI Systems

Tricks for AI Assisted Coding


























Tools
Nano Bot (OpenClaw alternative)


Evals

Pending



New Agentic Fw



---

New




-------

Inferencing

Ideas

ML System Design:
🔹Youtube videos:
1. How to Master ML System Design
→ https://lnkd.in/d89ywdkp
→ Step by step guide (went through it - its good)

2. Stanford MLSys Seminars (playlist)
→ https://lnkd.in/gP3UDqwn
→ How Netflix/Uber actually scale

🔹Interview examples:
1. Spotify ML Question - Design a Recommendation System 
→ https://lnkd.in/dr87ADq8

2. Instagram ML Question - Design a Ranking Model 
→ https://lnkd.in/dqMxqgNc

🔹 Github Repo:
1. System Design Primer
→ https://lnkd.in/gmAg4nBb
→ Master the patterns that matter


Protocols:
• ACP – Standardizes cross-agent workflow communication
• AGP – Secure gateway between agents and enterprise systems
• Tool Abstraction Protocol – Makes tools structured and callable
• OAP – Open interoperability across frameworks
• RDF-Agent – Knowledge graph–driven semantic reasoning
• AgentOS – Runtime for long-lived enterprise agents
• TDF – Structured task planning and constraints modeling
• FCP – Safe function calling and structured execution




Embedding models:
static-retrieval-mrl-en-v1
all-MiniLM-L6-v2 model 



Skills Vs Tools
Every tool's schema, name, and description are added to the system prompt, consuming significant tokens upfront.
Skills are stored as Markdown files. They are only loaded into the prompt when needed. (Progressive disclosure)

Comments

Post a Comment

Popular posts from this blog

Pivotal Cloud Foundry (PCF) Integration with Elastic Cloud Storage (ECS)

-
Recently, I was involved in integrating Pivotal Cloud Foundry with Elastic Cloud Storage (ECS), an object storage solution from EMC. In this post, I'm going to document the hiccups we faced during this integration and how did we resolve this, so that it is easier for other folks who would like to carry out this integration. References: We followed the blog post, https://blog.codedellemc.com/2016/01/05/emc-ecs-with-cloud-foundry/ and the service broker code from git hub, https://github.com/spiegela/ecs-cf-service-broker .  1. application.yml file Configuration: First task would be to update the application.yml file in the broker code to have the correct configuration. Note the spring profiles created in yml file. The active spring profile is defined in the build.gradle file to be 'development'. So based on that, we need to update the correct section in the yml file. Under the broker section: a) Provide a valid ECS namespace name (The namespa...
Read more

Restful code example using Spring MVC

-
In my previous blog, I talked about Restful webservices design. In this blog, I will provide sample code for the same using Spring MVC framework. Spring MVC 3.x supports annotation based controllers, which are Restful in nature.  Let us create a sample Spring web application for performing CRUD on User entity. Creating project: Create a Maven project with a archetype 'maven-archetype-webapp'. pom.xml <!-- version property --> <properties> <spring.version>3.1.2.RELEASE</spring.version> </properties> <!-- servlet dependency for compilation --> <dependency> <groupId>javax.servlet</groupId> <artifactId>servlet-api</artifactId> <version>2.5</version> <type>jar</type> <scope>compile</scope> </dependency> <!-- Spring MVC dependencies --> <dependency> <groupId>org.springframework</groupId> <artifactId...
Read more

Spring Integration - Bulk processing Example

-
Image
In this post, I'm going to share my experience in using Spring Integration for a bulk processing task. This was the first time I was using Spring Integration and it did not disappoint me. It was pretty robust in terms of error handling and definitely scalable. Disclaimer: The actual task being performed has been modified into a fictitious e – commerce domain task, since I cannot reveal the original task which is customer specific and confidential. However, the nature of the task in terms of processing of clob and xml remain the same and the complexity of the original task has been retained. Also, the views posted on this post are strictly personal. Objective: The objective is to work upon customer information from an ecommerce application and process it and save it into database as clob data. The task is to perform data processing by reading xml from a file in the file system and then store it as clob into a table. Following is the high level of tasks required t...
Read more