APIs

Introduction Tool calling (also called function calling or plugins) is the capability that turns large language models from text predictors into general-purpose controllers for software. Instead of only generating natural language, an LLM can: Decide when to call a tool (e.g., “get_weather”, “run_sql_query”) Decide which tool to call Construct arguments for that tool Use the result of the tool to continue its reasoning or response This post is a deep dive into the anatomy of tool calling: the moving parts, how they interact, what can go wrong, and how to design reliable systems on top of them. ...

Batching API requests is a proven technique to improve throughput, reduce overhead, and tame the N+1 request problem across web and mobile apps. But batching is more than “combine a few calls into one.” To do it well you need to consider protocol details, error semantics, idempotency, observability, rate limiting, and more. This article explains how batching works, when to use it, and how to design and implement robust batch endpoints with real code examples. ...