How to Build LLM Applications When the Scaffolding Collapses

Introduction

The era of heavy frameworks for LLM applications—indexing layers, query engines, retrieval pipelines, and finely tuned agent loops—is ending. As models grow more capable, they can reason over massive datasets, self-correct, and plan ahead without manual orchestration. According to Jerry Liu, CEO of LlamaIndex, this isn't a crisis; it's an opportunity. The collapse of the scaffolding layer means developers can focus on what truly matters: context. This guide provides a step-by-step approach to building LLM applications that thrive in a post-scaffolding world, leveraging model intelligence and data extraction while minimizing code.

What You Need

• Access to powerful LLMs: GPT-4, Claude, or equivalent models with strong reasoning and tool-use capabilities.
• Context-rich data sources: Files (PDFs, images, spreadsheets), APIs, or databases containing domain-specific information.
• Basic familiarity with agent frameworks: Understanding of Model Context Protocol (MCP), Claude Agent Skills, or similar managed agent patterns.
• A coding assistant: Tools like Claude Code, GitHub Copilot, or other AI coding agents that can generate and refactor code from natural language prompts.
• Document parsing tools: OCR or intelligent document processing (IDP) solutions (e.g., LlamaIndex’s agentic OCR) to extract structured data from unstructured files.

Step-by-Step Guide

Step 1: Recognize the Shifting Landscape

Before you build, understand that the old paradigm—where every workflow required custom indexing, retrieval pipelines, and deterministic orchestration—is fading. Today’s LLMs can reason over vast amounts of unstructured data, often surpassing human accuracy. They can handle multi-step planning, self-correction, and tool discovery without explicit integration code. Accept that your application will rely more on model intelligence than on hand-coded scaffolding. This shift reduces the need for heavy frameworks like LlamaIndex’s early versions; instead, you’ll focus on providing rich context and letting the model do the work.

Step 2: Prioritize Context Over Orchestration

As Jerry Liu puts it, “Context is becoming the moat.” In a world where any LLM can be swapped in, the differentiator is the quality and accuracy of the data you feed it. Invest heavily in data extraction and parsing—especially from file formats like PDFs, images, and spreadsheets that often lock away valuable information. Use agentic document processing with OCR (optical character recognition) to convert these files into machine-readable text. For instance, LlamaIndex’s parsing tools can extract tables, figures, and metadata with high fidelity. The goal: give the model exactly the context it needs, cheaply and accurately.

Step 3: Adopt Managed Agent Patterns

Instead of building custom orchestration loops for every agent, embrace a managed agent diagram. This consists of a harness layer combined with tools, MCP connectors, and skills plug-ins. For example, using Claude Agent Skills, you can teach your agent how to interact with external APIs or file systems without writing integration code for each one. The Model Context Protocol standardizes how agents discover and use tools. By adopting these patterns, you reduce the amount of brittle, hand-crafted code and make your system more adaptable to new models and tools.

Step 4: Let AI Write the Code

Stop writing code manually for routine tasks like data retrieval or API calls. Today, an astonishing 95% of LlamaIndex’s own code is generated by AI. Engineers type natural language instructions, and the coding agent produces the rest. For your projects, use tools like Claude Code or GitHub Copilot to generate scaffolding, handle integrations, and even debug. Simply describe what you want—for example, “write a function to fetch stock prices from Yahoo Finance and parse them into a JSON list”—and let the agent handle the specifics. This collapses the layer between programming and natural language, making development much faster.

Step 5: Build with Simple Primitives

Don’t overcomplicate your architecture. Instead of building complex retrieval-augmented generation (RAG) pipelines from scratch, point your agent directly at your data sources. For instance, use Claude Code to scan a directory of PDFs, extract relevant text via OCR, and then query the LLM over that content. Liu notes that “this type of stuff was either extremely inefficient or would break the agent three years ago.” Now, models handle it naturally. Keep your stack minimal: a data source, a parsing layer, and a model with tool-use ability. Avoid unnecessary abstraction.

Step 6: Ensure Modularity and Swap-Ability

While your stack is simple, it must remain modular. The scaffolding collapse doesn’t mean lock-in to one vendor. Build your application so that you can swap out the underlying LLM (e.g., from OpenAI Codex to Claude) or change your parsing tool without rewriting the entire system. Use standard interfaces like MCP for tool connections and keep your context pipeline independent of the model. This future-proofs your app as models evolve—because, as Liu says, “the thing that they all need is context.”

Tips for Success

• Test context quality relentlessly: The best model fails if fed noisy or incomplete data. Validate your parsing pipeline with real-world files and measure extraction accuracy.
• Use few-shot prompts to guide agents: Even with managed patterns, a small set of examples can dramatically improve how an agent discovers and uses tools.
• Embrace natural language for development: The new programming language is English. Train your team to articulate tasks clearly—it’s a superpower.
• Monitor and log agent behavior: Since models now handle reasoning and planning, keep logs of agent decisions to debug and improve context provision.
• Stay updated on model capabilities: The collapse of scaffolding is driven by rapid model improvements. Regularly evaluate whether your hand-coded components can be replaced by model reasoning.

By following these steps, you can build LLM applications that are lean, context-driven, and ready for the future—where the scaffolding is gone, but the value is higher than ever.