"use strict"; const {Fragment: _Fragment, jsx: _jsx, jsxs: _jsxs} = arguments[0]; const {useMDXComponents: _provideComponents} = arguments[0]; function _createMdxContent(props) { const _components = { a: "a", code: "code", em: "em", h2: "h2", h3: "h3", h4: "h4", li: "li", ol: "ol", p: "p", pre: "pre", strong: "strong", ul: "ul", ..._provideComponents(), ...props.components }; return _jsxs(_Fragment, { children: [_jsx(_components.p, { children: "I first started using AI coding assistants in early 2021, with an invite code from a friend who worked on the original GitHub Copilot team. Back then the workflow was just single-line tab completion, but you could also guide code generation with comments and it'd try its best to implement what you want." }), "\n", _jsx(_components.p, { children: "Fast forward now to 2025, and there's a wide range of coding assistants that are packed with features, the models have gotten substantially more powerful, and the way I develop with them has changed too. They're an essential part of coding for me in various contexts." }), "\n", _jsx(_components.p, { children: "However, many developers remain skeptical of the utility of AI coding assistants. This is usually because they tried a vague task with a free AI model in the past and noticed incorrect code, hallucinated API calls, or some other issue. Others have incorporated these tools, as have their teams, but use of these tools hasn't resulted in more working software. In some cases, the tools have lowered overall productivity because teams had to hunt down bugs where the root cause involved a developer who blindly trusted AI-generated code." }), "\n", _jsx(_components.p, { children: "I've chosen to write this post specifically for those who are skeptical or have came away unimpressed. I can't promise that if you follow what I say that you'll fall in love with AI coding assistants. But I do believe that if you adopt some of the following tips, you'll come away substantially more impressed than you might be today." }), "\n", _jsx(_components.h2, { children: "But first, skill issue" }), "\n", _jsx(_components.p, { children: "AI-assistants are tools, not magic, and to use them effectively, you need to develop skills to be effective. If there's one single thing to take away from my post, it's that AI is a tool that requires developing skills to wield effectively. If you do not invest in these skills, you will be ineffective using AI for coding." }), "\n", _jsx(_components.h2, { children: "Use Claude and pay for it" }), "\n", _jsx(_components.p, { children: "Before you stress about getting skilled at using AI coding tools, first you need to actually get a good tool." }), "\n", _jsxs(_components.p, { children: ["Right now, the best AI model for coding is ", _jsx(_components.a, { href: "https://claude.ai/new", children: "Claude" }), ". There are ", _jsx(_components.a, { href: "https://www.anthropic.com/research/swe-bench-sonnet", children: "coding benchmarks" }), " that Claude scores well for, but you'll find in the AI community that everyone claims that these benchmarks don't come close to doing it justice. Somehow, in some way, Claude has this ", _jsx(_components.em, { children: "je ne sais quoi" }), " where it doesn't veer off course much as you use it, the code it writes for your codebase seems to fit the style of your code, and it just doesn't make up API calls that don't exist."] }), "\n", _jsx(_components.p, { children: "You can suffice with ChatGPT or Gemini or other models, but you'll need to be on your guard more because these tend to \"drift\" from the style and goals you set in the code that they suggest over time." }), "\n", _jsxs(_components.p, { children: ["Do not take this recommendation lightly. Claude is the only model that you can reliably ", _jsx(_components.em, { children: "not" }), " have to second-guess every time you use it. I have found that it can still make mistakes, but more often than not, these mistakes have come becuase I didn't provide enough (or accurate) context, such as forgetting to mention that I updated some other part of the codebase. And as I'll discuss later on, you can eliminate most of these problems as well with particular prompting techniques."] }), "\n", _jsxs(_components.p, { children: [_jsx(_components.strong, { children: "If you formed your opinion about AI coding assistants from using free ChatGPT a year ago, you need to radically update your priors" }), ". The gap between free AI tools and premium models like Claude has widened substantially, and the capabilities have advanced dramatically."] }), "\n", _jsx(_components.p, { children: "Maybe at some point, another model will de-throne Claude for coding. Claude is also not necessarily the best for non-coding workflows. But for now, it's the one to use." }), "\n", _jsx(_components.h3, { children: "Use Claude in whatever tool you like" }), "\n", _jsx(_components.p, { children: "Since Claude is not just the chatbot, but a model, it can be used to power any number of coding assistants -- Copilot, Cursor, WindSurf, and more. I personally don't use these tools much and usually just copy/paste from the Claude web interface, but little that I'll describe in this post prohibits the use of other tools." }), "\n", _jsx(_components.h2, { children: "The kind of code you write matters" }), "\n", _jsx(_components.p, { children: "AI coding assistants vary wildly in their effectiveness based on the kind of code you're writing. Generally speaking, I think about this in three ways." }), "\n", _jsx(_components.h3, { children: "Task commonality" }), "\n", _jsx(_components.p, { children: "The more common the kind of code you're trying to write, the more likely an AI model will do a good job. For example, if you're writing a NextJS app with Tailwind CSS, most modern models will not only be up to date on the exact patterns that work best, but they'll also work with older versions too. But if you're writing code in your custom backend with significant domain-specific constraints, you'll probably find that the AI model struggles to generate code that fits your needs." }), "\n", _jsxs(_components.p, { children: ["That said, this doesn't mean you can't be effective with AI and domain-specific constraints. ", _jsx(_components.strong, { children: "It just means you'll need to be more explicit about what you want" }), ", and you'll need to carefully specify what those constrains are up front. If don't know these contraints, or you can't explain them clearly to another person, you probably won't be able to get an AI model to work well with them either."] }), "\n", _jsx(_components.h3, { children: "Likelihood that similar code is widely available online" }), "\n", _jsx(_components.p, { children: "If the code you're writing for your domain has a lot of similar code available online, chances are an AI model will write good code for you without much work." }), "\n", _jsxs(_components.p, { children: ["Web development, relatively boring backend development, mobile development, library development, and \"", _jsx(_components.a, { href: "https://www.cncf.io/", children: "CNCF" }), "-adjacent\" cloud development (i.e., written in Go and deployed to kubernetes), are some domains that I've found AI models do well with. This is mostly due to the vast amount of this sort of code available that modern AI models pre-train on."] }), "\n", _jsx(_components.h3, { children: "Rapid feedback cycles" }), "\n", _jsx(_components.p, { children: "The easier it is to run and verify code, the more effective AI assistance becomes. This is why tasks with quick feedback loops (like frontend development or unit test writing) tend to work particularly well, but tasks with slower feedback loops (like non-kubernetes infrastructure code) can be more challenging." }), "\n", _jsx(_components.h2, { children: "Rethink the decision to use libraries" }), "\n", _jsx(_components.p, { children: "One unexpected lesson I learned is that because AI makes it very cheap to generate code, I think differently about if I want to use a library or not." }), "\n", _jsx(_components.p, { children: "I view libraries as falling into one of two categories:" }), "\n", _jsxs(_components.ol, { children: ["\n", _jsx(_components.li, { children: "Libraries that solve genuinely hard problems for me" }), "\n", _jsx(_components.li, { children: "Libraries that save me from writing a bunch of code that I don't want to write" }), "\n"] }), "\n", _jsx(_components.p, { children: "For the first category, I will obviously use a library. There are many libraries that are rigorously tested, have a large community, and are well-maintained. And in fact, AI models like Claude will often recommend using good libraries in the first place." }), "\n", _jsxs(_components.p, { children: ["Sometimes, if a model recommend a library for me, I'll interrogate the suggestion to see if it's ", _jsx(_components.em, { children: "really" }), " needed, or if it requires a lot of code to replicate."] }), "\n", _jsxs(_components.p, { children: ["Below is an example from a conversation where Claude recommended using ", _jsx(_components.code, { children: "react-router" }), ":"] }), "\n", _jsx(_components.pre, { children: _jsx(_components.code, { className: "language-text", children: "\nWhy do you think I should use `react-router`?\n\n\n\nReact Router is worth considering for your web app for several key reasons:\n\n1. Client-side routing - Instead of making full page requests to your server, React Router handles navigation smoothly on the client side, making your app feel more responsive and app-like rather than website-like.\n\n2. URL-based state management - It gives users the ability to bookmark specific views and use the browser's back/forward buttons naturally. This improves user experience and makes your app's state more predictable.\n\n3. Dynamic routing - You can easily handle dynamic route parameters like `/users/:id` or `/products/:category/:id`, which is essential for most modern web apps.\n\n4. Built-in features that solve common problems:\n - Nested routes for complex layouts\n - Protected routes for authentication\n - Route matching priorities \n - Loading and error states\n - Query parameter handling\n\nFor example, if you later decide to add user authentication, protecting certain routes becomes straightforward:\n\n\n \n \n }\n/>\n\nWould you like me to show you some specific examples of how these features could benefit your particular use case? Or would you prefer to explore alternative routing solutions?\n\n\n\nHow much code would I have to write if I didn't use `react-router`?\n\n\n\nLet me show you a basic custom routing implementation without React Router so you can compare:\n\n(...generated code)\n\nAs you can see, even for a basic router you need to:\n\n1. Create your own Link component that manages history updates\n2. Build a Router component that tracks URL changes\n3. Manage event listeners for browser navigation\n4. Handle route matching manually\n\nAnd this basic implementation is still missing many features that React Router provides out of the box:\n- No nested routes\n- No dynamic route parameters\n- No route ranking/matching priorities\n- No built-in handling for query parameters\n- No loading states\n- No error boundaries\n\nYou'd need to write significantly more code to implement these features yourself. Plus, React Router's code is battle-tested by thousands of developers and handles many edge cases you might not think of initially.\n\nWould you like me to show you what the code would look like to add any of those missing features to our custom implementation?\n\n\n\nlol okay\n\n" }) }), "\n", _jsxs(_components.p, { children: [_jsx(_components.strong, { children: "In general, it's a good idea to \"question\" an AI model's suggestions here." }), " The first suggestion you get may not always be better. But importantly, asking an AI model to explain its answer in a critical light often leads to a better suggestion."] }), "\n", _jsx(_components.p, { children: "All that said, AI has made me completely re-evaluate the second category -- where a library is brought in because it saves some code and adds convenience. Because code is so cheap to generate now, I will usually just generate the code and tests for it, package it up somewhere, and have one less hellish dependency to manage over time." }), "\n", _jsx(_components.p, { children: _jsx(_components.strong, { children: "Dependency management is much harder than re-generating code." }) }), "\n", _jsx(_components.h2, { children: "A different development loop" }), "\n", _jsx(_components.p, { children: "With AI assistance, my development loop is more dynamic and iterative than the before-times, and requires several techniques that I've learned over time to be effective." }), "\n", _jsx(_components.h3, { children: "Build durable context" }), "\n", _jsxs(_components.p, { children: [_jsx(_components.a, { href: "https://arxiv.org/abs/2005.11401", children: "RAG" }), " is something most people building with AI have learned is an essential pattern, and the quality of an AI integration is a direct function of the ", _jsx(_components.a, { href: "./llms-three-properties", children: "relevancy, magnitude, and quality" }), " of the contextual data you pass to an AI model."] }), "\n", _jsx(_components.p, { children: "This is especially true for using AI for real-world coding instead of tiny little side projects." }), "\n", _jsx(_components.h4, { children: "Use projects/rules or paste context into a chat session" }), "\n", _jsxs(_components.p, { children: ["I usually start by creating an instruction/rules file or \"project\" in the AI tool I'm using. In Claude or ChatGPT, this is literally called \"projects\", and these are a way to provide a bunch of files that make the model produce significantly better code. ", _jsx(_components.a, { href: "https://docs.cursor.com/context/rules-for-ai", children: "Cursor" }), " and ", _jsx(_components.a, { href: "https://copilot-instructions.md/", children: "Copilot" }), " support custom instructions in various ways too."] }), "\n", _jsx(_components.p, { children: "Note that depending on your tool, it's not just written text files that are supported, but also digrams. I've found that diagrams are particularly useful for showing things like architecture or general flows of a working app to contextualize how the code makes something work. You can also provide custom documents that reference other documents, such as a file that explains the names of several screenhots and how they relate to the codebase. Things like Claude Projects will pick this up." }), "\n", _jsxs(_components.p, { children: ["If you're not using a tool that supports this sort of thing, you can even get by with simply pasting in a single file -- preferably following the ", _jsx(_components.a, { href: "https://llmstxt.org/", children: "llms.txt standard" }), " -- and your single chat session will produce ", _jsx(_components.em, { children: "shockingly" }), " better code."] }), "\n", _jsxs(_components.p, { children: ["Just writing some custom rules or instructions or pasting in an llms.txt file will make a huge difference in the quality of the code you get back. However, you can take this a lot further by cross-referencing important context with the structure (and substructure) of your codebase. Athough tools like Cursor and Copilot do support ", _jsx(_components.a, { href: "https://docs.cursor.com/context/codebase-indexing", children: "codebase indexing" }), ", I've found that this only helps with finding likely-relevant code for a task and increasing the likelihood that the generated code can compile -- it doesn't embed actual knowledge of the codebase. To make sure it doesn't \"go off the rails\", you need to provide that context yourself."] }), "\n", _jsx(_components.h4, { children: "Sharing existing code structure and important context" }), "\n", _jsxs(_components.p, { children: ["I typically start by dumping the output of ", _jsx(_components.a, { href: "https://github.com/dduan/tre", children: _jsx(_components.code, { children: "tre" }) }), " to a text file and including that with some basic instructions about code areas."] }), "\n", _jsxs(_components.p, { children: ["For larger projects, I spend some time writing up a proper ", _jsx(_components.a, { href: "https://llmstxt.org/", children: "llms.txt" }), " markdown file that describes the high-level goals of the system, key areas of the codebase with important knowledge, and then the full file structure. This is a bit of work, but it's a one-time cost that pays off in the long run."] }), "\n", _jsxs(_components.p, { children: ["For example, here's one that could be used for a production system, ", _jsx(_components.a, { href: "https://github.com/honeycombio/refinery", children: "Refinery" }), ":"] }), "\n", _jsx(_components.pre, { children: _jsx(_components.code, { className: "language-md", children: "# Refinery\n\n## Refinery overview\n\nRefinery is a tail-based distributed tracing sampler that:\n\n1. Collects and distributes trace data across the cluster it's run on\n a. It defaults to using trace locality, meaning that all data in a trace is forwarded to be on the same node\n b. If trace locality is turned off, only metadata is forwarded across a cluster\n2. Once a trace is considered fully recieved (its root span is collected or a timeout is reached), it processes the trace by looking at the configured sampling rules, and decides to either sample (keep) or drop the trace.\n3. If a trace is sampled, it is enqueued for transmission.\n a. Sampled trace IDs are also saved to a cache, which allows any late-arriving spans to also be transmitted instead of dropped.\n4. Operational metrics, trace data, and logs are created throughout Refinery's operation and transmitted as OpenTelemetry data by default.\n a. In particular, Refinery uses a notion of \"stress level\" to determine if and when it should start backing off on processing and applying rudimentary (but compute-inexpensive) sampling rules. This stress level is often monitored to determine if a Refinery cluster has adequate resources to run.\n\nIn particular, it specializes in dynamic sampling, which is a technique that adjusts the sampling rate of traces based on the uniqueness of the data and the desired throughput or sample rates. By combining dynamic sampling with rules-based sampling, Refinery allows a high degree of flexibility and sophistication in how traces are sampled.\n\n## Key refinery operational characteristics\n\nRefinery is also itself a sophisticated distributed system. It gossips its internal state and information about data in its nodes across the rest of the cluster. It also supports distributing trace data across a cluster in different ways using Redis.\n\nRefinery also defines a notion of a \"stress level\", which is a measure of how much load a Refinery node is under. This stress level is used to determine if and when Refinery should start backing off on processing and applying rudimentary deterministic sampling, which ensures that data is still sampled rather than always dropped, but comes at the cost of not being able to apply more sophisticated sampling rules.\n\n## Key areas of the codebase\n\nRefinery has many files and folders, but these in particular are most helpful to understand the system and know where to make tweaks.\n\n1. collect/ and collect/collect.go, which contains much of the logic for collecting and distributing trace data\n2. types/ and types/event.go, which contains the data structures for trace data (confusingly, the term \"event\" is used for any unit of a trace, not just log events)\n3. pubsub/ and pubsub/pubsub.go, which contains the logic for gossipping information about refinery nodes and the data resident within them across the cluster\n4. sample/, which contains a variety of samplers. In particular, the dynamic samplers (such as ema_throughput.go) are heavily used in production because they automatically adjust sampling rates based on uniqueness of data and desired throughput or sample rates.\n5. config/ and config/config.go, which contains the configuration loading and validation logic for Refinery. Refinery uses a TOML configuration file, but also supports YAML and JSON for configuration.\n\n## Full file structure\n\n├── .circleci\n│ └── config.yml\n├── .github\n│ ├── CODEOWNERS\n│ ├── ISSUE_TEMPLATE\n│ │ ├── bug_report.md\n│ │ ├── feature_request.md\n│ │ ├── question-discussion.md\n│ │ └── security-vulnerability-report.md\n│ ├── PULL_REQUEST_TEMPLATE.md\n│ ├── dependabot.yml\n│ ├── release.yml\n│ └── workflows\n│ ├── add-to-project-v2.yml\n│ ├── apply-labels.yml\n│ ├── release.yml\n│ ├── stale.yml\n│ └── validate-pr-title.yml\n├── .gitignore\n├── .tool-versions\n├── CHANGELOG.md\n├── CODE_OF_CONDUCT.md\n├── CONTRIBUTING.md\n├── LICENSE\n├── Makefile\n├── NOTICE\n├── OSSMETADATA\n├── README.md\n├── RELEASE_NOTES.md\n├── RELEASING.md\n├── SECURITY.md\n├── SUPPORT.md\n├── app\n│ ├── app.go\n│ └── app_test.go\n├── build-docker.sh\n├── build-pkg.sh\n├── cmd\n│ ├── refinery\n│ │ └── main.go\n│ └── test_redimem\n│ └── main.go\n├── collect\n│ ├── cache\n│ │ ├── cache.go\n│ │ ├── cache_test.go\n│ │ ├── cuckoo.go\n│ │ ├── cuckooSentCache.go\n│ │ ├── cuckoo_test.go\n│ │ ├── kept_reason_cache_test.go\n│ │ ├── kept_reasons_cache.go\n│ │ └── traceSentCache.go\n│ ├── collect.go\n│ ├── collect_test.go\n│ ├── mockCollector.go\n│ ├── stressRelief.go\n│ ├── stress_relief_test.go\n│ ├── trace_decision.go\n│ ├── trace_decision_test.go\n│ ├── trace_redistributer.go\n│ └── trace_redistributer_test.go\n├── config.md\n├── config.toml\n├── config.yaml\n├── config\n│ ├── cmdenv.go\n│ ├── cmdenv_test.go\n│ ├── config.go\n│ ├── configLoadHelpers.go\n│ ├── configLoadHelpers_test.go\n│ ├── config_test.go\n│ ├── config_test_reload_error_test.go\n│ ├── file_config.go\n│ ├── file_config_test.go\n│ ├── level.go\n│ ├── memorysize.go\n│ ├── memorysize_test.go\n│ ├── metadata.go\n│ ├── metadata\n│ │ ├── configMeta.yaml\n│ │ └── rulesMeta.yaml\n│ ├── metadataFS.go\n│ ├── mock.go\n│ ├── sampler_config.go\n│ ├── sampler_config_test.go\n│ ├── validate.go\n│ └── validate_test.go\n├── config_complete.1.x.toml\n├── config_complete.yaml\n├── generics\n│ ├── fanout.go\n│ ├── fanout_test.go\n│ ├── mapttl.go\n│ ├── mapttl_test.go\n│ ├── set.go\n│ ├── set_test.go\n│ ├── setttl.go\n│ └── setttl_test.go\n├── go.mod\n├── go.sum\n├── internal\n│ ├── configwatcher\n│ │ └── watcher.go\n│ ├── health\n│ │ ├── health.go\n│ │ └── health_test.go\n│ ├── otelutil\n│ │ └── otel_tracing.go\n│ ├── peer\n│ │ ├── file.go\n│ │ ├── file_test.go\n│ │ ├── mock.go\n│ │ ├── peers.go\n│ │ ├── peers_test.go\n│ │ ├── pubsub_redis.go\n│ │ └── pubsub_test.go\n│ └── redimem\n│ └── redimem.go\n├── logger\n│ ├── honeycomb.go\n│ ├── logger.go\n│ ├── logger_test.go\n│ ├── logrus.go\n│ ├── mock.go\n│ └── null.go\n├── metrics.md\n├── metrics\n│ ├── legacy.go\n│ ├── legacy_test.go\n│ ├── metrics.go\n│ ├── metrics_test.go\n│ ├── metricsnamer.go\n│ ├── mock.go\n│ ├── multi_metrics.go\n│ ├── multi_metrics_test.go\n│ ├── nullmetrics.go\n│ ├── otel_metrics.go\n│ ├── prometheus.go\n│ └── prometheus_test.go\n├── preinstall\n├── pubsub\n│ ├── pubsub.go\n│ ├── pubsub_goredis.go\n│ ├── pubsub_local.go\n│ └── pubsub_test.go\n├── refinery.service\n├── refinery.upstart\n├── refinery_config.md\n├── refinery_rules.md\n├── route\n│ ├── errors.go\n│ ├── errors_test.go\n│ ├── middleware.go\n│ ├── middleware_test.go\n│ ├── mock.go\n│ ├── otlp_logs.go\n│ ├── otlp_logs_test.go\n│ ├── otlp_trace.go\n│ ├── otlp_trace_test.go\n│ ├── proxy.go\n│ ├── route.go\n│ └── route_test.go\n├── rules.json\n├── rules.md\n├── rules.toml\n├── rules.yaml\n├── rules_complete.1.x.toml\n├── rules_complete.yaml\n├── rules_conditions.md\n├── sample\n│ ├── deterministic.go\n│ ├── deterministic_test.go\n│ ├── dynamic.go\n│ ├── dynamic_ema.go\n│ ├── dynamic_ema_test.go\n│ ├── dynamic_test.go\n│ ├── ema_throughput.go\n│ ├── ema_throughput_test.go\n│ ├── rules.go\n│ ├── rules_test.go\n│ ├── sample.go\n│ ├── sample_test.go\n│ ├── totalthroughput.go\n│ ├── totalthroughput_test.go\n│ ├── trace_key.go\n│ ├── trace_key_test.go\n│ ├── windowed_throughput.go\n│ └── windowed_throughput_test.go\n├── service\n│ └── debug\n│ └── debug_service.go\n├── sharder\n│ ├── deterministic.go\n│ ├── deterministic_test.go\n│ ├── mock.go\n│ ├── sharder.go\n│ ├── sharder_test.go\n│ └── single.go\n├── smoke-test\n│ ├── .gitignore\n│ ├── README.md\n│ ├── config-tls.yaml\n│ ├── config.yaml\n│ ├── docker-compose-tls.yaml\n│ ├── docker-compose.yaml\n│ └── rules.yaml\n├── test\n│ ├── EMAThroughput_rules.yaml\n│ ├── TotalThroughput_rules.yaml\n│ ├── WindowedThroughput_rules.yaml\n│ ├── config.yaml\n│ └── rules.yaml\n├── tools\n│ ├── convert\n│ │ ├── Makefile\n│ │ ├── README.md\n│ │ ├── configDataNames.txt\n│ │ ├── helpers.go\n│ │ ├── helpers_test.go\n│ │ ├── main.go\n│ │ ├── metrics.go\n│ │ ├── metricsMeta.yaml\n│ │ ├── minimal_config.yaml\n│ │ ├── ruleconvert.go\n│ │ ├── ruleconvert_test.go\n│ │ └── templates\n│ │ ├── cfg_docrepo.tmpl\n│ │ ├── cfg_docsite.tmpl\n│ │ ├── configV2.tmpl\n│ │ ├── genfield.tmpl\n│ │ ├── genfile.tmpl\n│ │ ├── gengroup.tmpl\n│ │ ├── genremoved.tmpl\n│ │ ├── metrics.tmpl\n│ │ ├── names.tmpl\n│ │ ├── rules_docrepo.tmpl\n│ │ ├── rules_docsite.tmpl\n│ │ └── sample.tmpl\n│ └── loadtest\n│ └── .gitignore\n├── transmit\n│ ├── mock.go\n│ ├── transmit.go\n│ └── transmit_test.go\n└── types\n ├── event.go\n └── event_test.go\n" }) }), "\n", _jsx(_components.h3, { children: "Get the LLM to update its own context" }), "\n", _jsx(_components.p, { children: "As you iterate, you'll find that the LLM environment needs more pushing to \"get things right\" with respect to the context you've provided. Usually it will make an assumption about a module or file that isn't correct. The solution here is very simple:" }), "\n", _jsxs(_components.p, { children: [_jsx(_components.strong, { children: "Program your development environment" }), "."] }), "\n", _jsx(_components.p, { children: "Think of the LLM has a wobbly and weird kind of computer that's bad (or at least inefficient) at tasks like adding numbers together, but extremely good at being directionally accurate about things. You can guide its force vector towards the particular outcome you want." }), "\n", _jsx(_components.p, { children: "Given this, when you want it to have updated context, you can simply ask it to update its own context once you know it got something right. For example:" }), "\n", _jsxs(_components.ul, { children: ["\n", _jsx(_components.li, { children: "If you generated a new nextjs component that does an important thing and it works correctly, ask the LLM to update its context with the new file and a brief description of the purpose it serves." }), "\n", _jsx(_components.li, { children: "If you added a new kind of protocol support for a library, ask the LLM to update internal documentation about protocol support." }), "\n", _jsx(_components.li, { children: "If you shuffle data off to a new AWS service, ask the LLM to update its description of the data flow and AWS services used, and for which purpose." }), "\n"] }), "\n", _jsx(_components.p, { children: "You will still need to review things, but as you iterate and continually have better and more up-to-date context, you'll find that you don't need to correct it that often anymore." }), "\n", _jsx(_components.h2, { children: "Ask for small code changes, not big ones" }), "\n", _jsx(_components.p, { children: "The advice here is no different than the general advice you find online about software engineering regarding small diffs, small deployments, etc." }), "\n", _jsx(_components.p, { children: "AI models are highly complex systems that increasingly get things wrong when you ask them to do too much at once. The solution is to simply have them do less. Some examples:" }), "\n", _jsxs(_components.ul, { children: ["\n", _jsx(_components.li, { children: "Don't generate an entire website at once. Generate a single component, then another, then another. Run your app and run tests each time." }), "\n", _jsx(_components.li, { children: "Don't generate an entire API at once. Generate a single endpoint, work on the conventions that fit best, adjust your data model incrementally, etc." }), "\n", _jsx(_components.li, { children: "Don't generate an internal library of utility functions at once." }), "\n", _jsx(_components.li, { children: "Don't generate an entire new feature at once. Start with a task description, then iterate on a spec, then generate some code for the spec, then generate tests, run the tests, generate more code for the spec, etc." }), "\n"] }), "\n", _jsx(_components.h3, { children: "Small changes with agents" }), "\n", _jsxs(_components.p, { children: ["These guidelines are not hard and fast, but also hold true with agents in my experience. While an agent can do a better job of defining an internal library of utility functions given a desired goal and constraints (or just a spec), ", _jsx(_components.a, { href: "https://bsky.app/profile/phillipcarter.dev/post/3lhosyazt7s26", children: "I've found that agents simply don't work well with existing codebases and nontrivial tasks" }), ". Right now, they seem especially tuned for generating brand-new codebases from sratch, and simple ones at that."] }), "\n", _jsx(_components.p, { children: "For agents, my inuition so far is that you really need to establish firm guardrails around their scope. Think of an agent as helping implement a small ticket end-to-end, with you dictacting how it should clarify requirements, test code, and perform the iterative loop. I've found that if I don't do this, the agent will wander off into the wilderness and eventually just generate nonsensical, often uncompilable code." }), "\n", _jsx(_components.p, { children: _jsx(_components.strong, { children: "You still have to think. Sorry!" }) }), "\n", _jsx(_components.h2, { children: "Where it's going from here" }), "\n", _jsx(_components.p, { children: "I'm genuinely looking forward to revisiting this blog post in a year now that LLMs have gotten faster and cheaper, and that agents are starting to pop up. They're all still really bad at doing what they're advertised for, but they can be highly useful with some work." }), "\n", _jsx(_components.p, { children: "The future I'm looking for is one where programming is lifted up to another level of abstraction. Instead of writing a ton of code, we focus a lot more on:" }), "\n", _jsxs(_components.ul, { children: ["\n", _jsx(_components.li, { children: "What, exactly, we want to build" }), "\n", _jsx(_components.li, { children: "How we define an experiment and measurement criteria for what we build" }), "\n", _jsx(_components.li, { children: "Exploring many different approaches to solving a problem all at once" }), "\n", _jsx(_components.li, { children: "Automatically embedding best practices into code like robust tests, good observability and internal documentation, etc." }), "\n", _jsxs(_components.li, { children: ["Finding out what we ", _jsx(_components.em, { children: "don't" }), " know about a problem and how to solve it, and using these tools to facilitate learning about systems and the way they get used"] }), "\n"] }), "\n", _jsxs(_components.p, { children: ["I think we're still a long way off from this reality. Even if we assume the current technical hurdles are solved -- which is absolutely, positively not a given -- there are other systematic hurdles involving human beings and the real world that need addressing. ", _jsxs(_components.strong, { children: ["To me, all signs point towards software engineering changing radically as a profession to be much more oriented around the ", _jsx(_components.em, { children: "what" }), " and ", _jsx(_components.em, { children: "why" }), " of software, and much less around the ", _jsx(_components.em, { children: "how" })] }), ". This will cause disruption at a massive scale in the long run. But in the short run, it's just a lot of fun to play with these tools and see what they can do."] })] }); } function MDXContent(props = {}) { const {wrapper: MDXLayout} = { ..._provideComponents(), ...props.components }; return MDXLayout ? _jsx(MDXLayout, { ...props, children: _jsx(_createMdxContent, { ...props }) }) : _createMdxContent(props); } return { default: MDXContent };

How I code with LLMs these days