Over the last year or so, I've been experimenting with AI dev tools in various ways. Starting from a fairly sceptical position, I wanted to answer a few questions about AI dev tools for myself:
- Is there any substance behind all the hype? Are they good for anything?
- Can they successfully generate code in a more niche language (specifically Elixir)? A popular belief is that LLMs are vastly more proficient with languages that provide a large training corpus, such as JavaScript and Python.
- What kind of tasks and projects are they suitable for?
Over the last few months, I've used these tools for a number of projects:
- ???
- Elixir site generator for korban.net. I started on this project before LLMs but then it stalled. Once I had Claude, I was able to knock out the remaining features quickly and put it into production, so to speak. This was also a success for me.
Many people point to the enabling aspect of AI dev tools and I think it's also been true for me in three ways:
- It's much easier to get past the blank page and start making progress, something that I often find difficult.
- I was able to make progress on these projects in small increments, while tired, late in the evening etc. It was substantially easier than a software project would normally be as I didn't have to spend much time loading context back into my brain and I didn't need to expend nearly as much cognitive effort when writing prompts compared to writing code.
- Some of these projects (eg. the migration from Storygraph to Obsidian) would have never made it to the top of my priority list and would have simply not happened if I had to budget for writing the code and figuring out the details (like OpenLibary API) myself. I just don't have enough motivation or time to make it happen.
Also:
- I often use LLMs as a substitute for search/research type tasks (eg. identifying useful libraries or, say, organisational paradigms for Obsidian notes)
- I've used LLMs to get past the (literal) blank page when writing documents at work. I typically do significant rewrites as the generated text is deficient in many ways, but getting some ideas and an outline is nonetheless quite useful.
- I've used Cursor, Zed, Claude, Copilot and Codex so far. I've seen little difference between Claude and Codex in terms of ability to generate useful results.
- I think the agent approach is the thing that really started making LLMs useful to me. I don't use inline code suggestions at all.