Right now, AI applications should be focused on enhancement instead of replacement.

It’s as if, we’ve learned nothing from the decade long discourse on autonomous vehicles which are “right around the corner”. And while this white whale has been left to a few outliers, the rest of the industry is already progressing towards full autonomy without trying to jump the shark.

Today, level 2 vehicle autonomy is already widespread, and some constructors are already starting to roll out level 3 under certain conditions. But we are far from the fabled level 5, promised under unrealistic time frames by some.

And with the recent announcements, software engineering appears to be in a similar bind. Which prompted me to take the time to try and apply this 6 level framework, validated by a century old industry, to my favorite field.

The 6 levels of autonomous software development

Level 0: Coding in a text editor. The environment doesn't provide any assistance.

Level 1: Coding in an IDE with auto-complete and linting enabled. The environment provides deterministic assistance and guardrails in well-defined situations.

Level 2: Coding with Github's Copilot or Supermaven enabled. Your IDE is now able to help you code on well known tasks (like highways for cars) all while taking into account a small context window.

Level 3: Automating bigger tasks across multiple files. Something that you can start to achieve by mixing multiple infrastructure layers. I've already had success on some of these tasks by leveraging the Open AI Assistant API which provides chain-of-thought capabilities on top of custom tool calling.

Level 4: The software environment is fully autonomous, but its user can intervene at any time and correct the course of action. The "developer" (whatever the role will actually be called at this stage) is also able to revert to any previous level of autonomy whenever he desires.

Level 5: Set a destination through prompt and let the AI do the driving.

This is what projects like AutoGPT or Devin have been trying to do.

Unfortunately, aside from impressive looking demos, their result mostly feel underwhelming under an expert eye. That's because these products haven't managed the previous levels and are building on top of tools which aren't even able to get them there.

Enabling Level 5 requires a high level of accuracy in the final result under any situation.

What does level 4 look like ?

As you may have noticed, I haven't mentioned any level 4 projects but multiple level 5 ones. That's simply because I don't know of any, and for a good reason.

Level 4 is incredibly difficult to both conceptualize and make. Imagining an autonomous agent is easy, but creating the means to take control at any time and change the configuration of every parameter of autonomy in real time is much harder. Much harder but necessary if we want to make the most out of those new systems.

And that’s what I call AI steering. And that’s also why those “fully autonomous” solutions are lazy, only chasing easy money from gullible investors and CEOs who drool at the idea of a cheap and silent workforce. Good thing I’m not the only one who wishes to preserve the fun in engineering.

Despite this, let’s end on a positive note, because people from adjacent domains are definitely smarter and making breakthrough in their integration of AI:

• MartinNebelong's work on preserving artistic intent while levering image generation is impressive, and I strongly encourage you to follow his work.

• I was also wowed by the following demo from game developers. I love that the controller input perfectly illustrate my vision of Level 4 "AI steering".

And my goal is to share more of these novel practical usages as they come by, instead of focusing on the latest hype from a demo. So if you’ve heard of such interesting projects, make sure to reach out 🙏

“In 2024, there will be no more UX designers.

Where will the role of designers be in 2024? With the massive arrival of generative AI, the proliferation of design systems, open-access components, and nocode solutions, where will the role of designers lie?

Doing only Figma can quickly play tricks on you, because the ability of new AI-based website creation solutions will gradually make this role useless.

It will be easy for anyone to type in a prompt, and then set up a few styling elements, to generate an interface.

So what will a UI designer be used for?

Doing only research will block you more and more from accessing many companies that prefer to pay an interface designer who knows how to do research than a researcher who doesn't know how to do interface.

But more than that, will research still be necessary when we know that it is now possible to create synthetic testers to evaluate an interface?

It is therefore likely that UX designers will see their job change profoundly in 2024 and that they will have to add new strings to their bow if they want to continue to be employable:

1️⃣ Know how to properly evaluate and measure the value of the UX and the ROI it brings to a company (and, I admit that this is not easy to do, but yet more and more essential)

2️⃣ Know how to set up the right tools within their organization to produce interfaces faster and by more non-UX collaborators.

3️⃣ Know how to better exploit the possibilities of generative AI to accelerate many research and production processes (which is related to my point 2)

Am I exaggerating? Not so much. Whether at home, in France, or in the US, the world of UX is going through a serious crisis. Even its role is not to be denied, it must know how to show (unlike the coding professions) again its true value to a company.

Making mock-ups, yes, but that's not enough.

Doing research, yes, but it doesn't pay off fast enough! “

LinkedIn post by Olivier Sauvage published 5th of December 2023

My take on a misled one

So, is 2024 the end of designers? To make it short, my answer is a resounding “No”. And in opposition to this post’s allegations, I believe that the most at-risk job in tech is being a developer.

The duality of designers

When we mention designers in tech, we’re talking about both UI and UX designers.

The UI is the visual identity of the product, in addition to its objective of ensuring the accessibility and readability of a product, it is often a key element of a brand whose codes are an integral part of the company's image. When it's done well, you can recognize the product and the associated company at a glance. It's no coincidence that Google has spent the last few years harmonizing its consumer products’ design system. The UI is not just a functional factor, but also plays an important role in a company's marketing. Sure, you can generate UI with ChatGPT, but:

• This UI is not necessarily in line with the brand image

• The result is either flashy but non-functional, or bland and uninspired. When I generate interfaces with AI, the model uses what it has been trained on. Which for the most part, is the examples and themes proposed by the different UI libraries. And none of that will help your product stand out.

UX on the other side has a whole different scope. It is about offering the best possible user experience while integrating the company's objectives. A good UX Designer is not just someone "tinkering" with interfaces but a person capable of going out in the field, understanding the uses, needs, and problems of customers, and coming out with innovative solutions. Moreover, advanced designers regularly call on psychology during the analysis of user behavior and when coming up with solutions.

However, one of the limiting factors of AI today is autonomous context intake. For ChatGPT to come up with a solution, it must be provided with all the required information. In the context of UX research, it is a field with far too scattered, and sometimes difficult to qualify information (how do you define a user's emotion?) to be mastered by an AI. And we won’t get into the quality of the analysis provided in its responses.

The designer’s ROI

"Even its role is not to be denied, it must know how to show (unlike the coding professions) again its true value to a company." - Olivier Sauvage (translated from French)

As a matter of fact, it’s quite the opposite. Unlike designers, development teams are the ones who struggle the most to justify their value in a company. Every company knows that they need developers but none are capable of accurately determining their ROI. It's not for nothing that MacKinsey has positioned itself in the market for estimating development productivity with an article that I won't quote as it outraged me with its absurdities. If you want to understand the complexity of the task, I invite you instead to read the response by two tech veterans.

However, a designer can have a direct product impact. There will always be more UX Designers than devs who can claim to have increased a sales channel's conversion rate by 20%. And there, the ROI is quite easily quantifiable.

Design is changing. For the better

Rather than diminishing the value of the designer, AI is going to increase it.

As of writing this, AI models already have an almost expert level in writing functional code on unit tasks. If you want to understand the extent of it, look at this video where functional interfaces are being created without a single line of code. For the moment, the limiting factor remains their ability to reason, break down, and combine tasks. But that is already changing fast. Recently, I used the latest OpenAI model with the Assistant API to develop a tool capable of adding its own features through multi-step instructions; all without any handwritten code.

In software engineering, we call this a level of abstraction. It is usually a new development language whose use is simplified and which calls on more complex underlying languages for execution. But this time, the new language is simply the natural language. And since (almost) everyone speaks, anyone can become a "developer".

So in addition to their irreplaceable job, designers will have tools to:

• Do ideation more quickly by discussing with an AI

• Be more efficient in user needs collection by letting an AI do the processing and aggregation of data. Surely the most boring part with little added value

• Make their ideas functional without having coding knowledge and without a developer to help them

Time for change

So yes, there will be a paradigm shift. I remain convinced that today's design systems will have to be reinvented for use centered around AI. And, as in any industrial revolution process, there will be different reactions, from promoting these new tools to complete rejection.

However, the designers who will best apprehend these tools and put them at the service of their craft will see a significant increase in their value in the job market.

Personally, I have only one way to deal with such feelings. I take a step back, study the road taken, pinpoint the current context, and chart the most likely trajectory. So that’s what I’ll be doing in this post.

What’s up, I’m ChatGPT

In November 2023, OpenAI released ChatGPT, its first B2C product based on GPT-3.5. By fine-tuning GPT-3 to work in a chat context, and releasing it to the public in a user-friendly platform, they’ve finally sparked interest outside of academia.

Although it wasn’t the first Chat Model, it certainly was the largest to ever be trained. But the real game changer was about to come with the rollout of GPT-4. By significantly improving the results of ChatGPT, this update sent ripples through the tech world. Within just a couple of months, the spotlight finally shifted away from the toxic and saturated crypto narrative to the potential of AI in everyday use.1

Meanwhile, as users delved into ChatGPT’s capabilities, the initial excitement quickly faded, revealing the limitations. Finite memory, outdated data, lack of context, hallucinations, the cumbersome need for manual input, and the generic default “persona” made it highly impractical to handle complex scenarios.

Because, of course, many eyed AI as a workforce substitute. But as most work often includes executing multiple tasks, thought processes, and planning; expecting a single Chat LLM to do it all with Q&A alone wouldn’t be cutting it.

Dealing with agents

As always with software engineering, it’s not even a recent problem with a new solution. Marvin Minsky wrote about this exact topic in “The Society of Mind” which was first published in 1986. He had already tackled part of this issue, pointing to what we now commonly refer to as "agents." Once again, the answer didn’t change; it was only rebranded.

And of course, in October 2022, a small library that was soon to become its own ecosystem was released under the name of Langchain. It put a heavy focus on creating pipelines for agents. With a point of entry managing the conversation, and being able to call agents for their expertise, the results became way more satisfying.

Using Langchain is basically giving LLMs their own phonebook. “Want to search Google? Ask this guy.”, “Need a copywriter for your article? Give it to this one”

To build an LLM agent, you need 2 things:

• A system prompt to tailor the response format to the requirements, which gave rise to the wonderful world of “prompt engineering” hustlers experts due to very unreliable consistency in the responses

• Optionally, tools to retrieve information to provide up-to-date and context-aware answers by querying data, or asking another agent.

The matter was, even if they could be chained, agents were still basic executants.

Artificial General Intelligence

After all, most solution-oriented thinking doesn't typically emerge when you're knee-deep in execution. This concept is best explored in Daniel Kahneman’s Thinking Fast And Slow (2011), introducing us to the dynamism between System 1 and System 2 thinking. And agents were still operating in the realm of System 1. Fast. Intuitive.2

So the response came swiftly under the guise of the badly named “AGI”3, with BabyAGI and AutoGPT being the most renowned projects at the time. By leveraging chain-of-thought agents that plan, split, and dispatch complex tasks into smaller units; they effectively mimic System 2's logical processing. These planner agents not only delegate tasks but also evaluate the outcomes of other executing or planning agents — essentially simulating a team of experts.

Such systems are still gaining traction as can be attested by the release of frameworks such as Microsoft AutoGen, the $12 million raised by AutoGPT, or the countless projects aimed at emulating teams.

So while these "autonomous" systems currently yield mixed results, they operate on increasingly abstract objectives, foreshadowing near-future capabilities.

Chat ecosystem 101

After a reminder of the recent history and the current context, we can extrapolate and map the playbook to building a complete LLM ecosystem. A task which all of the previously mentioned projects have been building upon.

The LLM ecosystem playbook

1. An entry point for every interaction

2. Leverage a list of executive agents

3. Leverage a list of persona agents

4. Provide instant context intake from sound, image, text, files

5. Access to personal data such as emails, chat, documents

6. Auto-select the appropriate agent

7. Create agents creation on the fly

   1. Agents with code execution

   2. Instant personalities for formulating responses based on the question

8. Create chain-of-thought agents on the fly to deal with multi-step actions

9. Fine-tune user-specific models

10. Allow payments for 3rd party services through the platform

11. Change the default interface4

12. Automate the creation of recurring tasks

What’s the moat?

On that matter, OpenAI’s chat.open.com is certainly the most advanced product of all. An incredible feat, considering it’s operating as a SaaS, Enterprise product, and Consumer product. With their latest announcements, they are blasting through the few items that were remaining on my list:

• They have a Site + App as entry points

• They are releasing the V2 of their “app store” with GPTs

• GPTs are introducing heavy customization

• They are introducing revenue-sharing

• ChatGPT can hear audio, see images, read documents

• Output isn’t limited to text, it can now generate images, audio, and other files

• ChatGPT can create on-the-fly executing agents with “Advanced data analysis”

• It can fetch up-to-date information with “online search”

• Agents are now auto-selected

• They demoed a GPT working with Zapier meaning countless integrations with your own data

• They still have the best and most performant model on the market

• They are working on improving the speed and introducing more chain-of-thought reasoning

• All of that is becoming available as a SaaS product through their API

Simply put, they are spearheading the paradigm shift from the traditional mobile/web application to a centralized experience with ChatGPT. And, of course, they are crowdsourcing every advancement through it, from training data to the creation of personalized assistants. We are giving them the tools to remain in this leadership position willingly.

Moreover, by abstracting context fetching, reasoning, and agent selection, they are rendering a lot of startups working as simple wrappers around GPT-4 obsolete. Also, by streamlining the integration of these tools through their API, they are eliminating complexities previously managed by the now-redundant open-source projects.

But, to me, the defining shift compared to the smartphone revolution is that anyone can create GPTs. Developer skills are a bonus, not a requirement.

Sharing the crumbs

So you may be like me, trying to position yourself in the current context and work with or around these “AI” offerings. If so, you’re probably wondering “What’s left for me?”

If you can’t beat the ecosystem, you might as well join it. And in this regard, there are a couple of ways you can still make it out.

• Join the gold rush to create the most popular GPTs

• Leverage any data you already have to make them

• Create micro-APIs to fill in the gaps not covered by the advanced data analysis tool or existing agents

• Become an expert in creating new agents/GPTs as a service

• Learn to integrate them into products that aren’t chats, and streamline existing user flows

• Leverage it to build faster yourself

The custom wrapper you’ve built or you’re building around GPT’s API won’t cut it anymore. The barrier of entry is being blasted to pieces by OpenAI and you might as well join the assault.

Where integration once reigned, in a landscape where anyone can create new user experiences, quickly seizing opportunities is the new key to success.

Also, I’d love to get your opinion on the matter in the comments 😉

Leave a comment

Sorry for going dark this last month, but I’m sure you’ll understand.

Between a full-time job, family duties, my own AI-related tinkering, Zelda: Tears of the Kingdom’s release, and the beautiful French weather we’ve had lately, it’s been hard to focus on writing. But I haven’t forgotten you 😉.

Last week I had the pleasure of helping host our company’s AI-oriented offsite. So just to get things back on track, I'd like to share some recent discoveries I've made through trial and error.

Disclaimer: AI tools were used for rephrasing some sentences and for generating the post’s title image

I changed my PoC setup

I’ve talked about my PoC setup for working with AI before:

But this time, I had to get 3 separate teams to iterate fast on topics I didn’t know in advance. So the stack required a couple of updates:

• It needed an easy setup and deployment method

• It had to provide the most extensive set of tools out of the box

• I needed both frontend and backend on the same stack

With those in mind, I had to build it back from the ground up

I ditched Javascript to focus on Python

I like Javascript, okay? I know it’s the perfect maelstrom of pure chaos of what any language is and could be. But despite all of that, I actually love its quirks and whatnot.

And I’ve done wonders with it, be it on the backend or frontend. But when I started working with data on my AI projects, oh boy, what a disaster it was.

Javascript is great at many things but is terrible at providing fully-fledged frameworks and libraries. In 2020 there were already over 1.3 million packages on npm, but to this day, there is still no consensus on what the default backend framework for NodeJS even is. And it’s worse when working with data, a lot worse…

So the obvious choice was Python. And while I loathe its heavily object-oriented syntax without proper typings, the Python packages for working with data are just in a different league. I knew it was the language for data scientists, but I had no idea how far ahead it was from other languages.

Just a quick sample so you get my point. I’m building my own AI-assisted tools to monitor news on tech. One of the core features is the ability to send an article’s URL to feed into my database. In JS, I considered myself lucky with @extractus/article-extractor. But in Python, we’ve got Newspaper3k which, on top of essential data, gives you:

• Summary extraction

• Keywords extraction

• Google trending terms extraction

And every data-related library is like this. Even Langchain is leagues better in Python than in JS. Just compare the amount of integrations between the two.

So, from now on, I’m sticking with Python on the backend side.

Langchain is great (kind of)

I just mentioned Langchain, and I have to discuss it briefly.

One thing Langchain does well is documenting use cases and making you feel powerful. Open up an integration, copy/paste the code, and voilà! It also thoroughly introduces the essential modules needed to effectively engage with an LLM.

Unfortunatunately, I can’t say the same about actually documenting the way it’s architectured and works inside. Going past the usual “getting started” felt incomprehensibly convoluted. I’m sure focusing on object-oriented instead of functional programming doesn’t help. Still, I had to implement each module myself without using it to finally get a good grasp of its inner workings.

However, due to its widespread use and numerous integrations, I had no choice but to use it as the primary framework for interacting with LLMs during the hackathon.

And it actually went pretty well. Of course, I split my template code into neatly named files to help my teammates understand and iterate quickly. But, since that experience, I’m considering using more of its parts in my mainline developments.

Streamlit is dope

(unless you want to go into production)

Being stuck with Python, I also needed a way for the devs to quickly iterate on features without spending hours on the frontend.

Fortunately, there is an awesome Python package for that called Streamlit. It allows you to build a pretty reactive frontend with a few lines of code. And even better, they offer their own push-to-prod hosting solution for the site you’ve made.

Here’s a preview of what I made with a dozen lines of code based on this article:

Regrettably, the downside of Streamlit is that while it’s neat for building demos, such as most of Hugginface’s model pages, the way it works makes it impossible to use to build a company’s product.

Security is a pain

Of the 3 PoCs presented at the end of the offsite, all were terrific demos, but absolutely none were safe:

• One allowed GPT3.5 to respond, potentially putting the user in danger or engaging the company's liability without any validation.

• Another ran generated SQL queries directly on our database, making it prone to SQL injections of the worst order (hello OWasp Top 10 👋).

• The last one didn’t do any of those, but has anyone actually figured out how to prevent prompt injection?

And while most of these can be prevented one way or another, the solutions won’t come without some severe feature limitations.

So, for now, the most straightforward use cases are definitely internal tools because I’m scared to death of releasing a public AI feature using proprietary data. Good thing OWasp has been branching out to AI lately to help us out.

A quick chat on use cases

Security aside, I’m starting to notice a pattern emerge from the many tools and use cases I’ve encountered lately.

Dealing with chaos

LLMs are great at dealing with unstructured information: we tried giving statements in an unorganized order, often going back on our words, and as you can imagine, GPT-3.5 dealt well with parsing it, and GPT-4 did even better.

They are also getting good at structuring said information. But even with GPT-4 and several system prompt iterations, I’m still getting misses. So we’re still far from ditching the infamous maybeItsJsonButIfNotRetry function, but I’m hopeful for Anthropic’s Claude “constitutional AI”.

Killing the ant with a bazooka

Another trend I’m noticing is that because OpenAI’s GPT is so good at dealing with such varied topics and tasks, it’s likely way too powerful for most of these tasks.

One such example I have in mind is that while building my own AI-assisted knowledge base, I wanted to extract both summary and keywords. 2 tasks that, let me remind you, are included by default in Newspaper3k using much smaller models which can be installed along the rest of your backend.

Another consequence of heavily relying on top LLMs is that I become dependent on private models and constrained to using APIs which I’m not really a fan of.

So what’s next

Now that I’m starting to get a good feel for the main tools, I’m looking forward to trying out and integrating smaller models to deal with specific use cases.

And, of course, I’ll keep on sharing my discoveries. So, if you’re not subscribed already, it’s only one step away 😁

While working on my projects, I wanted to share my setup to test out new technologies and concepts as quickly as possible.

Disclaimer: AI was used for image generation and to help rephrase a few sentences.

What I avoid in a PoC

Before we start, let’s talk about a few time wasters I tend to avoid when dealing with a PoC:

• UX/UI: Even if I talk about it here, UX/UI is my biggest time waster when working on PoCs and is often the last thing I want to work on. Also, as a maker, a good UI is worthless if the feature doesn’t work.

• Design systems: there are dozens out there. You can find my (current) favorite one below, so I don’t waste time writing mine.

• Validation: I know what I can and can’t do with my code. No need to lock it down at first.

• Global state management: Whenever I’ve used a state management library while working on a PoC, I lose myself in it instead of focusing on features. So now, I stay pure React and use custom providers as much as possible. I even wrote some code I re-use across projects to infer type safety.

Dev environment

While I was used to setting up local projects for developing PoC, I now use Replit as much as possible for a few key reasons:

• Browser-based IDE: I can work on my code from anywhere (there is even a mobile app) as long as I have a good-enough internet connection.

• Env variable handling: This allows me to easily manage protected environment variables without using extra tools.

• Cmd+Enter to run and get an HTTPS URL: I can inspect and share my dev URL in a matter of minutes. It’s also really helpful when I need to test and develop webhook endpoints.

• Repl templates: When looking for a base setup to get started with development in a new language or library, I just select a Repl template, and I’m ready to run the code instantly.

• Integrates completion / formaters: All my usual tools (typescript, prettier, eslint) are available as soon as I install the required dependencies.

• Ghostwriter: I enjoy working without Github Copilot, and I was positively surprised when trying out Replit’s Ghostwriter. On top of auto-complete, it’s available within a chat window and as a contextualized command when selecting code.

• Collaborative programming: For experimental development, it’s way faster than any git-based workflow.

Regarding pricing, there is a free tier. But I recommend paying the 20$ a month subscription to access private Repls, Ghostwritter, a boosted server, and the ability to keep your dev server up. And no, in case you’re wondering, I didn’t get anything from Replit for this article.

When necessary, I switch to a traditional local git-based setup for projects. However, I've noticed that Replit takes me further every time without having to do so.

Data storage

Few applications are complete without some kind of data storage. I’m a huge PostgreSQL fan because it can do almost everything other databases do:

• Relational storage

• Key-value storage

• On event functions

• Vector-based comparison (necessary for embeddings)

• And many more I don’t even know of

So it shouldn’t be a surprise that my go-to service to get a project running is Supabase.

• Perfect free tier for tinkering

• High-level configuration through UI

• Low-level configuration through SQL scripts

• Authentication + Row level security when needed

• Webhooks

• And a Firebase-like JS SDK with the ability to generate Typescript typings.

Stack

I’m still on the edge between a Javascript or Python environment for LLM development. But as I feel that UX/UI will be king for LLM integration, and because I’m most familiar with it, I’m sticking to JS only right now.

Here is the stack I’m currently using out of order:

• GPT4: Which I wrote in length about in the previous article

• NextJS: This React framework handles 80% of my use case. I can write an API or server server-side rendered pages, which allows me to easily combine UX/UI and data.

• ViteJS: Sometimes, building a SPA is required. In that case, I instantly switch to Vite for bundling.

• Tailwind + DaisyUI: I still hate raw Tailwind with a passion (just look at the code tab of this example), but I always found it the easiest way to get started with styling. Fortunately, DaisyUI was the component library to win me over. It’s still pure CSS and comes packed with themes and components to make my life easier with just a few readable class names.

• Typescript: Type safety and auto-completion is king. Thanks to it, I develop way faster without looking up API specs as often as I used to.

• Prettier: Format on save. Enough said.

• ESLint: It’s baked in with NextJS, so I use it, but it tends to slow me down for no good reason when starting up and experimenting, so sometimes I wait a bit before enabling it.

• Jest/Vitest: When working on a PoC, I only use it if I plan on going TDD on a particular function.

• Luxon: Functional date manipulation. No more date mutation.

• Lodash: Structures manipulation. Utils for objects, arrays, …

• Axios: Easy data fetching for 3rd party APIs

• TanStack-Query: easy query management with hooks. Combines well with Axios and other async-based APIs.

I probably missed a few and will update this article accordingly if I remember them. In case you have any recommendations for me to upgrade my PoC stack, I’m all for it, and the comments are always open.

Leave a comment

So thanks for reading through. If you found this article helpful, I would greatly appreciate it if you could share it with others. See you next time 😉

So I want to take a step back and reflect on how AI has changed my approach to software development, where it shines, and where it’s still lacking. In this context, I will focus on the basic usage of ChatGPT4 without plugins to go through the following tasks:

• Research & planning

• New project setup

• Writing API-heavy code

• Writing tests

• Implementing the function to pass those tests

• Debugging

• Upgrading an existing feature

Disclaimer: AI helped with the header image and enhanced a couple of phrasings

Lean’s approach to productivity

Graphics are courtesy of the wonderful Tycho Tatitscheff. Who also taught me about those terms. You can follow him on Twitter.

Let’s analyze the development process of a new minor feature in terms of time.

Lead time in this context is the time taken to deliver value, and it’s the elapsed time between the start of development and the moment it reaches production.

Touchtime is the time spent concentrating on the task. I tend to include time spent understanding the business process, testing out tools, and technical planning, on top of the development process itself.

Value-added time is the time spent on producing pure value, and most of it is the time spent coding. And by coding, I mean actively typing code to introduce the feature. You don’t count refactoring, documentation, or reading, … This means that this chart is actually pretty optimistic because, as a matter of fact, value-added time, in this case, would probably be around 20 minutes.

The gray part is off-time, it delays the lead time, but it’s not time spent actively working on this feature.

And what about the yellow parts? It’s waste, pure waste. It’s time spent understanding the business, planning things, waiting for the CI to run, (testing)… Basically, everything you wish you wouldn’t have to do.

As a craftsman and maker, increasing my productivity means reducing my lead time by:

• Reducing off-time in between touch-time

• Reducing touch-time by:

  • Reducing the required value-added time

  • Getting value-added time/touch time ratio as close as possible to 100% by:

    • Eliminating waste

AI’s impact

Now that we speak the same language, how has ChatGPT4 impacted my productivity? Well, it’s been an insightful investigation. Of course, the honest answer is “It depends”. And for that, I’ll analyze a couple of everyday tasks.

Research and planning

Remember, research and planning are actually waste and should be eliminated as much as possible.

ChatGPT4 has been a massive help for me when dealing with unknown languages or domains.

For example, I’m researching how “extend the memory” of ChatGPT. Asking the following got me started quickly:

I'm trying to "extend the memory" of ChatGPT when using its API, as I'm limited by context size. What are the steps to achieve this programatically?

I then ask for details for each step and an explanation of the relevant terms.

What are the steps once I get a new response to store it? (even better if you can reference a step it provided previously)

What’s a vector database?

Thanks to the responses:

• I have a detailed plan of the required steps to achieve my target result

• I know the relevant terms to Google instead of guessing

This allowed me to reduce my waste for research and planning by ~80%.

New project setup

The “getting started” task has probably been the one most impacted by my usage of AI. While I started giving out precise instructions, I’ve become a lot less specific in my directions after a while.

In a non-AI-enabled approach, I would have to

• Search for the best language/framework/libraries combo

• Setup my machine if I’m missing tools

• Go through the getting started documentation and run the commands

• Look for ways to tailor the solution to my standards

• Make sure it runs

At most, in 1h of touch time, my value-added time is around 2 minutes, as most frameworks give you command lines to copy-paste in your terminal to set things up.

But with AI, the result looks completely different because my process is the following:

• Write out my specs:

  I want to setup a blog. Give me the commands and code to achieve this:

  • Use NextJS

  • Articles written in markdown files

  • Possibility to create custom components in React render part of the articles

  • Uses TS, Eslint, Prettier

• Wait for it to produce the response

• Copy-paste terminal commands

• Copy-paste file contents

• Review

• Fix/improve

  

So while the value-added time doesn’t change, in this case, the ratio goes from 3% to 13% as less time is spent getting to the commands to run.

One caveat to this approach is that it relies on known libraries and frameworks. This means that it’s impacted by ChatGPT4’s knowledge cutoff and may require a lot of specifications to achieve what I consider to be a good result.

The bad:

• In my case, I had to specify NextJS as it kept recommending create-react-app without.

• I had to clean up some of the base configs it provided

The good:

• I get the commands immediately

• It recommended using .mdx files (markdown with react) and used a library to get them to work with Next, which saved me time.

So a lousy prompt can quickly turn this time-saving process into extra cost. When using this approach, I try to stay with the most common tools (in this case, React, Next, …) to achieve the best results.

Writing API-heavy code

The notion of API here refers to the API of SDKs, libraries, or frameworks. Not only HTTP requests.

One of my top uses of ChatGPT4 is to achieve HTML DOM manipulation. It works well because the API has been stable for a while, and it’s probably been trained on many examples. So I can give out a task such as:

Give me a function to watch for every button element that’s right after a textarea element and change the text content to “PUSH THIS!”.

The code usually works instantly, and it also works with much more complex examples. A task that has often cost me a lot of time with constant interruptions to dig through MDN’s docs is now almost instantaneous.

Unfortunately, without plugins or using an agent to fetch extra context, it’s impossible to achieve the same result with up-to-date libraries that it hasn’t been trained on. In this case, I’m back to Googling.

Writing tests

I’ll write on the principle that unit tests, while considered waste, will lead to reducing it later on.

AI may be the answer to spreading TDD (more on this in a later post). Writing tests often feels like a drag and requires a lot of practice to get efficient with. For complicated business rules, translating them to tests can also be time-consuming.

This is where ChatGPT4 helps me. For example:

In TS, with jest, give me a set of tests to comply with the following rules for "getTermStartDate"

• The input is a date and a term duration in months (1,3,6 or 12)

• It should always return the start of the current period based on the input date

• The first term of the year always starts at january 1st

I know it’s pretty simple, so here is how it fared.

The good:

• My rules were turned into a good set of tests to check my function

• It covers every case I could have thought of

The bad:

• Not every test is meaningful, and some are repetitive. In this case, it focused too much on the “January 1st” rule and gave me 2 tests to check it. While not very problematic, it can raise the waste of maintaining those tests later.

Overall, it’s an excellent starting point and provides a working first draft of the required specs, all with business rules which could have been written mainly by a non-technical person.

Once again, this is a considerable reduction in waste time, primarily due to the translation to code ChatGPT operated.

Implementing the function to pass those tests

This is where most of the value-added time can be found

Whenever I give it the tests, ChatGPT4 is very efficient in implementing the function itself. Some back-and-forth may be required by giving it the test runner error output, but I’ve achieved good results.

In a case where the context is still small and previous messages are included, I usually just write:

Implement the getTermStartDate method to pass those tests using luxon

Here, I didn’t have anything to write. The function was provided just along the test cases. Value-added time is reduced to (almost) 0, along with the touch time, and my effectiveness ratio goes to nearly 100%. It’s clearly a win.

However, there’s a significant limitation in realizing this objective. Optimizing ChatGPT -4’s ability to generate code requires careful planning and breaking down extensive features into smaller, more manageable functions.

Fortunately, if you remember “Research and planning”, it can also help you with that.

Debugging

I haven’t been able to achieve complex debugging with ChatGPT-4 because of a few reasons.

First, debugging is mostly about dichotomy and requires opening and going through multiple files to determine where the issue is. Thus, debugging involves a lot of context, most of the time above GPT4’s capacity.

Second, it requires back and forth between the code and your API/Website/Logs/… to retrigger and read the error. It should be possible with some advanced AI integration (see Langchain agents), but GPT4 alone can’t do so.

So no net gain on this side, although it once provided with helpful guess on where to look within a bugged small function.

Upgrading an existing feature

This is an area where ChatGPT4 both shines and lacks. So let me throw a quick summary:

I use it for:

• Explaining a new language: With it, I was able to understand and write Python code in a matter of minutes. A language I have never used before.

• Explaining small functions: When checking out the jumbled-up code of another GPT Chrome extension (I don’t get how they can move around in their codebase), it gave me a good summary of a function with way too many responsibilities

  • Loads conversation from nodes.

  • Retrieves user data (name, avatar) and settings.

  • Reconstructs conversation by traversing nodes.

  • Renders messages for user and assistant.

  • Inserts rendered messages into DOM.

  • Sets up event listeners and updates UI elements.

• Refactoring: By telling it to adhere to DRY, SOLID, DDD, TDD, and as many acronyms as I could think of, it gave me basic refactoring steps, which I can ask an implementation for later on

  • Extract helper functions for better readability and separation of concerns.

  • Utilize async/await with Promisified chrome.storage API.

  • Replace for loop with Array.prototype.map().

  • Simplify DOM manipulation with a more straightforward approach.

  • Create a separate function to set up event listeners and update UI elements.

• Updating small independent functions: I’ve successfully got GPT4 to update functions under ~40 lines.

I don’t use it for:

• Tasks expanding across many functions: too much context is required to be effective. And the answer is often cut off by the max response size.

• Updating big functions: Same reason. I try to ask for a refactor first (or do it myself). Otherwise, the response is cut off.

• Updating functions dependent on others: Sometimes work, but the other function names must be very explicit. And the chat won’t know about available functions which aren’t in its context.

The context of working on an existing feature varies too much for me to do any accurate measurement. But all in all, it’s still a net gain. Either I reduce my lead time, or it remains the same.

Where ChatGPT4 hasn’t helped me yet

Please keep in mind that while it’s been a very effective tool for increasing my productivity, there are a couple of areas where I’ve been unable to use AI:

• Up-to-date APIs: When developing my Chrome extension, it primarily recommends the deprecated v2 Chrome API. So I now have to rely on the doc. This might be fixed by ingesting the documentation first and providing it as context.

• UI: I haven’t managed to get it to produce a good complex layout which is not a default “getting started layout”. Also, every time I’ve tried, there’s been too much back and forth, maxing out the context. I guess that low-code/no-code platforms are the best equipped to deal with this.

Summing it up

Considering everything I’ve talked about, in almost every situation, GPT4 increased my productivity. I got me to:

• Reduce my lead time

• Reach the first value-added time task faster (the fun part)

• Raise my value-added time/touch time ratio

• Get to production more quickly (my guess is that it would have taken me 3x more to release my Chrome extension without it)

What’s remarkable is that this isn’t just a tool replacing another with its typical trade-offs. It’s just a pure boost that speeds up all my coding activities. So if you’re not already a user, I advise you to at least try it out and make your own opinion.

That was a long read. If you’ve reached this point, thanks a lot for reading through 😉.

As always, if you found this helpful, it would significantly help me if you could share this post.

In my next one, I’ll talk a bit more about my AI-enabled development environment. See you next time…

Disclaimer: an AI was only put to work for the header image, glossary definitions, and examples (screenshots)

Glossary

• LLM: A Large Language Model (LLM) is an AI model trained on massive amounts of text data. LLMs can understand and generate human-like text, allowing them to perform various natural language processing tasks, such as translation, summarization, and question-answering

Get access

As you might have understood, I’ll focus on chat LLMs in this post. So here are my favorite ones by order of preference:

• ChatGPT-4:

  • Price: 20$ (+tax) / month for ChatGPT+

  • Why: Allows most personalization of the chat’s “personality”. Has been providing me with the best results and can help with multi-step projects if used well

• Bing:

  • Price: Free, requires Microsoft Account + Edge

  • Why: well worth the price of $0. It seems to be a mix of GPT3.5 and GPT4. A good place to start if you don’t want to pay.

  • Why not: Bing search is integrated but actually gets in the way when using it for software development. UI is not great with code blocks.

• ChatGPT-3.5:

  • Price: Free, Boosted with ChatGPT+

  • Why: Really fast to answer. Ok for a single question or working on a really small subset of code.

  • Why not: Response quality is way lower than GPT4’s and Bing’s. Not great with following multiple steps.

• Bonus, Repplit Ghostwritter:

  • Price: 20$/month (with other advantages)

  • Why: It’s integrated into Repplit, a fully online code editor with hosting capacities. It’s now my favorite tool for tinkering with code.

  • Why not: Responses are good but not up to GPT4’s level.

What you should not do

• Try to search with it: All of them are trained with data as recent as September 2021. Also, note that apart from Bing Chat, none are connected to the internet by default. And be wary that Bing’s answer may not match the provided reference’s content.

• Expect up-to-date library/framework standards: Again, due to the knowledge cutoff, they don’t know about recent releases and breaking changes of your favorite framework.

• Try to get it to write the complete projects: I think we’ll get there fast, but right now, the main limitation is the chat’s memory. I already wrote more about this on Twitter.

What you should try

Here are the basic things I now use ChatGPT-4 exclusively for:

Catch up & learn

I’ve started using Python to integrate LLMs into my own projects. I chose Python because it’s the most commonly used language for AI-related projects and has the most cutting-edge libraries.

But I don’t know Python. I’ve only dabbled with it. So I’ve been using GPT4 to teach me the basics:

• “How do you set up a python project?”

• “What’s this poetry.lock file ?”

• “How do you do X in python”

• And even better if you know another language, “Translate this JS code on Python” with some JS code right under

  

Tip of my tongue prompts

When explaining some concepts or trying to find resources on a language/library, I often can’t remember the correct term to define a particular element.

So I pop up ChatGPT and ask for the name with a rough description or example

Plan

While ChatGPT-4 can’t write a full project easily for you yet. It is very helpful in planning things out. Right now, I’m working on prompting my own data. But I don’t know where to begin. Well, let’s ask:

And I make sure to ask for sub-steps:

Implement

If the step is small enough, I ask it to do it for me:

A nice addition is that it also recommends useful libraries to achieve your goals. But be sure to do your due diligence and watch out for deprecated recommendations.

Fix code

If the code doesn’t work, copy-paste the error into the text box, and ChatGPT will fix it.

I also use it in a code review format. It works with complex code but won’t process the whole response if the code is too long.

Limitations to watch out for:

I’ve already talked about this, but it’s always nice to set a reminder. So here are the main limitations once again:

• The Chat will start to forget after some time

• The libraries and standards used in the code may not always be up to date

• If the response is too big, it will stop halfway through

• Do not use it for search

Ok, we’re done with the first post. Hope you found it useful. If so, please help me by recommending and sharing my Newsletter.

🙋 Also, while I have your attention, I’d love to know what you’re most interested in 👇

And if you’re looking for something else, you’re always welcome to leave a comment:

Leave a comment

You were promised a VSCode plugin’s news, but here you are with the first post of some guy’s newsletter on AI and development.

Well, let me tell you, I’ve learned and rambled a lot on the way to creating this VSCode plugin. And while I might actually end up releasing it, I’ve wanted to take the opportunity to share as much as possible about my discoveries on working with AI as a Software Developer. But Twitter’s future looks bleak, and it’s not appealing to everyone.

So I’ve decided to launch the pAIrprog newsletter for the AI-curious developer.

With it, my goal is to:

• Help you stay up to date with the latest news and trends of ChatGPT and alternatives

• Help you get started with it either for development or integration

• Share the best resources I’ve come across

• Talk about products or experiments I’m doing with AI

• And, of course, share my random thoughts on the topic of AIs

• All in a condensed (probably) weekly format for you to read 👌

Disclaimer: No AI was hurt used in the making of this post except for the image