Copilot Made Me An Augmented Programmer

GitHub Copilot advertises itself as "Your AI pair programmer". How does it feel to develop with an AI assistant? How much does it improve the practice of programming, and what are the risks?

Copilot Writes Code For You

Tools leveraging artificial intelligence (AI) are transforming the everyday life of workers in every domain. In programming, the biggest leap forward has been OpenAI Codex, an AI system that translates natural language to code. This system is accessible via a service called GitHub Copilot and integrated into modern IDEs like VSCode.

I've been a GitHub Copilot user from day one:

I've just seen the future of programming.

— François Zaninotto (@francoisz) June 10, 2021

Since then, I've been using Copilot almost every day. To say I'm blown away is an understatement.

How does it feel to code with an AI assistant? Let me share the screencast of a coding session with Copilot.

A Quick Primer On Self-Descriptive Numbers

To showcase coding with Copilot, I wrote a Node.js program to find the 7 self-descriptive numbers in base 10. It's a classical algorithmic problem, based on simple math constructs. But what's a self-descriptive number?

Take the number 213,013. It's made of the following digits:

• 1 zero
• 2 ones
• 1 two
• 2 threes
• 0 fours

Let's write that "12120". We'll call that the "description' of 213,013.

Here are other examples:

• the description of the number 12,345 is '0111110'
• the description of the number 222 is '0030'
• the description of the number 2020 is '2020'

That last example is interesting: 2020 is its own description. We'll call that a "self-descriptive number".

In base 10 (with 10 digits, 0 to 9), there are only 7 self-descriptive numbers. My programming challenge is to code an algorithm to discover these 7 self-descriptive numbers in a reasonable amount of time.

Watch Me Code With Copilot

Here is a 5 minutes screencast of my humble attempt at solving this problem. Copilot came to a valid solution in seconds; the gist of the time was spent on implementing multi-threading to get to the solution faster.

The speed of the video is x2, so the actual session lasted 10 minutes. I stopped the recording before running the program in base 10. Running on 8 threads, the program gave the actual solution in 13 minutes:

$ node index.mjs
[
        1210,      2020,
       21200,   3211000,
    42101000, 521001000,
  6210001000
]

So what does this video show about the experience of programming with Copilot?

Copilot Makes Me A Better Programmer

First, let's state the obvious: I am absolutely not capable of writing that code in 10 minutes. If I had to write it on my own, it would probably take at least 1 hour and require a lot more trial and error. So AI makes me write code a lot faster.

Finding self-descriptive numbers is a very common coding challenge - no wonder copilot found an initial solution in seconds. It probably had the solution in its learning corpus. But the fact that it wrote the entire solution based on a single line of comment is impressive.

In fact, writing clear descriptions is key to getting the most out of Copilot. I often start by writing a comment, or a function name, and wait for Copilot to propose an implementation. I have learned to describe what I want in a way that Copilot can better understand - the same way as graphical designers must learn how to write prompts for Stable Diffusion to generate nice images.

Copilot removes the need for a search engine, or a StackOverflow tab. If there is a syntax that I don't know by heart, I don't need to look for documentation or code snippets. I just need to write down what I'm after (like "adding multithreading" or "flattening an array"). So I spend more time in my IDE, and less time in my browser. With less context switching, I keep my train of thought and my focus longer.

Since copilot takes care of the easy parts (data transformation, pure algorithmic functions, repetitive tasks), I can focus on the hard parts (design decisions, and of course, cache invalidation and naming things). I spend more time thinking about the problem, and less about the implementation details.

Sometimes Copilot suggests bad solutions. I have found that bad suggestions don't slow down the coding process. Copilot gives me more code to look at and improves my capacity to judge if the solution is good or not. I have more possibilities to choose from, and this results in better code. Besides, the time lost on bad suggestions is negligible compared to the time saved on good suggestions.

Finally, Copilot isn't only good at solving math problems. It has helped me with frontend code (HTML, CSS, React), API endpoints, asynchronous workers, SQL queries, automated tests, documentation, and even writing blog posts like this one. Copilot is a general-purpose AI assistant for developers, and it facilitates many of a developer's daily tasks.

Copilot Makes Me A Worse Programmer

There is no such thing as a free lunch. Copilot has some drawbacks, too, and they are not negligible.

Since Copilot handles most low-level technical code, I became less efficient at solving simple problems. During the latest year, I haven't exercised the same programming muscles as before. I'm also getting older, but I don't think it explains that simple coding challenges require more effort. That's an expected consequence of automation, which can be seen in every domain. Today, most people (me included) need a spreadsheet to do basic addition, while their parents could do the same in their heads.

Sometimes Copilot proposes large code snippets, which look all right but don't work correctly. Copilot code may contain bugs - I'd say it writes buggy code about 30% of the time. I write buggy code, too, so I have learned to fix that. But as I trust Copilot more every day, I still need to carefully review the code it generates. In fact, I spend more time reviewing code than writing it. I don't mind reviewing code, but I know that some developers feel they don't spend enough time coding.

Sometimes, Copilot suggests code that I am not allowed to use. That's because it was trained on code with various open-source licenses. These licenses impose certain obligations to the people using the code, the minimum of which is attribution. Copilot doesn't tell me when I have to attribute code. Besides, if Copilot copies code with a copyleft license (say, GPL-v3), I legally don't have the right to use it in a project with a different license. Finally, if a GitHub repository has no license, it's copyrighted by default, but Copilot thinks otherwise. This problem is serious enough to justify an ongoing class action.

I never look for the origin of the code that Copilot suggests - and that's the problem. I did look for instances of the code of the above video though, and couldn't find it anywhere on GitHub or Google.

And the problem that bothers me the most is that Copilot sends all my code to Microsoft. Copilot doesn't run on a workstation, it runs in the cloud, so my IDE sends a huge amount of data to the cloud, possibly including personal data. For open-source code, it's not that big a deal. For copyrighted code, I have to trust Microsoft... And since 2013 and the PRISM scandal, I don't trust Microsoft anymore. So I try to turn Copilot off whenever I work on a project that I don't want to share with Microsoft - but sometimes I forget.

It's Inevitable

Despite all these problems, my conviction is that AI has changed the way developers work forever. As it allows me to program faster, Copilot made me more productive than "classic" developers. CS students already use Copilot in class. They skip the basics and will probably never know how to find self-descriptive numbers in base 10 on their own. But it doesn't matter, because, during their career, they will be assisted by AI all along.

In the long run, I believe developers without AI assistance will be left behind. And again, this is the case with most innovations. When looms were invented, weavers were replaced by machines. Copilot was invented, and soon developers will be replaced by machines. At least for the easy parts of the job.

So I think that AI-assisted coding will become the norm soon. How does a world where developers always code with AI look like? The knowledge and productivity of Senior developers will become the basic requirement. Developers will write less code, work more on analysis, and talk more with users. In short, developers will spend less time in front of a computer. Software will be even more ubiquitous and solve even more problems.

And then one day, developers won't be necessary at all, and we'll all float around in hoverchairs, watching advertisements and sipping soda, while our planet is dying. But that's another story.

Conclusion

Copilot is a paid product, but as an open-source developer, I got it for free. But even at $10 USD/month, I think it's totally worth the dime. Note that I'm not affiliated with Microsoft in any way, and to be clear, I think that Windows and Azure are garbage.

My day job changed radically on June 21, 2021. I have embraced that change, and I think that I won't come back to the way it was before. Coding without Copilot now feels slow and painful.

I'm also concerned with the impact on the developer profession. But we've been developing programs that take people's jobs for decades, so it's about time we suffer the same consequences.

Did you try Copilot? What do you think about it? Let me know in the comments below.