Making AI Part of Us

Much has been talked about how to get the AIs to do what we actually want. Some of this is in the context of "if we're going to build superintelligent machines, how do we ensure they don't turn us into paperclips"... but there is another, somewhat more prosaic meaning to this as well.

Namely... even with the current level of AIs... if your prompt is three sentences and their output is a ten thousand line software artifact, there is a very real sense in which "I wrote this" is less true vs. if I sat down and hammered it out by hand.

This situation is, actually, not that different from cases that existed before. As in... if I write a file in C++ and then I go on pointing out the beautifully optimized assembly output of GCC as my own, "look at my decision to unroll this loop was the right call here", ... I might end up not agreeing with your worldview thoroughly. There is also the unsatisfying feeling when the compiler takes about as long to run on your much-valued code as just the headers and a single line of "hello world"; at least if it was me making the compiler work hard, I must have done something important in the process?

The feel gets somewhat stronger though, the more AI we pump into this. If you have a song you wrote the lyrics for at least, even though the rest is AI output? That song is more "you" than if you tell it something like "write me a catchy punk rock song about whatever comes to your mind first".

You can, however, slowly, exert influence though still. As you write more examples of code the agents should follow, your preferences are more likely to apply. In the same way as company culture and leadership direction does count when looking at the resulting products, even though the CEO will not be reading through every line of code. It's still not a very direct influence though; you need a lot of adjustment on the AI's side to actually follow these examples, as opposed to the style it has been trained on.

Meanwhile, compare the above "I told the AI to write me a song" example with being the actual lead guitarist of the band. Instead of just vaguely hinting at "make it more punchy", You can just play the actual thing I'd like to hear. (Well, assuming you're an actual good guitarist.) Having the actual instrument in your hands gives you not just the resulting song but a high-bandwidth communication device; there will be something in your head that's meaningfully "you", connected to what sounds come out. You can adjust input parameters easily, by thinking; you can retrain the network by practicing and listening to songs by other bands.

Note that the distinction between "part of me" and "not part of me" is somewhat arbitrary here. If you imagine an especially strict and brainwashy guitar school that makes students sound exactly the same & like the exact same songs by the time they finish... well, it sounds dystopian for a reason, but for this case, your uniqueness is somewhat limited to "I picked the guitar school". But then... if picking songs for a playlist is a form of self-expression...

... actual songs by actual bands are easier to recognize though than "this is your style of playlist".

It all boils down to bandwidth and connectivity.

All we are is a set of well-connected brain regions and body parts, with pretty good reaction times.

Is the skill of driving a car "us"? well, the way it feels: definitely yes. You decide where to go, you take actions... in a way that is significantly more direct than e.g. sitting at the wheel of a (mostly) self-driving car and watching it do its thing.

Except... if you're driving on a long stretch of highway, the amount of interaction between your conscious brain and the "driving" part is... not a whole lot. You're not producing extra conscious input; you're also not going to remember minor details about how you sped up the 87th time when the car in front of you accelerated. You're doing this on "autopilot"; yes there are quotes around the word, but there is a lot of similarity.

The difference starts where something actually interesting happens. You want to turn onto a certain street. If the autopilot is inside your brain, you can just give it high-level instructions and it will happen; you're also getting good, high-level diagnostics, such as "we didn't make this turn because you made this decision in the last moment and we couldn't move 3 lanes to the right in 5 seconds". You can use this subsystem to emulate hypothetical outcomes accurately, too: you can imagine trying to do the lane changes a bit earlier & assess prospective success.

An actual self-driving car won't give any of this feel to you. It will drive smoothly, safely and opaquely.

What if you'd get all those communications though? Let's say you're driving on the highway; you notice that your self-driving car is going oddly slowly. As you begin wondering why, it blasts a package of info into your consciousness a couple hundred milliseconds later, including a reference to a "50" speed limit sign it saw 8 seconds ago on the right, a pointer that directs your attention to the rainy road surface, and the fact that we're in the right lane so it's OK to be slower. It also picks up on the slight sense of urgency due to you being late from work, and starts driving marginally faster.

If we had this level of integration... would we really insist on there being a sharp separation between "the car" and "us"? If we can recall every single speed limit sign in the past 50 miles, either by thinking of the relevant highway stretch or by them just popping up in your head whenever they're relevant... this is close enough to how actual human memory functions; do we really need to insist on pointing out the difference?

In the sense of... technically, when you're "sending an email", it's not you doing the sending part, you're just pressing keys on a keyboard, which results in your computer sending the email, but... you don't quite think about it this way?

The human sense of ego seems to be neatly tolerant of attaching extra parts to itself, as long as we have high bandwidth, subsecond communication channels. After all, you could argue that we ourselves consist of a bunch of brain areas bolted together. Some of them more opaque than others; for example, how much introspection do you have into how "riding a bicycle" functions? Is it enough so that you can explain it in words to someone who has never seen one so that they can ride one next time? (OK you can also use pen and paper to explain if you really want to.)

We have none of this for AI yet. They're, in fact, so slow that you need to run several agents concurrently to be efficient; you do get lengthy explanations (if you bother to read them) but it's decidedly not the feeling of "I know this". It's like talking to another person, not a "motor skill".

This is relevant for alignment, too. Directing AI to do what you want does make sense if you're the one who has a better view of the situation overall; once this flips... not even the best-aligned CTO of the company can make optimal decisions, aligned with the overall direction by the CEO, if the CEO is your dog. As in... how do you even define "good" in such a scenario?

Compare this with a version where your AI agents feel like they're part of you. You don't literally make every decision in the code base, but you can sort of remember every one of them since you can see what happened right away, with sub-second tooling that might or might not be directly connected to your brain. Instead of updating your AGENTS.md, your immediate reaction gets backpropagated into this & similar decisions so that the agents (feeling like: "me next time when I'm not paying attention") can do it better.

Value drift is still a problem; there is still the failure case of you becoming "mostly Claude with some human approvals attached"... but, in a way, value drift is already happening, with or without you being part of the process. If you outsource 10% of your thinking to a substrate that follows the general shape of your thoughts, which then gradually becomes 90%, there is still a good sense of continuity, and you suddenly do not have to solve the problem of aligning something that's 10x the size of your soul.