In the summer of 1970, during the Marian Sessions, I recorded a humorous song I'd written called Alice's Resistor, about a guy in love with a gal in love with her television. (Remember, in 1970, TV sets were built of transistors, resistors, capacitors, and a big vacuum tube that actual people wired together, in a factory, possibly even located here in the United States.) I recorded it on a stereo reel-to-reel tape recorder (stereo recorders being high-end at that time) one channel at a time so I could sing a duet with myself. In 1970 it didn't sound professional by any means, but it was cool and I could play it and impress my friends who, even if they didn't care for my singing, could laugh at my intentional mispronunciation of "Mission Impossible" to force a rhyme. I mean, who doesn't love poetic humor? In any case, here is the original, 1970 recording of Alice's Resistor.

A few weeks ago I finally started playing with AI music sites. The one I started with, and have the most experience with, is Suno. Their basic premise is that you can upload any old recording of yourself singing an original song (Suno doesn't allow covers of other people's songs), and Suno can turn it into a professional-sounding recording in the style of your choice. I'd already seen what that could mean for one song: I fed it Whippoorwill, a little tune with lyrics by my mom, and came away with sixteen usable arrangements of it, from Baroque chamber ensemble to 2020s pop diva, every one of them something I'd actually listen to. (I have an unusually wide range of musical tastes. Italian opera is my least favorite, but only because I don't speak Italian.)

So, I figured, what the heck? And sent it the above. And it sent back:

I was amazed. It was my voice, with most of the inflections in my original recording—yet somehow better. Remember how we all hated Auto-Tune, until Cher made us all Believe? Auto-Tune is basically a filter, like the visual filters we've all played with on our phones. It's a tool, and like any tool, it can be wielded by artists but is mostly (mis)used by morons.

Still, even if misused, I see no harm in using a filter to amuse your friends. Of course I disapprove of creating lies, especially when intended to deceive. But you surely can't be deceived by a "fauxto" of me riding the back of a kangaroo, even if I caption it, Look! I rode a kangaroo!

Songs can certainly be used to deceive, and have been used that way since the first proto-Cher. (Or rather, the first proto-Kid Rock.) But a filtered song is only a deception if it is used to suggest something non-factual. For example, the recreation of my 1970 recording could be used to deceive if I told people I used to own an orchestra, or sing with Bert Kaempfert, and even then it wouldn't be serious unless I were using it as an audition tape to get a job. And I am certainly not doing anything of the sort. (In fact, I'm doing the opposite, in taking you on this journey of investigation.)

So: What if I asked Suno to completely rearrange my song as a symphonic pop instrumental? Facetious lyrics aside, I do pride myself on my ability to create catchy melodies; so maybe as an instrumental it wouldn't be so dated. (The lyrics name-drop Marcus Welby, M.D., which tells you how dated.) The result:

Wow. Remember, I have not unemployed any musicians. I'm a 75-year-old disabled vet and can't afford to hire a harmonica player, much less an orchestra. If AI hadn't made this possible, it wouldn't have been done. (Well, maybe with a big enough lottery win I could also have done it, but it would have during a world cruise.)

So…how the heck does this work?

Here's where I have to admit I had only the vaguest idea. Text AI I kind of grok, and I could even stretch that to composing music. But stems??? So I asked Claude, the AI I write code with, to walk me through what happens between uploading my tape and downloading a new version of it. As I do when learning new things now, this was in the form of a conversation, as I asked more questions as they occurred to me. So: Suno doesn't publish its blueprints, but the parts are the same parts every published system of this kind is built from, and here is how they fit together, with my 1970 tape as the specimen on the table.

Step One: The Tape Is Already Numbers

When someone talks to you, sound is produced by a vibrating vocal cord, which ripples the air passing it. That's what you hear: a continuous ripple of air pressure reaching your ears. When you listen to an old vinyl record, you hear a physical copy of that ripple, cut as a wiggling groove that the needle traces back into sound, along with some crackle and hiss the groove picks up. When you listen to someone on a cellphone, or play an audio file, there's no groove at all; only a stream of numbers that recreates the original sound (more or less) so you can hear it.

A microphone measures a sound wave tens of thousands of times a second, and those measurements are stored in formats like WAV (every number kept), FLAC (every number kept, but packed tighter) and MP3 (the numbers you're unlikely to notice thrown away to save space). They're all just numbers that recreate the original, some more faithfully than others.

A FLAC file is nothing but numeric samples: 44,100 (or, in hi-def, 48,000) measurements of air pressure per second, times two channels, for 170 seconds. Call it fifteen million numbers. That's how a computer stores a song, but it's a terrible way to think about one. A text AI doesn't read your sentences one letter at a time, either; it chops them into tokens, chunks about the size of a short word, and works with those, exactly the same way a human brain does when we read. Music AI pulls the same trick on sound.

Step Two: Sound Gets a Vocabulary

You can think of a neural audio codec as an MP3 that went to school. An MP3, which is a compressed format, throws away the parts of a sound your ear won't miss. Yes, a FLAC sounds better, but unless A/B testing, most people can't hear the difference. A neural codec is trained to do that, by squeezing all those hours of audio down to a trickle of symbols and then rebuilding the audio from the symbols, over and over, until the rebuilt version fools the ear. What it ends up with is a vocabulary of thousands of "sound-words", each standing for a few hundredths of a second of audio, that no human designed or could pronounce. Run my tape through the front half of the codec and the fifteen million numbers become a sequence of tens of thousands of tokens. Now a song is a "sentence", albeit a very long one.

But once a song is a sentence, the same programming that powers a text AI can be pointed at it: a transformer, trained on a staggering amount of recorded music to answer one question over and over: Given everything so far, what comes next? It never took a music theory class—but neither have most singers. It learned the statistics of recorded music, which "contain" music theory the way a million overheard conversations contain grammar: a verse implies a chorus, a snare on two and four implies a kick on one and three, a melody note implies a chord underneath it, and a 1970s folk-pop vocal implies a particular kind of acoustic guitar. The model is also fed the lyrics and a plain-English description of the style alongside the sound-tokens, so it learns which words sing to which notes, and what "symphonic pop" sounds like. Just like we did.

Step Three: Stems

Now, stems. In a modern recording studio, each instrument is recorded on its own track. Mixing adds all the tracks together into the two channels you listen to, and once they're added, they're added: the voice and the guitar share the same wiggling waveform the way eggs share an omelet. A stem is one ingredient of a mix: the vocals, the drums, the bass, "everything else". Unscrambling a mix back into stems is called source separation, and that's done by another neural network, trained on recordings where the separate tracks were known, until it learned what a human voice looks like, versus a snare drum, versus a guitar, when you draw sound as a picture of frequencies over time. Again, this is exactly the way we humans learn to distinguish a voice from its orchestral backing. It doesn't do a perfect job—neither do we, especially as aging takes its toll—but it doesn't have to be, to be useful.

Split my tape and you get two stems that matter: my voices (both of them) and my accompaniment. From those, smaller specialized tools pull out the things a cover needs to know. A pitch tracker follows the vocal and writes down the melody, note by note, the way a human transcriber would. A beat tracker finds the tempo and the bar lines. A speech-recognition model, cousin to the one that captions your videos, reads off the lyrics and lines them up with the melody. A chord estimator listens to the accompaniment and labels the harmony. (Suno sells this unscrambling as a feature in its own right: it will hand you back a finished song as a dozen separate stems, so you can remix it yourself. Which is a thing I never expected to say about a 2-channel tape recording I made at nineteen.)

Step Four: The Prompt Is the Arrangement

Here's the part that answers why does it sound like me, only better? When I ask for a cover, the model is not editing my audio. It is generating an entirely new sequence of sound-tokens, from silence to fade-out, with three things pinned down in advance: the style I typed, the lyrics it read off my tape, and the melody, timing, and vocal character it extracted from my recording. Everything else it's free to invent, and free to invent means will choose the most probable professional recording split-second that fits those constraints.

So it re-performs the song. Where my 1970 self drifted flat, or inhaled mid-phrase, the most probable professional moments don't; so the new one doesn't, either. That's the Auto-Tune part, except that it isn't a filter over my voice; it's a new take, sung in my voice by a statistical model of what my voice would sound like if I'd had another hundred takes, a producer, and, let's face it, a better voice. Call it an idealization of my singing. Most of my inflections survived because they create the melody and phrasing it was handed. The ones that didn't survive were the ones it judged to be mistakes. While I wouldn't have called it that at the time, it is the result of the fact that, as much as I'd love to be a golden-toned Michael Bublé, I'm more of a Willie Nelson wannabe.

The symphonic version is the same trip with one constraint removed. I told it the song was an instrumental, so there's no vocal to reproduce; but the melody it extracted still has to be played by something, and in a symphonic pop recording the thing that plays a lead line is a section of strings or winds. The chords from my accompaniment became the orchestra's harmony; the tempo and the bar lines stayed; the words went away. The harmonica player I couldn't afford turned out to be a probability distribution.

Step Five: Back to Sound

Finally the back half of the codec, the decoder, turns the finished sequence of sound-words back into a waveform, with a last pass of polish for loudness and balance. The file Suno handed me is 48,000 samples a second at 24 bits, a higher-resolution waveform than my reel-to-reel ever carried. None of the samples in it came from my tape. Every one of them was predicted.

It basically took what I did create in 1970, and gave me what I wanted to have created, and would have, if only I'd had the talent, time, money and motivation to do it right.

So, is it a "cover"?

Technically, yes, and the word fits better than you might think. A cover band that learns a song from a demo tape instead of from the radio streaming service is still playing a cover: the composition carries over, the performance is new. That's exactly what crossed the gap here. My melody, my words, my structure, my phrasing went in; not one specific sample of my 1970 audio came out. It's also why Suno insists the song be one's own. The upside is that, for once, the cover is better than the original, and I'm allowed to say so, because I'm both of them.