How to make an AI podcast about an academic subject

I’ve been trying Google’s Notebook LM to create podcast-like conversations from uploaded academic papers.

My reaction to what it came up with reminds me of how I felt when Google itself first came along. Early users were so impressed that Google grew its customer-base by word of mouth alone - my mouth being one of them.

I’m not going to be evangelising about NotebookLM in the same way because its practical value is still limited. But it’s still an impressive demonstration of what AI is already capable of.

Notebook LM is a service that focuses on the material you give it, in addition to drawing on more general AI knowledge. You provide it with sources in collections it calls Notebooks, which are separate projects each devoted to a particular subject. The sources can be documents, websites, videos or pasted text. These are the ingredients of the meal it will cook for you.

Once you have given it material to work with, you have the chance to interact with your project in many ways - asking it to generate a briefing document, a study guide, a timeline, a list of FAQs or simply have a chatbot text exchange on the subject.

But the most eye-catching is its ability to create a podcast-style conversation about the content of the material you have provided.

On the left below, you can see a list of the three PDFs of academic papers I gave it. It immediately summarises them in the central Chat section. If you click Audio Overview at the bottom of that section, it comes up with what sounds a lot like a podcast (minus any title music or ads).

If you just let it generate its Audio Overview without further instructions, it produces a chat between two people who seem a bit naïve and, to my mind, rather over-enthusiastic. They’re always saying “wow!” and “that’s really interesting!” and congratulating each other on making fairly obvious points.

But you also have the chance to instruct the AI on what you want it to do in creating the audio. So I had a second attempt, this time telling it to tone down the ‘gee whizz’ quality of its first effort. This was my instruction:

Make a conversational podcast, which the host announces as an analysis of the three papers, naming them, their sources and authors at the start, and then asking the guest to go through each in turn, summarising their main points. Then the host should invite the guest to compare the three papers and to say what's learnt from them together. The speakers should be enthusiastic, but also restrained. They should not say "wow" or sound surprised.

Version two of the podcast was better. The basic plausibility of all of these conversations is excellent. They comes complete with realistic “ums” and “ers” and the kind of idiomatic language and variable timing that makes it sound like a real conversation, rather than smooth chatbotese.

I had three main reservations:

- It didn’t distinguish between the ‘presenter’ and the ‘guest’. Instead, you are listening to two equal partners, which makes it a little confusing if you expect to hear a podcast host talking to someone who’s an expert on the subject, or at least, to have one of them introduce the subject and be responsible for the structure of the conversation. 

- Some of the facts were wrong. For instance – and here we need to get into the detail of the subject I gave it, which was Harriet Beecher Stowe’s views on the Highland Clearances - Stowe was accused of having listened to her aristocratic friend, the Duchess of Sutherland, whose estate saw some of the most violent evictions. The podcast said that the Duchess was responsible, when in fact she had only married into the family long after the Clearances had happened. So it can make mistakes. (Of course, it could defend itself by saying that makes it more like a real podcast.)

- The second half of the podcast abandoned the task I’d set it, which was to talk about the three papers. Instead, it slid into a general discussion of Uncle Tom’s Cabin, and ideas of racism in the nineteenth century compared to today. It was as if the speakers had run out of material and had to fall back on a more general subject, perhaps the subject which is most often associated with Stowe’s name and Uncle Tom’s Cabin.

There was a preachy quality to this part of the conversation:

Speaker 1: It's a challenge to us to really examine our own biases, our own motivations, and to recognize that the fight for social justice requires constant introspection.

Speaker 2: Absolutely. And I think in the end, that's where the enduring legacy of Uncle Tom's Cabin lies. It's in its ability to spark those conversations, to challenge us, to confront uncomfortable truths and to inspire us to work towards a more just and equitable world.

Overall I’d give it β- for its second attempt.

I decided to give it a chance to address my complaints. Here was my new instruction (the instruction box is limited to 500 characters):

The host of this academic podcast starts by naming the 3 papers, together with sources and authors. The host then introduces the guest and asks the guest about each paper, and then to discuss overall conclusions. The discussion should stick to what’s in the papers and not include general points about Stowe, her novel or slavery. We are interested in the scholarship in the papers, and perhaps other academic sources - not in general commentary on social values then and now.

The result was a definite improvement. Now one speaker was clearly the host and the other the guest, although the guest didn’t seem to realise she shouldn't interrupt the presenter before she was introduced and that she was supposed to know the papers she was talking about, and not to say at the start that they “sound great”.

To my amusement, there were a couple of commercial breaks, with no commercials in them – “We'll be back in a flash to continue our deep dive into the world of Uncle Tom's Cabin …And we're back, folks”.

And just as there were too many “wows” in the first version, there were no less than eight “deep dives” in this one.

You can hear that version here. (It may take a few seconds to start playing.)

Finally, there’s another feature that NotebookLM offers, which it calls Interactive Mode. Still in Beta, it gives you, as the listener, a chance to interrupt the podcast and join in the conversation.

That’s a lot of fun. I’ve been told by the presenter that I’m “making a good point” and then heard them both going on to discuss it. When I’ve corrected a factual point using this feature, I’ve found that they agree with me rather than defending the previous wrong version, even coming up with new evidence to back up what I’m saying.

None of this is perfect yet, but AI is moving so fast that paying attention to these kind of abilities is worthwhile, not because they are immediately useful - rather than just entertaining - but because they point to the way technology is taking us, probably sooner than we expect.