Experiments with Midjourney and ChatGPT: Does AI Enhance or Replace Creativity? Self-Publishing News Podcast with Dan Holloway and Howard Lovy

Today on the Self-Publishing News podcast: Experiments with Midjourney and ChatGPT. ALLi News Editor Dan Holloway and News and Podcast Producer Howard Lovy discuss the latest developments in artificial intelligence. Apple turns to AI for audiobook narration, and many authors are up in arms over a children's book written and illustrated by AI. Howard talks about his experiments with AI in fiction and how the technology can be incorporated into your creative work without replacing it.

Find more author advice, tips and tools at our Self-publishing Author Advice Center, with a huge archive of nearly 2,000 blog posts, and a handy search box to find key info on the topic you need.

And, if you haven’t already, we invite you to join our organization and become a self-publishing ally. You can do that at allianceindependentauthors.org.

Listen to Self-Publishing News: Experiments with Midjourney and ChatGPT

Don't Miss an #AskALLi Broadcast

Subscribe to our Ask ALLi podcast on iTunes, Stitcher, Player.FM, Overcast, Pocket Casts, or Spotify.

OR, sign up to get notified via email right when a broadcast is about to go live on Facebook and when a new podcast is published (#AskALLi advice on Fridays and indie inspiration on Sundays).

Show Notes

AI for Authors: Practical and Ethical Guidelines

About the Hosts

Dan Holloway is a novelist, poet and spoken word artist. He is the MC of the performance arts show The New Libertines. He competed at the National Poetry Slam final at the Royal Albert Hall. His latest collection, The Transparency of Sutures, is available on Kindle.

Howard Lovy has been a journalist for more than 30 years, and has spent the last eight years amplifying the voices of independent publishers and authors. He works with authors as a book editor to prepare their work to be published. Howard is also a freelance writer specializing in Jewish issues whose work appears regularly in Publishers Weekly, the Jewish Daily Forward, and Longreads. Find Howard at howardlovy.com, LinkedIn and Twitter.

Read the Transcripts: Experiments with Midjourney and ChatGPT

Howard Lovy: Hello, and welcome to the January 2023 edition of Self-Publishing News from the Alliance of Independent Authors. I'm Howard Lovy, ALLi's news and podcast producer and book editor at howardlovy.com. Joining me is ALLi News editor Dan Holloway. Hello Dan, how are you?

Dan Holloway: Hi Howard. Happy New Year.

Howard Lovy: Happy New Year to you too. With 2023, that sounds like such a futuristic date, we have the future to talk about in today's podcast.

Dan Holloway: Yes. So, it's sort of a special edition. We're all tech this week.

Howard Lovy: Right, all tech, all the time. Everybody's talking about artificial intelligence and whether it's good or bad, or a combination of the two.

So, a couple of things are converging in my life. I decided to write a piece of fiction for the first time after spending my whole career writing non-fiction, and at the same time, I'm learning more about AI as part of the creative process. So, I thought I'd use AI in my work and see what all the fuss is about.

Apple Turning to AI Narration for Audiobooks

But first, before we go into that, let's talk about the news and what's happening in the world of AI. I guess we can start with Apple. They recently announced that they are turning to AI for narration in audiobooks.

Dan Holloway: Yes. So, they have announced they're going to have a special collection of AI voice-generated audiobooks. They're not the first to be working on this, it's the first sort of big announcement of an actual collection. So, Google have been working hard on this. At the Self-Publishing Formula conference last summer, I spoke to the team from Google and they're working very hard on this; they clearly see it as part of the future.

Obviously, the bigger piece in this area is Spotify. They've bought a text-to-speech generation company, Semantic. They haven't publicly launched whatever they're going to be doing with it, but it's clearly going to be something of the same.

This is one of those areas, I remember just over a year ago at FutureBook, there was a presentation explaining that this is a big gap in the market, that 95% of eBooks don't have an audio equivalent, and I think this was SpeechKey who were making this pitch. They saw that as the gap that AI could fill as a way of bringing these books to the public, and also of making narration cheaper for indie authors.

It's basically takes a zero off the price of producing an audiobook, though obviously voice narrators are going to have concerns. For a little while, I think people were ignoring that part of the story, it felt like a little bit of a lack of solidarity. There are a lot of people who've suddenly got interested in this side now that authors are being affected. More were getting involved when cover artists were being affected, maybe fewer than might have been involved when it was just voice narrators.

So, I think one thing that we've learned from this whole thing is that we are a creative community, and it is going to affect us all. So, we all need to be thinking collectively about what our response to it is going to be.

Howard Lovy: Meanwhile, I was reading your column earlier this week, or last week, you talked about the reporting of this story, and one reporter said that the AI-generated narration will never capture the magic of the human voice.

And you say, well, that's all well and good until it does. There goes that argument.

The Limitations of AI Art

Dan Holloway: It was really interesting watching people's response to AI art last year. If 2021 was about narration, 2022 was about art, and this year's clearly about text. AI-generated art went from really rubbish, sort of only a few months ago, to being suddenly really good, which is, I guess, something you can comment on, because I've seen your experiments with Midjourney, and the results are really quite startling.

Art created by Midjourney. The violin is a little too big and the girl has too many fingers. The technology isn't quite ready to replace real artists … yet.

Howard Lovy: Yeah, they are, they're incredible. They're also limited, and it could be because, you know, garbage in, garbage out, and maybe I still need to learn how to do the correct prompts, but part of what I'm doing is using Midjourney to picture what these characters I'm creating might look like.

Some of the limitations that I've seen, well, it can't do something complicated like one character playing a violin and another playing a guitar. I kept trying over and over again. You would do one character playing a violin, and the other one's sitting there or the other, or one character playing a guitar, and the other one's sitting there. But it's too complicated to ask for it to do both, and I tried every prompt I could think of.

Then, when it does it, sometimes the violin in looks too big, or sometimes there are too many fingers on the hands.

Dan Holloway: Yeah, this is something I've come across a lot. I've come across memes with this, with AI hands.

Howard Lovy: Right, exactly. So, I actually specified once, ‘five fingers on each hand', and I think this thing has a personality and is being sarcastic with me because then it gave me a picture with no fingers.

So anyway, there seems to be a great deal of trial and error, but I'm learning how to create different images, like sometimes it can be a cartoon image, sometimes it could be a 3D realistic image.

My feeling is that it's not yet replacing a real human artist. I think from what I hear, the human artists, you create things on Midjourney and then use Photoshop to take care of the rest, like, have five fingers on each hand, for example,

But it is pretty amazing to think about what it is I'm writing and then have it pictured, and then I can go back and actually describe what I'm seeing on the screen, and that creates this feedback loop, I guess.

Dan Holloway: That's what I've been essentially using it for, is for creating settings.

So, settings for libraries and cityscapes and things like that, which it's a lot better at. It's quite vague and blurry, but you don't have these obvious, horrendous errors, like having seven fingers on a hand, and it is really effective for that.

But I think, and a lot of people seem to be saying this, that exactly what you describe is going to be ultimately how it's used, that what you might call the grunt work of creating whatever it is we're going to create can be done by the AI, and then we as artists do the polishing. So, it can produce, in the case of a novel, it could produce a hundred thousand words of text. It could be the first draft, and then we craft that into our voice and put our shape and finish on it.

Howard Lovy: Yeah. Well, I was experimenting with text too, and we'll get to that later in the show, but what I wanted to say here is that, what first changed my mind about this was my son goes to film school and he came home for a winter break, and he was telling me about how he's using it in the creative process, and how it's not doing his work for him, but it's doing a lot of, like you say, the grunt work.

It'll create an initial image for him that he can then manipulate on film to what he wants.

Dan Holloway: Yeah, exactly. So, it's saving us a lot of time so that we can actually spend our time doing what we might call the high creative value stuff.

Howard Lovy: And that's not to say 10 years from now you push a button, and everything's done for you. I don't know what the future is, but I do know that it's here and we either have to make it work for us or let it pass us by, I suppose.

Dan Holloway: Well, yeah, the way I've seen it put a lot is, if we don't have a positive or a hopeful outlook on it, then we might as well give up, because if we really think it's going to take our place, then what's the point of carrying on?

Writing a Children’s Book Using ChatGPT

Howard Lovy: That's AI when it comes to art, let's go into writing itself. In the news, you reported that someone has written a children's book using ChatGPT.

Dan Holloway: Yeah, so this was really quite interesting, and it got people really upset.

Ammaar Reshi, who is a tech engineer, and I think this is part of what got people upset is that he is a tech engineer and not a writer, he's combined ChatGPT with Midjourney to create a children's book. So, bringing the two, the illustration and the text, together, obviously with his own prompts.

It's caused quite a lot of anger, as I say, I think in part because he's sort of seen as a bit of a tech bro.

Howard Lovy: Do you know any of the specifics as to how he did it? How much did he guide the process along? From what I discovered, ChatGPT will only do what you tell it to do to some extent, and then it goes off in the direction you may not want it to go.

Dan Holloway: Yeah. So, what he did was, he started with a prompt, and then, I mean, I linked the Twitter feed that he's got in an article, he's got a really quite interesting video of the backwards and forwards between him and ChatGPT. So, he asked it for a prompt, it sent back some text, and then there was almost like a backward and forwards communication as he refined it, never taking it offline. So, he didn't actually take the text and then put the finishing touches on himself, but he got ChatGPT to modify it until it was at some point ready. So, the phrase that he uses was, “it was like having a constant brainstorming partner who I could ping pong ideas off.”

Which actually, I like, and he may be a tech engineer but that feels to me like the creative process.

The Legal Implications of Using AI to Write

Howard Lovy: It seems like a children's story would be a good start. From what I discovered, its writing is uninspiring and juvenile if you ask it to just write something on a theme.

Now, if you ask it to mimic your favourite author, then it starts getting a little more creative, but in a way that's obviously derivative. I think it still only produces what you tell it to produce.

Dan Holloway: Yes, absolutely, and this is one of the controversial aspects that artists in particular, and clearly, it's going to be the case with writers too, have been upset about, is essentially when you use it to create something, you then own that thing, but the AI that has produced it has been trained on all these other pieces of work.

So, if you ask it to produce a story in the style of Roald Dahl for example, or a picture in the style of Vang Gogh, there are a lot of those going around, or a picture in the style of Picasso, or more worryingly a living artist, then the AI that produces that will have learned how to do that by training itself on the work of that artist, but that artist then doesn't get any of the benefit, I guess you'd say, of the final commercial product.

Howard Lovy: I'm not sure Van Gogh is very interested in taking any credit for mimicry.

Dan Holloway: No, but people are using their favourite cover artists, or their favourite illustrators, or their favourite manga artists and saying, produce me something in the style of, and these people are saying, well, hang on a minute, it's using my work to produce this image, I'm not getting royalties on it, and they're then able to sell it.

So, that's where it's slightly blurry, and the companies, this is where they are acting like tech companies and they're saying, well, it's a little bit too difficult for us to ask everyone's permission, these works are already out there, they're basically scraping content, if you were to send us in something, then we would stop it using your work, but that's up to you to do it. So, there's a slightly blurry conversation around how this AI is trained.

Howard Lovy: It's very blurry because I'm not sure it's actually scraping content, because it's not taking it word for a word.

Dan Holloway: I can absolutely understand why people feel upset, that they think that they should be getting a license.

When the UK government did its consultation on AI and copyright, one of the things that ALLi's responses was quite clear on was that, if your work is being used for things that are going to then have some sort of commercial element, then the people who generate that content should be paid a license fee for that work. So, that's probably what I'd say on that.

The Potential Benefits of AI Generated Text for Indie Authors

Dan Holloway: But from a creative side, what interests me is, if I were to ask it to produce a work on a particular subject, but in my style. So, if I were to feed it my existing novels, then I could see how it thinks I would write it. So, that would give me a really interesting basis for then polishing and tuning.

Howard Lovy: Right, and I tried that. I used ChatGPT and also Sudowrite. I fed it passages that I had already written, and I asked it to rephrase it, to see if it can come up with a better way of wording it, and it came up with different words and different emphases and different phrases, but I can't say that it was all better. It's just different.

But if you ask it to compose a scene from scratch, and you define the parameters specifically, like, who's in the scene, where they are, what mood the characters are in, it doesn't do too badly, and then you can fill it in from there.

So, I guess where we really get into trouble is if I say, well write this opening scene in the style of an existing author.

Dan Holloway: Yeah, but if that existing author's you, then wouldn't that be interesting?

Howard Lovy: Right. Well, I'm not well known enough for it to know what the style of Howard Lovy is, I have to tell it by feeding it the information.

Dan Holloway: Yeah, that's the problem with being a first-time novelist. If you've already got lots of books out there, and they're all on open access somewhere, then I think that would be a really interesting way of, we talk about producing high volume of content, so that would be a really interesting way of producing high volumes of content that were authentically yours. You could get it to write the 50-60,000 words, and then you would essentially come in at the editorial stage.

Howard Lovy: Right. So, what about you? You're a poet and a writer in different genres, how do you see yourself or your process being enhanced by ChatGPT?

Dan Holloway: I think I'm waiting to be able to, I like the idea of using it in that way, of saying, these are my books, I want to write a book about, I don't know, caving. So, that's something that I might conceivably write a book about, because I like extreme sports, I've written quite a lot about other extreme sports.

So, I want to write a thriller about caving, which is a closed room mystery, which has a happy ever after ending, for example, and please write it with my unique story development and the emotional beats are the same as they would be in a Dan Holloway novel, and then see what it comes up with. I think that would be really interesting.

Howard Lovy: If you're not an experienced caver, it could tell you what experienced cavers go through.

Dan Holloway: It could. It could also tell me as a writer what my ticks are that I might not be aware of, for example.

Howard Lovy: Oh, like the words you repeat often, things like that?

Dan Holloway: Yeah, words I repeat that I might do unintentionally. I was told, I remember the first time I queried to an agent, the thing I was told is I had a really repetitive sentence structure, I guess, subject, verb, object, subject, verb, object. So, ever since that my sentence structure has been all over the place, but it was really helpful to hear that.

So, I imagine seeing it on the page from words I haven't produced, but I'd asked it to produce in my style would be really a good way of bringing that home.

Howard Lovy: There are other AI programs that can help with that, I'm thinking about Fictionary. I signed up for an initial free trial on that, but I haven't really used it.

What ChatGPT does really well is research. I asked it, what did a music studio look like in 1984? And it gave me, well, back in 1984, there would've been a reel-to-reel tape machine, and this is what the control board would've looked like, et cetera.

I guess, in the olden days, I would've gone to a library or asked a music producer what that would have entailed, but now I can ask ChatGPT. Or how do buskers set them up to play music in New York City, and how do they get the permit to play in a street corner? Just very specific things that'll end up details in my story.

Dan Holloway: The kind of things that we are used to as writers, if we get it wrong, then that's the thing the readers will point out to us.

Howard Lovy: Exactly. Right. So, I see that as being very helpful.

How is ChatGPT Affecting Companies Like Google and Microsoft?

Dan Holloway: This of course is what's got Google angry.

Howard Lovy: Right, and you mentioned in one of your columns that the Google has called a ‘code red'. I'm not sure what that means, but some sort of alert; our dominance is being is being challenged.

Dan Holloway: Yeah. Well, from what I gather, it literally means, this is a potential existential threat because obviously Google makes its money from the adverts that go into its search. So, when you search, the top items are adverts, and there are adverts in the sidebars, but if you're not using Google for search, then it's not going to be able to make that ad revenue.

So, if you were to ask it, what did studios in 1984 look like, then it might also show you some ads for, I don't know, synthesizers or audio interchange boxes or whatever, because it might figure that you were really interested in that, or it might show you an advert for Grammarly, which is probably also going to be out of business quite soon. But it's no longer going to be able to do that if you're just going to ChatGPT to find this out instead. So, it's having to do some serious thinking about what this means for its business model.

Howard Lovy: Yeah, and I'm wondering, is Google coming up with its own version? Is it going to buy it out? Will ChatGPT become the next Google, and will all the ads rush to that platform?

Dan Holloway: And that brings us to the story that's coming out in this week's news, which is around Microsoft. Because no, Google isn't, but Microsoft is, is the answer.

So, Microsoft is investing 10 billion into Open AI, which is a lot of money, and it's valuing it at 29 billion. So, that's an eye watering sum for something that's essentially really new technology.

So, they're essentially going to be using it, at the very least in their Office suite. So, as I've said in the column for this week, it's essentially an updated version of the old paperclip that we all fondly remember.

Howard Lovy: Right, exactly, but only this one will be a little smarter than a paperclip.

Dan Holloway: Yeah. So, even writers who think they're going to have nothing to do with it, essentially the technology that's going into ChatGPT is going to be part of Microsoft Word in not very long.

Howard Lovy: Interesting, and you mentioned Grammarly might be in trouble. Grammarly has a specific purpose for many people, but do you think ChatGPT might replace it?

Dan Holloway: I mean, grammar is one of those things that, you were talking about research, grammar is something we can research. What do people like to hear in certain contexts? How should this sentence read if I'm sending it to my manager? How should this sentence read if I'm sending it to my date for the night? How should this sentence read if I'm sending it to my agent?

There's some more nuanced versions of what Grammarly does with its contextual, this is how you should write an email, this is how you should write a formal letter.

Howard Lovy: Before the show, I asked ChatGPT to write an intro to this podcast. Do you want to hear what it came up with?

Dan Holloway: Okay, let's have a listen.

Howard Lovy: All I did was prompt it to write an intro. I gave it our names and what the topic is, and it came up with:

Welcome to the AI Creativity Podcast, where host Dan Holloway and co-host Howard Lovy dive into the exciting world of artificial intelligence and its growing impact on the world of creativity. From AI generated art and music to AI powered writing and filmmaking, we'll explore the latest developments in this rapidly evolving field, and what it means for the future of human creativity.

Dan Holloway: That's pretty impressive.

Howard Lovy: It's not too bad, it got the name of the podcast wrong, but I didn't tell it the name of the podcast, so it made it up.

Dan Holloway: It also really fancies itself.

Howard Lovy: Yes, actually, if I do say so myself.

Before we end this, I should mention that we have a great blog post up on our site, AI for Authors: Practical and Ethical Guidelines, and contributing there's Orna Ross, Joanna Penn, Laurence O'Bryan of BooksGoSocial, Amit Gupta of Sudowrite, and J.D. Lasica of Authors AI, and they all talk about the ethical and legal implications of AI. It's a really good read, we'll link to it in the show notes for this podcast,

Dan Holloway: Super, yeah that's something we haven't gone into, and obviously has been going quite rapidly again, is the legal position around it in various different countries, and what copyright means in the world of machine learning.

Howard Lovy: Well, that might have to be a story for another day, we can talk about this for a long time.

Well, Dan thank you very much for this interesting discussion, and I know that we'll be talking about AI into the future.

Dan Holloway: Brilliant. Thank you very much indeed.

Howard Lovy: Okay. Thank you. I'll talk to you next month.