.png)
<aside>
</aside>
<aside>
</aside>
<aside>
Recently, I independently produced my first animated short film, Behind the Door, which is 3 minutes and 37 seconds long, with a theme inspired by a dream I had.
https://vimeo.com/1060358308/21de05ed2e
In the traditional film and television process, the process of doing an animation can be broken down into the following steps: script - art concept design - sub-script - original drawing --Animation production - Audio (music, dubbing, sound effects) production - Editing After AI intervened in the animation workflow, I used the following tools in each step:
Story: chatGPT, Kimi
Art Concept Design: Midjourney
Scenario Script: chatGPT
Original Drawing: Midjourney, Liblib
Animation: Dreamina, Pika
Music: Udio, Suno
Dubbing: Capcut
Editing: Capcut
The following is a detailed description of the production ideas of each process in turn.
1.Story
I wrote the script for “Behind the Door” myself, and I talked to the AI about it, but since I was inspired this time, I chose to keep my own ideas completely to myself. When I saw the theme was “Dreams”, my first thought was to make a video of the dreams I've had. Luckily, I have a habit of waking up and recording my dreams, and they became my creative source material. The story is inspired by a dream I had (the world behind the dark door), and the story inside the door is a combination of four other dreams I had (the fish pond at home, the ocean of illusions, the Lilliputian bar, and the brain room). Since I dreamed the main one in a pie-in-the-sky, not much of a logical plot, I adapted it mostly to make the plot complete and logical, with a little bit of thematic depth added to reflect character growth. The following is a synopsis of the script I originally wrote, and then made a few changes because of realization and other issues:
There is a very small door on one wall in the town, no stairs appearing out of thin air at a height of one meter, and everyone says don't go in that door. When I was little I heard my parents arguing and I wished I would disappear, so I fled the house in the middle of the argument and carried a ladder into that dark door. Crawling down the darkness, I came to a beautiful garden porch, and at the end of it, I bumped into a cat dressed as a demon king lying on the couch. I woke it up by sneezing because I was allergic to cat hair. The cat took me on a dog ride to the Lilliputian bar for a drink. I went back to it a few times afterward, sneezing each time, and it told me I was weird for being allergic to cats and still coming to it all the time. Across from the couch was a colorful building, and the cat said that every room in there was a person's brain. Some have wide windows and can see the world outside, while others have only narrow slits in their windows and have to look out at the universe through a telescope on the windowsill. I asked which room was my own, and the cat said it wasn't recommended, but I could take you there if you wanted to see it. I pushed the door open and fell into pitch blackness. In the darkness, I heard my mother yelling for me to get up, and my mother, who couldn't stand my father's domestic abuse, took me and moved me out of there in the night. Getting in the car I remembered that I hadn't had a chance to say goodbye to my cat. Years later, I returned to my hometown, but I could no longer find the door. Where that wall used to be, it was turned into an art piano shop. The owner asked me what I needed and I told him I was looking for a cat. I walked out of the store and looked at a street that was completely different from what it had been years ago. The end credits BGM played.
The title The World Behind the Door is actually a pun, and there are a total of two open doors in the film: the first is when the heroine, wanting to escape from reality, trespasses on a door rumored to be off-limits, a door that symbolizes escape. The second is when Mr. Cat persuades the heroine to open the door that represents her own inner self, a door that symbolizes bravery and facing oneself, the opposite of escaping. By opening the first door, the heroine enters a fantasy world that is much better than reality, while the second door brings the heroine back to reality. These two doors reflect the heroine's growth and change. The world behind the door could be either a blissful illusion or an imperfect reality, which door would you be willing to push open?
If you want to use AI to assist in writing a script, you can also try asking questions from the following perspectives:
1. How to make the story more thematic and deep?
2. How to make the story more creative?
3. How to reflect the growth of the characters?
4. What exactly happened from plot A to B?
For each question, have the AI simply list multiple possibilities and then pick the right one for the AI to expand on in detail.
II. Art Concept Design I used Midjourney to generate character drawings for the art concept design, and the resulting heroine and Mr. Cat I really like. Let's take a look at the finished product first: Childhood protagonist:
Mr. Cat:
The main character of adulthood:
Generating a prompt for a character concept design is actually quite simple, the formula: Character sheet + describe the character image + style + suffix parameters The prompt I generated for the Mr. Cat character sheet is as follows: Character sheet, White arrogant cat in black cloak, purple eyes, wearing a crown.Ghibli style, fine brushstrokes, pastel colors, fantasy,clean background --ar 4:3 --niji Break it down -- Characterization: White arrogant cat in black cloak, purple eyes, wearing a crown Style: Ghibli style, fine brushstrokes, pastel colors, fantasy,clean background colours, fantasy,clean background suffix parameter: --ar 4:3 --niji (frame 4:3, use niji model) So how do you keep the main characters of childhood and adulthood looking the same? It is also very simple, Mr. Cheng into the childhood character map, and then through the --cref pad map, in the prompt clear to twenty-year-old girl, midjourney will be in accordance with the pad map looks generated twenty-year-old look.
III. Split-screen scripts I also write my own scripts. If you want to write split-scene scripts through AI, you need to use trained intelligences, you can try a few more split-scene scripts bot on coze, GPTs. split-scene table has the following contents:
1. lens number used to generate one-to-one correspondence with the material, easy to find 2. screen content scenes, roles, actions, transitions, etc.
3. sound music, sound effects, dubbing
4. lens size, long view, medium view, close-up, close-up, etc.
5. lens movement static, advancing, pulling the distance, up and down, left and right movement, etc.
6. duration single lens duration. The role is to count the total length of time, to see if you need to delete or supplement the lens, so as to control the overall rhythm of the story.
In addition I added my own screen progress and video progress to aid in content generation progress management. Below is the script for “ Behind the Door”. The final cut doesn't actually follow the script, there is a lot of improvisation going on. The main purpose of the script is to help organize what material needs to be generated, and then follow the script to execute the generation.
IV. Screen Generation I'm still using midjourney for the original drawings, which I named by shot number to make them easier to find.
Pick a few favorites and give them a zoomed in look:
During the process of creating the original artwork, I encountered several difficulties, and I'll review them one by one:
AI currently has difficulty keeping two characters in the same screen, that is, it is difficult to have both Mr. Cat and the female lead appearing in the screen at the same time, which puts some limitations on my creation. Solution: only a single character can appear in one screen, and show the interaction between the two through editing. As well as discarding some of the previously desired same frame shots.
Maintain scene and character consistency at the same time. In the process of the actual map, I found that the performance of the plot in a scene, the need for the role and the scene at the same time are coherent. Solution: This is a good solution, as long as the scene picture mat as a picture reference, while the character sheet mat as a character reference on the line.
How do I draw the back of a character when there is no back of the character picture? Solution: My solution finally relied on luck - midjourney bugged me with an illogical character's back, which gave me the matting material. I took a screenshot of the back and saved it as a separate file, and then matted out a character back that fit the logic of the scene.
It's this picture, and it wasn't easy to make it.
V. Animation Generation I used instant dreamdreamina to get these stills moving. Most of the shots performed well, and quite a few of them were pumped up to satisfaction in a single pass. Due to the different tools of different long, individual several need small dynamic or need to move left and right shots, I used with dreamina complementary pika. In the video generation, one of the slightly tedious shots is the female lead opening her eyes from sleep. I used dreamina's first and last frames to control stability. The specific approach is:
use midjourney to generate the girl lying on the bed, because the pad of the picture are open eyes, so the generation is also open eyes
2. with liblib will be the girl's eyes selected as a mask partially redrawn to closed eyes 3. the girl's closed eyes as the first frame, open eyes as the last frame, with dreamina to generate the video
V. Music sound effects I've put some thought into the sound design of this movie. Look at my editing project, the picture is only one track, but the audio takes up five tracks:
The dubbing of the short movie was done by Cutting Screen's AI dubbing with subtitles to read aloud. The sound effects are the ones that come with Cutscene. This one-stop workstation really brings a lot of convenience to movie creators. I used a total of three pieces of music, all AI-generated. Two of the main music tracks were created by udio specifically for the movie. The other one is using stock that was previously generated by suno. I divided the movie into five parts according to the music:
Part 1: The suspenseful opening No music, the last sentence “do not go in” with a big reverb, overlaid with a number of transitions, ambient sound effects, the sense of suspense full.
Part 2: Parents fight, the heroine to the world behind the door The mood of this part is very depressing, so the soundtrack is a more depressing piece of music generated by Suno. In the voiceover part, the girl remembers being teased by her classmates through her parents' fight, and the reminiscence part is added with reverb to show that it's a reminiscence and not something that's happening right now. Sound effects are added to match the story: glass breaking, fighting sounds, running sounds, all to make the story more immersive.
Part 3: Entering the World Behind the Door (Before the Colorful Building) From the time the girl gets up and heads for that door until just before the plot of the colorful building, the same music is used, sung from the intro to the end of the main song. As for the chorus, I put it at the end of the clip. By cutting the same piece of music into multiple sections and placing it in different parts of the movie, it makes the whole movie more coherent and echoes the beginning and the end. I must talk about this song, which I call the theme song of the movie. First of all, I used kimi to generate the lyrics because the sequence with the fish, the jellyfish, and the wine was not expressive enough for me to narrate next to it, so I thought I would use music to express it, so that the lyrics would correspond perfectly to the images.
<aside> 🎵
Lyrics: In the realm of dreams where I take flight
Picture: The heroine is on an excursion
Lyrics: Swimming with the fish through the pale moonlight
Picture: Fish swimming in the room
Lyrics: Gentle jellyfish, with their glow so
Picture:Jellyfish swimming in the air
Lyrics: In this slumber, I am in awe, in awe so right
Picture:Bar. Girl asleep on the grass.
</aside>
After the lyrics came out, I threw them to udio to compose, using mainly soundtrack, jazz, female vocal as keywords, and adding some other words to iterate and try. In the end, I chose this one from a dozen or so, because the female vocal sounded loose and soft, and the accompaniment was quiet, which fit the dreamy, sleepy feeling. After the song was finished, I put in a blank shot of a bird flying away to serve as a natural ending to the clip. The music stops and there is only the sound effect of the bird chirping and flying away.
In fact, the sound here I finished the dream traveling part first, before I finished the music for the world just after the girl walks through the door. I actually changed the music for that section several times, and was never particularly happy with it. It wasn't until I expanded the theme song (which Mr. Udio made into the middle section) forward into an intro that a miracle happened. The strings in front of the intro were too heavy and didn't go well with the soothing dreamy female vocals in the back. When I first heard it, I was almost ready to give up, but then I had a flash of insight - the feel of the intro blended perfectly with the images of this part of the show, and since it was the same song that naturally transitioned from the intro to the verses, there was no need to cut the song, and the overall musical performance was more The overall musical performance was much more coherent! I think a good soundtrack is one where you don't feel the presence of the music, and the music and the images are naturally integrated together.
<aside> 🎵
</aside>
Part 4: Colorful Architecture I used a relatively bright piano piece for this part of Colorful Buildings. The most difficult part of this part is the design of the image and music in the mood of the opening of the door. The atmosphere of the story shifts from lighthearted to suspenseful, and I needed to express the heroine's transformation from escaping to facing herself bravely, but the scene was static, and there was only one door, so the only way to tell the story was through sound and special effects.
In order to tell this story, I did these designs:
Part 5: Moving + Returning to Town Here is the second half of the piano piece used, also to make the movie as a whole more coherent and consistent by cutting the same piece of music into multiple parts. To the part of the speech, I stopped the music, here used a doorbell ding-dong sound to transition, one is to tell the girl into the store plot, the second is as a transitional sound effects to let the audience on the music sound of the first stop no perception. Listen to the pure enjoyable version of the piano piece:
<aside> 🎵
</aside>
Part 6: End of the Story The end credits use the chorus of the theme song to achieve the effect of echoing the beginning and the end. The last shot is the mysterious door, which also echoes the beginning, but at this point the mood is already completely different from the suspense at the beginning, and here is where the difference in mood of the same image is expressed through filters and music. At the end of the final music, I added the sound of birds chirping, and the beginning of the short film is the same bird chirping, but also to echo the beginning and end.
VI. Editing For editing, I'll briefly share a few tips I used.
Lastly, I would like to thank Dreamina for this competition. I'd like to thank this contest of Imagine Dreams for letting me start trying to make a short story with AI for the first time. Contests really are the best way to learn. In order to complete a good work, I swiped a lot of award-winning animated shorts (not made in AI) to learn the way others tell stories, and also brushed up on my knowledge of audio-visual language and editing. In the process of practicing, I kept encountering problems and then solving them. Thanks to AI, I've gotten a taste of animation directing, something I couldn't have imagined even a year ago. After sharing so much, I realized that what I shared seems to be more about creative thinking rather than how to use the tool. Because AI is just a tool, I value human ideas more than AI technology. I don't like to use AI to do scripts and album planning, AI can help me quickly complete the art, music material, but fragmented material is not art, the material is carefully arranged to show the results is art. And the connotation of the theme of art, it must be the crystallization of human thought - I hope that no matter what degree of AI development in the future, this point will not change. Finally, it's egg time, put an interesting scrap:
</aside>
<aside>
</aside>
<aside>
</aside>