arXiv:2409.17263v1 Announce Type: new
Abstract: This study presents a theory-inspired visual narrative generative system that integrates conceptual principles-comic authoring idioms-with generative and language models to enhance the comic creation process. Our system combines human creativity with AI models to support parts of the generative process, providing a collaborative platform for creating comic content. These comic-authoring idioms, derived from prior human-created image sequences, serve as guidelines for crafting and refining storytelling. The system translates these principles into system layers that facilitate comic creation through sequential decision-making, addressing narrative elements such as panel composition, story tension changes, and panel transitions. Key contributions include integrating machine learning models into the human-AI cooperative comic generation process, deploying abstract narrative theories into AI-driven comic creation, and a customizable tool for narrative-driven image sequences. This approach improves narrative elements in generated image sequences and engages human creativity in an AI-generative process of comics. We open-source the code at https://github.com/RimiChen/Collaborative_Comic_Generation.
A Collaborative Approach to Comic Generation
In recent years, there has been a surge in the application of artificial intelligence (AI) in creative fields such as music, literature, and visual arts. One area that has seen significant progress is the generation of visual narratives, specifically comics. This study introduces a theory-inspired visual narrative generative system that combines human creativity with AI models to enhance the comic creation process.
Comic creation is a multi-disciplinary endeavor that involves storytelling, visual design, and sequential decision-making. Traditionally, comic authors rely on their own creativity and manual skills to craft compelling narratives. However, with the advent of AI, there is an opportunity to leverage machine learning models to support and augment the generative process.
The core concept behind this system is the integration of conceptual principles, referred to as comic-authoring idioms, into the generative process. These idioms are derived from existing human-created image sequences and serve as guidelines for crafting and refining storytelling. By translating these principles into system layers, the system facilitates comic creation through sequential decision-making.
One of the key contributions of this study is the integration of machine learning models into the human-AI cooperative comic generation process. By harnessing the power of AI, the system is able to generate image sequences that exhibit improved narrative elements. This collaboration between human and AI empowers creators to explore new possibilities and push the boundaries of comic storytelling.
Furthermore, the deployment of abstract narrative theories into AI-driven comic creation adds another dimension to the generative process. By incorporating principles from narrative theory, such as panel composition, story tension changes, and panel transitions, the system ensures that the generated comics have a coherent and engaging storyline.
Lastly, the authors provide a customizable tool for narrative-driven image sequences, which allows creators to experiment with different narrative structures and visual styles. They have generously open-sourced the code, making it accessible to the wider community and encouraging further exploration and development in this field.
In conclusion, this theory-inspired visual narrative generative system represents a significant step forward in the integration of AI and human creativity. By combining machine learning models with comic-authoring idioms and abstract narrative theories, the system enhances the comic creation process and opens up new possibilities for storytelling. This interdisciplinary approach has the potential to revolutionize the field of visual narratives and inspire future collaborations between humans and AI in creative endeavors.