This video is awe-inspiring on all fronts. Soon, you'll be able to remove objects, add objects, extend clips, and more. You may not need to buy all the props a scene needs, worry about having brand logos in the background of a shot, or freak out about missing some B-roll shots. Imagine telling your client you can extend a clip another 5 seconds even if the footage doesn't exist.
Open AI's Sora is a game-changer. While Adobe's Firefly model, Pika, and Runway ML offer similar functionalities, Sora stands out as the most advanced video model to date. Its unique features enable you to perform tasks that were once unimaginable.
If you are unfamiliar with Sora from OpenAI, take a look at this tech demo, and you'll be able to imagine what you could possibly do with it!
It makes me reminisce on jobs or projects where I wish I had some of these possibilities. Naturally, you would want to avoid using this in a documentary, but if the authenticity of the clips isn't of utmost importance, this can really take storytelling to new heights. You could craft a film like you would a photo in Photoshop. You may not have the budget for props or locations, but you can shoot a solid base and build all the elements by prompting you to create a final vision in no time.
To sum it up, here are some of the things you can expect by looking at the video:
Seamless Object Addition and Removal
One of the standout features is the ability to effortlessly add or modify objects within footage using simple text prompts. Editors can select an area in the footage, write a prompt, and watch as new elements, like the showcased diamonds created by the Firefly model, seamlessly integrate into the scene.
In addition, Adobe streamlines the process of removing unwanted objects from videos. With AI-based smart masking, editors can quickly erase items such as utility boxes, props, crew members, and logos, maintaining focus on the desired content.
Intelligent Scene Extension with Generative AI
Premiere Pro's new Generative Extend feature intelligently adds frames to a clip, proving invaluable when a shot is too short and needs extending. The example demonstrates how the feature preserves the continuity and flow of the video by seamlessly extending a shot of a woman by a few frames.
Sora And Other Models Join Premiere Pro
Adobe is also exploring integrating third-party generative AI models within Premiere Pro. These include Pika and Sora from OpenAI, which assist in generating b-roll and extending scenes through text prompts, and Runway AI, which helps create new video clips and integrates them directly into the editing timeline.
Transparency and Content Credentials
To maintain transparency with AI, Adobe introduced content credentials in Premiere Pro. This feature informs users whether AI was utilized in the creation of media and specifies which model was used, ensuring that editors and viewers are aware of the creative process. Whether you believe it needs to be disclosed is a different topic.
Implications for Videographers and Photographers
Introducing AI-driven features in Adobe Premiere Pro has far-reaching implications for videographers and photographers. These tools have the potential to open up new avenues for creativity!
For videographers, the ability to seamlessly add or remove objects within footage using simple text prompts can streamline editing. That makes it easy to modify scenes without requiring extensive reshoots or complex post-production. The Generative Extend also comes in handy when a shot is too short.
However, it's necessary to consider the ethical implications of using AI-generated content. The introduction of content credentials in Premiere Pro should ensure transparency and allow professionals to make informed decisions about using AI in their work and be upfront about it. Granted, much of that also comes from Adobe's usage licenses and where you can use it based on what model you use to generate content.
Personally, I think it's great that Adobe allows the use of other models beyond Firefly, as different models render different results! Having it all in-house seems shocking, and it's a smart move by Adobe to partner with other models. This collaborative approach can only make these tools better for all of us.
Overall, AI-powered features should enable videographers and editors to push the boundaries of their creativity and deliver high-quality content more efficiently than ever before. As these tools become more widely adopted, they may reshape the industry and redefine what is possible, and we can't wait to see what that could look like! Imagine all the stories that have yet to come to life because of a roadblock that can now be overcome.
