This AI movie camera transforms films into whatever you can imagine

10 July 2024, 03:23 published

[ad_1]

Artificial intelligence to produce videos has become increasingly popular thanks to tools like Sora, Runway, and Pika. But, a new AI-augmented film camera has taken the idea to the next level.

Creative technology agency SpecialGuestX and mixed-media production house 1stAveMachine have debuted the CMR-M1 to tackle that goal. The result is a camera that doesn’t just film the world around it; it enhances it using a generative AI video-to-video model. The digital film is sent to the cloud, where a Stable Diffusion-based model transforms them. Furthermore, the AI can also link with other APIs to further play with what it can do to the film.

“We believe that movie creation is inherently physical. Filmmakers are used to using cameras, lenses, tripods, lighting, and that AI has to be a new tool that enhances creativity,” said Miguel Espada, Co-founder and Executive Creative Technologist of SpecialGuestX. “In the spirit of creative experimentation, we designed a camera that serves as a physical interface to AI models.”

The camera features a wheel on the side of the case, allowing operators to transition from real to AI-generated overlays smoothly. There’s even a slot for a style card containing an NFC chip that connects to the user’s model trained with their images and personalized prompts.

Though it’s not a real-time setup due to latency, models in development like StreamDiffusion could make that a reality. There are currently five different filters through which the film runs, including the jungle-infused “Blooming Nature” and the tuxedos and gold of “Old Money.” You can see an example of “Old Money” below.

Film AI

The CMR-M1 is currently a prototype and not intended for commercial release. It is part of SpecialGuestX’s research into creating physical interfaces for generative AI. However, the design of the CMR-M1 has been developed with scalability in mind, allowing for potential commercial production in the future.

CMR-M1, which stands for the somewhat basic name Camera Model 1, deliberately tries to invoke the dawn of film in its look. The camera has a design reminiscent of the 1924 Cine-Kodak camera, one of the first and most popular portable 16mm cameras.

“Sometimes to imagine what the future might be like you have to prototype it. This is especially the case when it comes to filmmaking, creative technologies, and the creative community,” expressed Aaron Duffy, Founder & Executive Creative Director of SpecialGuest. “The CMR-M1 is a prototype for how hands on creators will use AI rather than sitting at a keyboard.”

Whether CMR M1 will have an impact on the greater world of film is hard to say. More immediate is the question of whether it will open a new lane of competition among AI video model developers like Runway and its new Gen-3 Alpha, OpenAI and its Sora model, and the efforts by Pika and Luma Labs.

The inclusion of open-source models from Stability AI suggests that kind of access may play a significant role in the evolution of AI videos, though what’s on the final reel in the competition is still very much a cliffhanger.