How to generate realistic videos with Sora from Open AI

Sora is a new AI model from OpenAI that can generate realistic videos from text prompts. According to the company’s blog post, Sora can create videos of up to 60 seconds featuring highly detailed scenes, complex camera motion, and multiple characters with vibrant emotions.

Sora can also use a still image or an existing video as a starting point and extend it with new content based on the user’s instructions. Sora is trained on a large corpus of videos from various sources, and it can produce videos in different styles and genres.

Introducing Sora, our text-to-video model.

Sora can create videos of up to 60 seconds featuring highly detailed scenes, complex camera motion, and multiple characters with vibrant emotions. https://t.co/7j2JN27M3W

Prompt: “Beautiful, snowy… pic.twitter.com/ruTEWn87vf
— OpenAI (@OpenAI) February 15, 2024

Is Sora available yet?

Sora is not publicly available, but OpenAI has opened access to some researchers and video creators who will test the model for potential misuse and ethical issues.

Today, Sora is becoming available to red teamers to assess critical areas for harms or risks. We are also granting access to a number of visual artists, designers, and filmmakers to gain feedback on how to advance the model to be most helpful for creative professionals.

How can Sora change the filmmaking industry?

Sora can change the filmmaking industry by reducing production costs and time, opening the door to unprecedented creative possibilities, and raising concerns about job security, authenticity, and originality.

Sora can potentially revolutionize the filmmaking industry by offering a convenient and accessible AI app for filmmakers. It promises to save time, enhance creativity, and elevate films to the next level. However, there are also concerns raised about its approach to filmmaking, particularly regarding prioritizing performance and camera features. Despite some skepticism, others believe Sora represents a new era in filmmaking, making the process more accessible and empowering creators with its capabilities, such as generating content based on prompts and redefining storytelling.

How accurate is Sora?

The Sora model can interpret the user’s prompt and generate scenes that match the user’s intention and the physical reality.

However, the model is not perfect and has some limitations. It may have difficulty with simulating the physics of a scene, such as how objects interact with each other and change over time. For instance, a cookie that is bitten may not show the bite mark afterward.
The model may also make mistakes with the spatial details of a scene, such as the direction and position of objects and characters. It may also have trouble with describing events that happen in a sequence or follow a specific path, such as a camera movement.

Examples of Sora-generated videos

Prompt: A stylish woman walks down a Tokyo street filled with warm glowing neon and animated city signage. She wears a black leather jacket, a long red dress, and black boots, and carries a black purse. She wears sunglasses and red lipstick. She walks confidently and casually. The street is damp and reflective, creating a mirror effect of the colorful lights. Many pedestrians walk about. (OpenAI)

Prompt: Animated scene features a close-up of a short fluffy monster kneeling beside a melting red candle. The art style is 3D and realistic, with a focus on lighting and texture. The mood of the painting is one of wonder and curiosity, as the monster gazes at the flame with wide eyes and open mouth. Its pose and expression convey a sense of innocence and playfulness, as if it is exploring the world around it for the first time. The use of warm colors and dramatic lighting further enhances the cozy atmosphere of the image.