Machine Learning
Nvidia Neuralangelo Can Turn 2D Videos into 3D Graphics
Nvidia’s research department recently unveiled an eye-opening new AI model that can transform dull 2D video clips into detailed 3D graphics called Neuralangelo. Nvidia Neuralangelo is based on neural networks and can successfully generate lifelike constructions, buildings, and other realistic imagery based solely on 2D videos. With this research, the company hopes to recreate virtual replicas of different structures including historical monuments, buildings, and sculptures.
Nvidia is forging a solid path through the AI industry in 2023, cementing itself as a global leader in the industry. Recently, it was announced that Nvidia’s AI-driven chipsets generates billions of dollars in gevenue for the company, and now a new AI model powered by neural networks promises advancements in architecture and 3D modeling on a level we haven’t seen since the early days of digital design.
Neuralangelo is quite obviously inspired by the famous Italian artist and sculptor Michelangelo, whose crafty hands were able to sculpt complex works of art from materials such as marble. Similarly, Nvidia Neuralangelo can generate detailed and intricate shapes and textures just based on simple 2D videos.
Nvidia Neuralangelo: 2D to 3D
Developing 3D models from scratch using traditional methods is extremely time-consuming and challenging. However, with this new AI-powered model, designers can generate structures such as “roof shingles, panes of glass and smooth marble” with impeccable details from 2D videos not only faster than traditional non-AI approaches but also more accurately than other AI-powered approaches.
All you need is a smartphone video, and Nvidia Neuralangelo will turn it into accurate renditions of your preference without sacrificing your time, money, and resources. This approach can help with landscape and architectural design, and video gaming for both corporate and personal needs.
“The 3D reconstruction capabilities Neuralangelo offers will be a huge benefit to creators, helping them recreate the real world in the digital world,” Ming-Yu Liu, senior director of research and co-author on the paper said in a statement.
He also added that developers will be able to import detailed objects into virtual environments for 3D modeling and video games starting from small objects like sculptures to massive skyscrapers.
Neural Rendering
Previously used AI models that could transform and reconstruct 3D objects had several issues including redundancy and struggling with homogenous and strong color textures. However, Nvidia Neuralangelo uses instant neural graphics primitives which helps generate intricate details and textures.
A 2D video of a scene or an object that needs to be transformed is fed into the system that selects the frames that can capture several viewpoints. This approach is similar to how different artists focus on objects they are trying to sculpt or paint from different angles.
Once the system determines the camera angle of each frame that needs transforming, Nvidia Neuralangelo transforms it into a rough 3D presentation, after which the model sharpens the details and optimises the entire 3D structure.
Neuralangelo will be presented at the Conference on Computer Vision and Pattern Recognition (CVPR) along with other Nvidia Research projects on June 18-22.
