Research Scientist, Multimodal Foundation Models

NVIDIA has been transforming computer graphics, PC gaming, and accelerated computing for more than 25 years. It’s a unique legacy of innovation that’s fueled by great technology—and amazing people. Today, we’re tapping into the unlimited potential of AI to define the next era of computing. An era in which our GPU acts as the brains of computers, robots, and self-driving cars that can understand the world. Doing what’s never been done before takes vision, innovation, and the world’s best talent. As an NVIDIAN, you’ll be immersed in a diverse, supportive environment where everyone is inspired to do their best work. Come join the team and see how you can make a lasting impact on the world.

NVIDIA's Spatial Intelligence Lab (SIL) is seeking outstanding research scientists with expertise in vision-language models (VLMs), large language models (LLMs), and multimodal foundation models. Our work lies at the intersection of multimodal learning, visual and spatial intelligence, world models, generative modeling, and embodied AI. We develop models that understand and reason about complex visual environments, learn from large-scale video and multimodal data, and generate and manipulate rich representations of the physical world, enabling intelligent agents to perceive, reason, and act in real and simulated environments. You will conduct research on VLMs and multimodal foundation models, including large-scale pre-training and post-training with a focus on advancing 3D grounding, spatial reasoning, video understanding and multimodal generation and interaction with world models and embodied agents. A key direction is moving beyond models that only reason about visual inputs toward 3D-native multimodal models that can generate, predict, and edit new modalities and structured representations of the world, including 3D and dynamic 4D content. You will have the opportunity to publish original research, lead ambitious technical projects, and translate research advances into NVIDIA products. We are looking for someone with exceptional research judgment, strong engineering skills, and a demonstrated interest in connecting foundational research with real-world systems.

What you'll be doing:

• Research, design, and develop novel multimodal foundation models using large-scale image, video, 3D, and language datasets, with a focus on visual, spatial-temporal, and 3D reasoning.
• Develop methods for large-scale post-training of VLMs/LLMs, including instruction tuning, preference learning, reinforcement learning, and related techniques.
• Explore representations and model architectures that enable reasoning about 3D geometry and semantics, motion, objects, agents, and interactions in complex environments.
• Advance VLM capabilities for 3D grounding, spatial reasoning, video understanding and embodied intelligence.
• Explore unified models that connect language understanding, multimodal reasoning, and generative modeling, allowing models to both interpret the physical world and synthesize or transform it.
• Investigate connections between VLMs, world models, generative models, and physical agents, including applications to autonomous vehicles and robotics.
• Publish original research at leading computer vision, machine learning, and robotics venues.
• Collaborate with research and production teams to identify important problems and transfer successful research into NVIDIA products.

What we need to see:

• A Ph.D. in Computer Science (or a related field), or equivalent experience!
• 2+ years of experience
• A strong research background in one or more of the following areas: vision-language models, multimodal foundation models, large language models, multimodal pre-training and/or post-training, video understanding and visual representation learning, 3D/4D perception. spatial reasoning, 3D/4D generative modelling or multimodal generation.
• Experience training, fine-tuning, and evaluating large neural networks and working with large-scale datasets.
• Strong programming skills in Python and familiarity with deep learning frameworks such as PyTorch.
• Experience developing large-scale machine learning systems, including efficient distributed training and experimentation.
• A publication record or demonstrated track record of research in machine learning, computer vision, natural language processing, or robotics.
• Experience leading research or engineering projects and translating research outcomes into products or deployed systems.
• Excellent communication and collaboration skills.

You will also be eligible for equity and benefits.

This posting is for an existing vacancy.

NVIDIA uses AI tools in its recruiting processes.

From the posting on NVIDIA's job site. Read the original before you apply.