In the Northeastern United States, the Gulf of Maine represents one of the most biologically diverse marine ecosystems on the planet — home to whales, sharks, jellyfish, herring, plankton, and hundreds of other species. But even as this ecosystem supports rich biodiversity, it is undergoing rapid environmental change. The Gulf of Maine is warming faster than 99 percent of the world’s oceans, with consequences that are still unfolding.

A new research initiative developing at MIT Sea Grant, called LOBSTgER — short for Learning Oceanic Bioecological Systems Through Generative Representations — brings together artificial intelligence and underwater photography to document the ocean life left vulnerable to these changes and share them with the public in new visual ways. Co-led by underwater photographer and visiting artist at MIT Sea Grant Keith Ellenbogen and MIT mechanical engineering PhD student Andreas Mentzelopoulos, the project explores how generative AI can expand scientific storytelling by building on field-based photographic data.

Just as the 19th-century camera transformed our ability to document and reveal the natural world — capturing life with unprecedented detail and bringing distant or hidden environments into view — generative AI marks a new frontier in visual storytelling. Like early photography, AI opens a creative and conceptual space, challenging how we define authenticity and how we communicate scientific and artistic perspectives. 

In the LOBSTgER project, generative models are trained exclusively on a curated library of Ellenbogen’s original underwater photographs — each image crafted with artistic intent, technical precision, accurate species identification, and clear geographic context. By building a high-quality dataset grounded in real-world observations, the project ensures that the resulting imagery maintains both visual integrity and ecological relevance. In addition, LOBSTgER’s models are built using custom code developed by Mentzelopoulos to protect the process and outputs from any potential biases from external data or models. LOBSTgER’s generative AI builds upon real photography, expanding the researchers’ visual vocabulary to deepen the public’s connection to the natural world.

At its heart, LOBSTgER operates at the intersection of art, science, and technology. The project draws from the visual language of photography, the observational rigor of marine science, and the computational power of generative AI. By uniting these disciplines, the team is not only developing new ways to visualize ocean life — they are also reimagining how environmental stories can be told. This integrative approach makes LOBSTgER both a research tool and a creative experiment — one that reflects MIT’s long-standing tradition of interdisciplinary innovation.

Underwater photography in New England’s coastal waters is notoriously difficult. Limited visibility, swirling sediment, bubbles, and the unpredictable movement of marine life all pose constant challenges. For the past several years, Ellenbogen has navigated these challenges and is building a comprehensive record of the region’s biodiversity through the project, Space to Sea: Visualizing New England’s Ocean Wilderness. This large dataset of underwater images provides the foundation for training LOBSTgER’s generative AI models. The images span diverse angles, lighting conditions, and animal behaviors, resulting in a visual archive that is both artistically striking and biologically accurate.

LOBSTgER’s custom diffusion models are trained to replicate not only the biodiversity Ellenbogen documents, but also the artistic style he uses to capture it. By learning from thousands of real underwater images, the models internalize fine-grained details such as natural lighting gradients, species-specific coloration, and even the atmospheric texture created by suspended particles and refracted sunlight. The result is imagery that not only appears visually accurate, but also feels immersive and moving.

The models can both generate new, synthetic, but scientifically accurate images unconditionally (i.e., requiring no user input/guidance), and enhance real photographs conditionally (i.e., image-to-image generation). By integrating AI into the photographic workflow, Ellenbogen will be able to use these tools to recover detail in turbid water, adjust lighting to emphasize key subjects, or even simulate scenes that would be nearly impossible to capture in the field. The team also believes this approach may benefit other underwater photographers and image editors facing similar challenges. This hybrid method is designed to accelerate the curation process and enable storytellers to construct a more complete and coherent visual narrative of life beneath the surface.

In one key series, Ellenbogen captured high-resolution images of lion’s mane jellyfish, blue sharks, American lobsters, and ocean sunfish (Mola mola) while free diving in coastal waters. “Getting a high-quality dataset is not easy,” Ellenbogen says. “It requires multiple dives, missed opportunities, and unpredictable conditions. But these challenges are part of what makes underwater documentation both difficult and rewarding.”

Mentzelopoulos has developed original code to train a family of latent diffusion models for LOBSTgER grounded on Ellenbogen’s images. Developing such models requires a high level of technical expertise, and training models from scratch is a complex process demanding hundreds of hours of computation and meticulous hyperparameter tuning.

The project reflects a parallel process: field documentation through photography and model development through iterative training. Ellenbogen works in the field, capturing rare and fleeting encounters with marine animals; Mentzelopoulos works in the lab, translating those moments into machine-learning contexts that can extend and reinterpret the visual language of the ocean.

“The goal isn’t to replace photography,” Mentzelopoulos says. “It’s to build on and complement it — making the invisible visible, and helping people see environmental complexity in a way that resonates both emotionally and intellectually. Our models aim to capture not just biological realism, but the emotional charge that can drive real-world engagement and action.”

LOBSTgER points to a hybrid future that merges direct observation with technological interpretation. The team’s long-term goal is to develop a comprehensive model that can visualize a wide range of species found in the Gulf of Maine and, eventually, apply similar methods to marine ecosystems around the world.

The researchers suggest that photography and generative AI form a continuum, rather than a conflict. Photography captures what is — the texture, light, and animal behavior during actual encounters — while AI extends that vision beyond what is seen, toward what could be understood, inferred, or imagined based on scientific data and artistic vision. Together, they offer a powerful framework for communicating science through image-making.

In a region where ecosystems are changing rapidly, the act of visualizing becomes more than just documentation. It becomes a tool for awareness, engagement, and, ultimately, conservation. LOBSTgER is still in its infancy, and the team looks forward to sharing more discoveries, images, and insights as the project evolves.

Answer from the lead image: The left image was generated using using LOBSTgER’s unconditional models and the right image is real.

For more information, contact Keith Ellenbogen and Andreas Mentzelopoulos.

Modern fighter jets contain hundreds or even thousands of sensors. Some of those sensors collect data every second, others every nanosecond. For the engineering teams building and testing those jets, all those data points are hugely valuable — if they can make sense of them.

Nominal is an advanced software platform made for engineers building complex systems ranging from fighter jets to nuclear reactors, satellites, rockets, and robots. Nominal’s flagship product, Nominal Core, helps teams organize, visualize, and securely share data from tests and operations. The company’s other product, Nominal Connect, helps engineers build custom applications for automating and syncing their hardware systems.

“It’s a very technically challenging problem to take the types of data that our customers are generating and get them into a single place where people can collaborate and get insights,” says Nominal co-founder Jason Hoch ’13. “It’s hard because you’re dealing with a lot of different data sources, and you want to be able to correlate those sources and apply mathematical formulas. We do that automatically.”

Hoch started Nominal with Cameron McCord ’13, SM ’14 and Bryce Strauss after the founders had to work with generic data tools or build their own solutions at places like Lockheed Martin and Anduril. Today, Nominal is working with organizations in aerospace, defense, robotics, manufacturing, and energy to accelerate the development of products critical for applications in U.S. national security and beyond.

“We built Nominal to take the best innovations in software and data technology and tailor them to the workflows that engineers go through when building and testing hardware systems,” McCord says. “We want to be the data and software backbone across all of these types of organizations.”

Accelerating hardware development

Hoch and McCord met during their first week at MIT and joined the same fraternity as undergraduates. Hoch double majored in mathematics and computer science and engineering, and McCord participated in the Navy Reserve Officers’ Training Corps (NROTC) while majoring in physics and nuclear science and engineering.

“MIT let me flex my technical skills, but I was also interested in the broader implications of technology and national security,” McCord says. “It was an interesting balance where I was learning the hardcore engineering skills, but always having a wider aperture to understand how the technology I was learning about was going to impact the world.”

Following MIT, McCord spent eight years in the Navy before working at the defense technology company Anduril, where he was charged with building the software systems to test different products. Hoch also worked at the intelligence and defense-oriented software company Palantir.

McCord met Strauss, who had worked as an engineer at Lockheed Martin, while the two were at Harvard Business School. The eventual co-founders realized they had each struggled with software during complex hardware development projects, and set out to build the tools they wished they’d had.

At the heart of Nominal’s platform is a unified database that can connect and organize hundreds of data sources in real-time. Nominal’s system allows engineers to search through or visualize that information, helping them spot trends, catch critical events, and investigate anomalies — what Nominal’s team describes as learning the rules governing complex systems.

“We’re trying to get answers to engineers so they understand what’s happening and can keep projects moving forward,” says Strauss. “Testing and validating these systems are fundamental bottlenecks for hardware progress. Our platform helps engineers answer questions like, ‘When we made a 30-degree turn at 16,000 feet, what happened to the engine’s temperature, and how does that compare to what happened yesterday?’”

By automating tasks like data stitching and visualization, Nominal’s platform helps accelerate post-test analysis and development processes for complex systems. And because the platform is cloud-hosted, engineers can easily share visualizations and other dynamic assets with members of their team as opposed to making static reports, allowing more people in an organization to interact directly with the data.

From satellites to drones, robots to rockets

Nominal recently announced a $75 million Series B funding round, led by Sequoia Capital, to accelerate their growth.

“We’ll use the funds to accelerate product roadmaps for our existing products, launch new products across the hardware test stack, and more than double our team,” says McCord.

Today, aerospace customers are using Nominal’s platform to monitor their assets in orbit. Manufacturers are using Nominal to make sure their components work as expected before they’re integrated into larger systems. Nuclear fusion companies are using Nominal to understand when their parts might fail due to heat.

“The products we’ve built are transferrable,” Hoch says. “It doesn’t matter if you’re building a nuclear fusion reactor or a satellite, those teams can benefit from the Nominal tool chain.”

Ultimately the founders believe the platform helps create better products by enabling a data-driven, iterative design process more commonly seen in the software development industry.

“The concept of continuous integration and development in software revolutionized the industry 20 years ago. Before that, it was common to build software in large, slow batches – developing for months, then testing and releasing all at once,” Strauss explains. “We’re bringing continuous testing to hardware. It’s about constantly creating that feedback loop to improve performance. It’s a new paradigm for how hardware is built. We’ve seen companies like SpaceX do this well to move faster and outpace the competition. Now, that approach is available to everyone.”