During his postdoctoral research, Notre Dame electrical engineer Scott Howard wanted to produce more precise images of the activity within a cell by increasing the power of the microscope’s laser.

When he experimented with different power levels, he discovered that his computer was receiving an extra copy of image data.

“When you push the lasers up too high, it would scramble the data because the different parts inside the cell get saturated and can’t absorb any more light,” Howard said. “But I was getting this extra copy that I called a ‘ghost image.’”

So Howard sat down, did some mathematical calculations and realized that the physics predicted this ghost image would happen, and that it would be a higher resolution image. It was a rare moment of pure discovery. He could push the laser power up and down, moving it slightly and then comparing what the cell looks like at different power levels to learn more about what the microscope could see.

“We take the difference between the two images and use an algorithm to see the signals coming from the laser beam’s center point,” Howard said. “You can’t get the laser any smaller, but the data coming out acts as if it’s from a smaller area. It’s super-resolution data in 3D.”

A black and white super resolution image.
The left side shows the cells in a live zebrafish through a conventional microscope, contrasted with a higher-resolution DeSOS version on the right.

He and his collaborators published a paper on the method in 2018. Even better, this method worked so fast it could be done on tissue in a living animal rather than just on a preserved section mounted on a slide. And it could produce time-lapse videos that show changes over periods of time — or what’s known as 4D images.

The discovery was so novel that when Howard demonstrated it at a conference at Indiana University School of Medicine, the participants thought they were watching a simulation – rather than the actual activity occurring inside a living cell.

An image from advanced electron microscopy.

Notre Dame Integrated Imaging Facility

The NDIIF is a research core that makes available an array of sophisticated microscopes and imaging stations.

Visit website

This kind of breakthrough is exactly why the University created the Notre Dame Integrated Imaging Facility (NDIIF), which recently celebrated its 10th anniversary. The NDIIF is a research core that provides sophisticated microscopes and imaging stations so that scientists and engineers can form interdisciplinary collaborations and, guided by expert staff, address the most complex research problems that were previously beyond their limits.

Howard’s new method, which he calls DeSOS, is a prime example of achieving these goals. It combines imaging techniques of blind deconvolution (De), which recovers blurred images, with his new technique, dubbed stepwise optical saturation (SOS), which extends resolution beyond typical saturation limits. The improved resolution has propelled into new realms the research of two primary collaborators, biologists Cody Smith and Siyuan Zhang.

Smith studies the neural networks and spinal cords of zebrafish to learn how nerves grow and regenerate, which can help doctors better understand a range of diseases and injuries related to the human nervous system. Zhang works on brain tumors connected to breast cancer, studying how breast cancer cells adapt to a new environment in the brain, as well as how the immune system responds.

Two people wearing lab coats and eye protection glasses. One sits at a computer and another stands near lab equipment.
Scott Howard, associate professor of electrical engineering (standing), in his Stinson-Remick lab with graduate student Yide Zhang.
Colorful wires and cords.
The complex wiring of a conventional microscope retrofitted with DeSOS in Howard's lab.

Much of their work has been published and made available to other scientists through open-source applications. The results of seeing more clearly the inner workings of a cell can have exponential effects. For instance, the new techniques could help doctors perform organ biopsies without the need for surgery. Other scientists may find their own unique uses.

“We’re working on a whole suite of techniques that work together,” Howard said. “All of these ideas open up new windows into biology that haven’t existed. We put it out there, and science as a whole can take what they need to enhance their research into everything from drug discovery to understanding neuroscience.”

A man poses for a photo in a lab. Lights and reflections shine behind him.
Scott Howard in his electrical engineering lab in Stinson-Remick Hall.

Howard grew up on Long Island and went to Notre Dame to study pre-med physics. But he liked his engineering classes enough to switch majors.

“A cell culture is a good place for isolated studies, but the body is more complex. You have to see how cells behave in their natural environment.” — Scott Howard

“I remember hearing people say you can’t see smaller than a certain size because you can’t squeeze light down small enough to see clearly,” Howard said. “And I remember thinking someone will figure this out.”

He never imagined he would play a part in that effort. Other scientists solved the problem in different ways, including some who used chemistry to change the way molecules respond to light so different parts can be distinguished. Several of these breakthroughs won Nobel Prizes.

But this and other cutting-edge methods had some drawbacks. Chemical dyes can wear out over time and can only distinguish a few elements at a time. Another method, using statistics to find likely locations, took so much time that it was difficult to perform on living (moving) animals. Also, super-resolution microscopes cost about half-a-million dollars and are often too specialized for most research departments to make the investment.

Howard wanted to create super-resolution images with conventional lab microscopes, and he wanted to be able to study living animals rather than just slides. “A cell culture is a good place for isolated studies, but the body is more complex,” Howard said. “You have to see how cells behave in their natural environment.”

A group of people pose next to the Play Like a Champion sign.
The Integrated Imaging Facility team.
A computer monitor displays 3D images of a mouse brain.
A computer screen in Howard's lab shows different 3D images of a mouse brain sample.

In 2016, he wrote up his ideas about using the ghost images to achieve these goals in an application for the National Science Foundation’s CAREER Award, which recognizes and supports junior faculty who exhibit a commitment to stimulating research. He was one of three Notre Dame engineering professors to receive that year’s award, which comes with $500,000 for research.

Howard tested his method and worked with his collaborators on applications in their research. He also created a computer application — along with a tutorial document, video and sample data  — so anyone could see how it works. He worked with the Notre Dame IDEA Center to request patents and to evaluate commercialization, which could help make the technology widely available.

“At first, I though the super-resolution images were the most important thing,” Howard said. “Now we’re discovering that our increased speed is opening a whole new direction of biological science.”

A man poses for a photo in a lab holding a small zebrafish tank.
Cody Smith, the Elizabeth and Michael Gallagher Assistant Professor of Biological Sciences, in the zebrafish lab in the Freimann Life Science Center.

Cody Smith, who studies zebrafish in the Freimann Life Science Center, uses the new techniques to see how spinal cords develop and how neurons form connections between cells. Watching a 24-hour time lapse of nerve filaments reaching out and forming branches helps him understand how these networks form and rebuild. Smith and Howard published their research in March.

“You can’t understand disease until you understand normal.” — Cody Smith

Smith said he has always had a keen interest in glial cells, which act as a glue between other cells and are often overlooked by those studying neurons, which are often characterized as more active. He came to Notre Dame in part because of its well-established reputation and research labs.

He said Notre Dame not having a medical school also frees him to pursue knowledge that isn’t purely medical. “You can’t understand disease until you understand normal,” he said. “There is a lot to be discovered about glial cells, which might play a more significant role in disease and learning than we thought in the past.”

A close up shot of a zebra fish in a tank.
Zebrafish swim in a tank in the Freimann Life Science Center.
Two men in a lab study an image on a computer screen.
Cody Smith analyzes microscopic images in his lab with Evan Nichols, who collaborated on his research as an undergraduate neuroscience major.

The DeSOS technique allowed Smith to see more clearly how neurons connect to the spinal cord of the zebrafish.

“We know that cells send out axons to communicate across the synapses, the tiny gaps between nerve cells,” Smith said. “Snapshots did not make clear what happened, how the axons poke through the cell membranes. But these high-resolution movies allow us to see the formation of what looks like a sword, poking through and making an entrance.”

Understanding how nerve cells respond to damage and how they regenerate and reconnect themselves can be crucial to accident victims or children who are injured during birth. But it could also apply to other processes in the brain, like the start and spread of Alzheimer’s disease.

Smith said the best part of his collaboration with Howard and the NDIIF is that his work is now driven less by hypothesis, and more by figuring out what they are seeing and explaining it. “We don’t know what to expect,” he said.

A man wearing a face mask and gloves inspects a white mouse.
Siyuan Zhang, the Dee Associate Professor of Biological Sciences, works in his lab at the Harper Cancer Research Institute.

Siyuan Zhang studies tumors in mice at his lab in the Harper Cancer Research Institute. His goal is to understand the growth of tumors in the brains of people with breast cancer.

“Speed is very important. Now we can do live time much faster, so we can observe it and apply it in real time.” — Siyuan Zhang

“How does the tumor cell adapt to its new environment, even though the breast and the brain are very different?” Zhang said. “If we can figure out that transformation, we can explore it as a therapeutic opportunity. We can disrupt it.”

Zhang said his research often uses green fluorescents that emit light in reaction to a microscope’s lasers. The color decays at different speeds along a spectrum, but this “live time” process happens too quickly for the old imaging techniques to capture, allowing background “noise” to distort the images.

One man sits at a computer desk while another stands, both observing an image on a computer screen.
Siyuan Zhang (standing) works in his lab at the Harper Cancer Research Institute with Ph.D. student Ian Gulder.
A man wearing gloves holds up lab mouse in a plastic cage.
A laboratory mouse nibbles on some food in Zhang's lab.

“Speed is very important,” Zhang said of Howard’s method in time-lapse videos. “Now we can do live time much faster, so we can observe it and apply it in real time.”

Howard said the last 18 months since publishing the first paper about DeSOS have led to a breakneck pace of new discoveries. He meets weekly with his group, which includes Smith, Zhang and a few graduate and undergraduate students, to discuss what’s next.

“I have to remind everyone that we’re the only people seeing these 4D images,” Howard said. “Now that we’ve developed these tools, let’s get it out there and teach people how to use it.”