Discoveries in the data: High Performance Computing cluster powers cutting edge research

Big questions, like those inquiring about the mysteries of our universe, often require “big data” to inform potentially out-of-this-world research findings.

Illinois State University’s High Performance Computing (HPC) cluster, launched in 2020 in Julian Hall, provides researchers a supercharged tool to efficiently analyze big data sets.

Assistant Professor of Physics Dr. Matt Caplan frequently uses the HPC for his research on astromaterials—the small yet remarkably dense solids that form inside dead stars many light years away from Earth. “I use big computer simulations to calculate the properties of very tiny bits of matter so that we can understand the properties of big things like the cores of stars, big bangs, and black holes,” Caplan said.

The HPC consists of 30 powerful, interconnected servers called nodes that process massive amounts of data simultaneously to perform complex calculations. What would take a standard computer years to compute can be processed by the HPC in a matter of weeks or months.

“In parallel computing, multiple brains are better than one,” said Dr. Rosie Hauck, executive director of the Office of Advanced Technology Support for Faculty. “All of the nodes are working together, which is why they’re able to do these heavy computational problems in a shorter amount of time.”

For example, Caplan uses an HPC simulation to determine how helium nuclei particles change the radioactive decay rates of thorium in white dwarf stars. He sets parameters—such as the size of the simulation, the temperature, the magnetic field strength, the number of nuclides, and the density of the matter—all from his Moulton Hall office computer.

“And then you just push a button, and it goes,” Caplan said.

HPC nodes process the data to Caplan’s specifications utilizing a research code that he and his astrophysicist colleagues developed over the past several years.

“If you want to do a simulation, what you really have to do is a lot of math—and there’s too much math for you to do by hand,” Caplan said. “It’s calculating where particles are, where they’re going, and how their interactions with every particle around them is going to change how they’re moving.”

A weeklong simulation run through the HPC generates thousands of files containing columns of small white numbers stretched across the black background of Caplan’s office computer.

“It’s sort of like watching the matrix when the numbers are streaming down,” said Craig Jackson, director of Infrastructure Operations and Networking in the Office of Technology Solutions. “But, to Dr. Caplan, the numbers have meaning.”

Caplan plugs the data into a graph for analysis. In his white dwarf simulation, a trendline illustrating the HPC’s computations shows Caplan how close the helium gets to the thorium and how readily it can prevent the thorium from undergoing radioactive decay.

“Oftentimes, it’s just turning 10s of terabytes of data into one number or one line that can go in a paper so that someone else can use it in their calculations,” Caplan said. “The bulk of science is built like a jigsaw puzzle, and everyone is trying to make a piece that will fit with other people’s pieces and let them place more pieces in the grand puzzle.”

Caplan is among more than 10 faculty members now utilizing Illinois State’s HPC. Their research areas include chemistry, economics, geography and geology, information technology, mathematics, and physics. Hauck hopes to expand HPC usership to faculty in disciplines that perform qualitative research as well. 

“The HPC can analyze a lot of text,” Hauck said. “You could run an analysis of all of Shakespeare’s work, for example.”

Charles Edamala, Illinois State’s chief information officer and associate vice president of Technology Solutions, said the HPC could also be used in multidisciplinary efforts to solve societal problems and envisions the HPC growing into a communitywide resource.

“This service opens the door for education scholars to work with fine arts faculty and mathematical researchers to, for example, model student success in low-income neighborhoods,” Edamala said. “It has the capability to develop large-scale epidemic simulations, to be part of a smart city grid, or to create aerodynamic vehicle models. The HPC has the potential to serve McLean County and its business community in diverse and innovative ways.”

Illinois State’s HPC is a necessity for Caplan. Without it, he would have to apply for “node hours” at other institutions—a process that would reduce his time to conduct research.

“The difference between having this machine and not is the difference between being able to do this research and not,” he said. 

Caplan is internationally recognized for studying nuclear pasta—exotic shapes like planar “lasagna” and cylindrical “spaghetti” that nuclei form deep in the crust of neutron stars where matter is a trillion times denser than anything on Earth. Using the HPC and a few external supercomputers, Caplan runs simulations to study the effect of nuclear pasta on observable neutron star properties.

“These are sheets of protons and neutrons that have completely rearranged into something that cannot exist on earth, and we want to know, ‘How strong is this material?’” Caplan said. “So, one of my sets of simulations that I’ve been running is, if you twist it and if you distort it and if you bend this simulation, how does the material respond? That tells you it’s elastic strength and the sort of energy that can be released in a star quake, for example. It comes down to the material properties of the stuff inside them.”

Caplan also studies coulomb crystals created in the outer layers of neutron stars. Immense pressure and density caused by the star’s gravity compresses nuclei into a crystalized structure despite heat exceeding 1 million degrees Fahrenheit. Ian Freeman, an Illinois State senior triple majoring in physics, computational physics, and mathematics, is collaborating with Caplan to create animated videos of the coulomb crystals that illustrate numeric research findings.

“What we’re seeing is that there’s like a little snowflake that sort of melts as we’re trying to figure out, ‘What’s the equilibrium?’” Caplan explained while playing the animation. “We’re simulating at two temperatures, and we’re trying to find the equilibrium between the solid phase and the liquid phase.”

Freeman, a Barry Goldwater Scholar who hopes to become a college professor of astrophysics, first learned about Caplan’s research through his contributions to the YouTube edutainment channel Kurzgesagt – In a Nutshell. Caplan has written more than a dozen episodes, including one about using a theoretical device that would capture the sun’s energy to propel our solar system away from a looming stellar collision.

“Physics has been explored for 500-some years, and supercomputing physics has been explored for about 40 years,” Freeman said. “So, it’s exciting to have the opportunity to be part of that and to have the HPC here at Illinois State. It’s completely revolutionized the way we do science.”

Computer science graduate student Jamie Johnston ’22 was the HPC’s first graduate assistant administrator. After initially helping to inventory and install the HPC’s hardware, Johnston served as the point of contact for system users such as Caplan.

For his final graduate research project, Johnston collaborated with Caplan to identify and classify the moon’s millions of craters through machine learning. Using a convolutional neural network (CNN), Johnston ran a comprehensive analysis of high-resolution digital elevation model images from NASA’s Lunar Reconnaissance Orbiter. Automating the process increased speed and reliability particularly for identifying the moon’s smallest craters.

Johnston has presented his research at the American Physical Society conference and has co-authored a few papers with Caplan. Johnston said his experience with the HPC—particularly the opportunity to interact with faculty—prepared him for his current role as a software engineer at State Farm.

“It’s awesome that I got to get in on the ground floor with the HPC,” Johnston said. He’s subsequently helped train and pass system administrator duties to current HPC graduate assistant George Jacob. “As the HPC grows, I think it’s going to be really cool to be able to look back on and say, ‘I was a part of that.’”

Using Johnston’s analysis, Caplan intends to search for small craters with novel features that could indicate a past lunar collision with dark matter—another potential puzzle piece in his ongoing exploration of the intergalactic frontier.

“When you get something that you didn’t expect at all, that’s when you’ve discovered something,” Caplan said. “And when you’re the first person on the planet to know something true about stars, plasmas, fundamental physics, that’s a cool feeling.”