2013 OSC Research Report

2014 Calendar inside

2013

Ohio Supercomputer Center Annual Research Report

Letter From the Executive Director

For more than 25 years, Ohio Supercomputer Center (OSC) has nurtured its users with a powerful resource for accelerating discovery.

The research featured on these pages provides a small snapshot of some of Ohio’s most innovative, and potentially life-changing, studies.

Nationally, the center continues to support the Extreme Science and Engineering Discovery Environment (XSEDE) as one of the 17 partner organizations. An initiative of the National Science Foundation, XSEDE serves as the foundation for a national cyberinfrastructure ecosystem; it’s a single virtual system that scientists can use to interactively share computing resources, data, and expertise.

The Ohio Supercomputer Center is proud to have three people involved in XSEDE leadership: Steve Gordon, Ph.D., senior education specialist, serves as education lead, senior researcher Karen Tomko, Ph.D., serves as a resource for Ohio researchers as a Campus Champion, and Dave Hudak, Ph.D., program director for cyberinfrastructure and software development, was recently named manager for XSEDE Industry Relations.

Our industrial engagement collaborations are expanding, too. OSC and our industry project partners received a multi-million dollar grant from the Ohio Third Frontier Commission to develop web-based “manufacturing apps.” The initiative, named AweSim, is being led by OSC’s Alan Chalker, Ph.D. You can read more about AweSim on page 6.

As an organization, we continually strive to provide the most powerful and appropriate supercomputing architecture for our researchers. This includes research collaborations, such as those with Intel on the Xeon Phi processing chips, as well as ongoing systems improvements led by our chief systems architect, Doug Johnson.

As a member of the Ohio Technology Consortium, a division of the Ohio Board of Regents, OSC works closely with our consortium partners to provide a seamless foundation for Ohio’s innovation efforts. In particular, the Ohio Academic Resources Network (OARnet) provides a 100 Gigabit-per-second network that ensures researchers have the ability to seamlessly collaborate with others around the world.

We believe that technology, whether supercomputers or networking, should be transparent. The differential should be – and, as highlighted on these pages, definitely is – the people: our staff members and our researchers.

Pankaj Shah

Executive Director Ohio Supercomputer Center & OARnet

Contents Director’s Letter....................................................................................... 2 Ohio Supercomputer Center Propels Ohio’s Economy........................ 4 AweSim: Boosting Manufacturing Competitiveness .............................. 6 OSC Resources: Providing Essential Hardware, Software ...................... 8 Virtual Environments: Turning Results into Realities ........................... 10 Education and Training: Preparing the New Workforce ........................ 12 Biological Sciences .................................................................. 13 Combating Nerve Agents ....................................................... 14 Modeling Bone Nucleation and Growth .................................. 15 Deciphering a Childhood Learning Disability .......................... 16 Discovering Keys to Controlling Malaria ................................. 17 Analyzing Plant Metabolism .................................................. 18 Advanced Materials.................................................................. 19 Understanding Colloidal Suspension ...................................... 20 Evaluating Silica Nanochannels .............................................. 21 Investigating Nematic Vesicles ............................................... 22 Modeling Transition Metal Alloys ........................................... 23 Enhancing Pavement Engineering .......................................... 24 Energy & the Environment...................................................... 25 Converting CO2 to Fuel .......................................................... 26 Increasing Engine Efficiency .................................................. 27 Learning About the Universe ................................................. 28 Controlling Supersonic Airflow ............................................... 29 Seeking Greener Energy Sources ............................................ 30 Research Landscape................................................................. 31 Optimizing 400 MPH Aerodynamics ....................................... 32 Enhancing Surveillance Techniques ........................................ 33 Accelerating Computer Communication ................................. 34 Exploiting Powerful Lasers ..................................................... 35 Analyzing Group Behavior Models ......................................... 36 Industrial Engagement............................................................. 37 Optimizing Plastic Containers ................................................ 38 Dissipating Thermal Energy ................................................... 39 Optimizing Fuel Cell Efficiency .............................................. 40 Improving Industry Collaborations .......................................... 41 Prototyping Fan Designs ........................................................ 42 Contacts................................................................................................. 43

Cover: OSC provides the computational, storage and analysis infrastructure that propels vital academic and industrial research across the state and around the world. The center accelerates discovery for many important investigations, such as the search for better antidotes to nerve agents by Ohio State University computational chemists Christopher Hadad, Thomas Magliery and colleagues at other institutions (see story, page 14).

Ohio Supercomputer Center Propels Ohio’s Economy The Ohio Supercomputer Center strives to propel Ohio’s economy, from academic researchers to industrial partners. To that end, Ohio possesses one of the most potent combinations of statewide cyberinfrastructure elements in the world: high-end supercomputing, research leadership and innovative workforce education programs.

The numbers tell our story best. Last year, 1,068 distinct users – representing academic, healthcare and industrial researchers from across Ohio – took advantage of OSC’s supercomputing and storage resources. In the process, they: • Ran 2.2 million computer jobs • Used 8.5 million computing hours • Leveraged access to OSC for more than $70 million in related research funding Since 2011, the Ohio Supercomputer Center has been a member of the Ohio Technology Consortium, a division of the Ohio Board of Regents. The consortium’s collection of technology and information members, which include OSC, OARnet, OhioLINK, eStudent Services, and the still-in-development Research and Innovation Center, provides Ohioans a foundation unlike that in any other state.

4

OHIO SUPERCOMPUTER CENTER

In particular, OARnet’s 100 Gigabit-per-second (Gbps) network ensures researchers around Ohio have reliable access to OSC systems. New network connections in 2013 also provide new collaboration opportunities. OARnet connected Wright-Patterson Air Force Base (WPAFB) to its high-speed, statewide data network, providing a direct link between the base and the state’s leading universities.

The OARnet-WPAFB connection also provides universities with access to the Defense and Research Engineering Network (DREN), the Department of Defense’s research and engineering computer network. DREN is a robust, high-capacity, low-latency network that provides connectivity to the Defense department’s high-performance computing user sites, resources and other networks across the nation. A similar OARnet connection to NASA’s Glenn Research Center in Cleveland connects researchers directly to 12 different NASA sites throughout the United States.

Connections such as these ensure OSC users have the ability to seamlessly collaborate with others around the nation, and the world. It’s all part of our mission to empower our clients, partner strategically to develop new research and business opportunities and lead Ohio’s knowledge economy.

WE EMPOWER RESEARCHERS

WE PARTNER WITH INDUSTRY

Recognizing HPC resources as an indispensable springboard for innovative breakthroughs, OSC empowers academic and industry researchers to achieve pioneering scientific discoveries in biosciences, advanced materials, energy & the environment and a host of emerging disciplines.

The charts below give a snapshot of the center’s system usage by academic field of science and by academic and commercial researchers.

OSC’s industrial engagement efforts continue with AweSim, our latest program that will design and deploy easy-to-use advanced manufacturing simulation apps. The goal of the public-private partnership is to provide Ohio’s small and mid-sized manufacturers with the tools they need to leverage simulation-driven design.

other 2% University of Cincinnati 1% Ohio University 1% Bowling Green State University 2% Case Western Reserve University 2% University of Akron 4% Nationwide Children’s Hospital 5%

Industrial Use 11%

System Usage

by Institution

The Ohio State University 72%

WE LEAD OHIO’S KNOWLEDGE ECONOMY

OSC directs strategic research activities of vital interest to the State of Ohio, the nation and the world community. Nationally, the center supports the Extreme Science and Engineering Discovery Environment (XSEDE) as one of the 17 partner organizations. An initiative of the National Science Foundation, XSEDE serves as the foundation for a national cyberinfrastructure ecosystem; it’s a single virtual system that scientists can use to interactively share computing resources, data, and expertise. Our in-house research staff specializations include supercomputing, computational science, data management and biomedical applications.

Training/Industrial Partners Research/other 2% Computer and Information Sciences 3% Geosciences 7% Biosciences and Social/Economic Sciences 7%

Engineering 14%

Field of Study

Mathematical and Physical Sciences, Astronomical, 67% Physics, Chemistry and Materials Research

OSC resources include: • Installation of hardware and software applications for use with OSC systems, which deliver a combined peak computing performance of more than 200 Teraflops. Our consulting staff also will work with you to resolve problems, port and optimize codes.

Read on to learn about the accomplishments of our researchers, and the role of OSC in those accomplishments. We think the research showcases some of Ohio’s most innovative studies.

If you agree, drop us a line on one of the sites below and mention #oscresearch.

TWITTER.COM/OSC FACEBOOK.COM/OHIOSUPERCOMPUTERCENTER LINKEDIN.COM/COMPANY/OHIO-SUPERCOMPUTER-CENTER

• Providing advanced computing technology solutions to small and mid-sized companies, allowing for a reduced time to market and developing new and improved products and services.

• Workforce education through innovative programs in computational science, including a baccalaureate minor, an associate concentration and workforce certification.

OSC Propels Ohio's Economy

5

AWESIM PARTNERSHIPS TotalSim USA, a partner in the AweSim program and other collaborations, leverages the power of OSC systems to meet their computational demands.

AweSim: Boosting Manufacturing Competitiveness Small and mid-sized manufacturers are under constant economic pressure to deliver high-quality, low-cost products. Many large manufacturers have embraced simulation-driven design to achieve a higher degree of competitive advantage. Simulationdriven design supplements physical product prototyping with less expensive computer simulations, reducing the time to market while improving quality and cutting costs. Smaller manufacturers are largely missing out on this advantage because they cannot afford to leverage such solutions.

Modern manufacturing simulation applications (or “apps”) integrate essential manufacturing domain knowledge, sophisticated simulation software and powerful computational resources inside a web-based work flow. For example, by clicking a few buttons, the strength of a virtual weld can be predicted through simulations. Similar prototype manufacturing apps already have been developed for companies in fields such as consumer goods, advanced materials and the automotive industry.

Based upon these concepts, the Ohio Supercomputer Center has developed unique technology capabilities and, together with its client partners, a product strategy to help reduce the barriers for entry into this largely untapped market.

With the financial support of Ohio’s Third Frontier Commission and Development Services Agency,

6

OHIO SUPERCOMPUTER CENTER

OSC and its partners have launched AweSim, a program to develop cloud-based manufacturing simulation applications sold through an e-commerce marketplace.

The team chose the name AweSim to reflect the sense of awe that often accompanies the “Aha! moment” when a student, professor, researcher, inventor or engineer unlocks the key to a challenging question or situation and to acknowledge that modeling and simulation are the means by which those insights can most readily be achieved.

The team has an extensive and impressive track record leading up to the development of the AweSim platform. OSC, a member of the Ohio Technology Consortium (OH-TECH), has been at the forefront of the national effort to help industry gain easy and affordable access to advanced modeling and simulation technologies, starting with the 2004 launch of its innovative and widely regarded industrial outreach initiative, Blue Collar Computing.

OSC and client partners Procter & Gamble, Nimbis Services, TotalSim, AltaSim Technologies and Kinetic Vision were heavily involved in the recently concluded National Digital Engineering & Manufacturing Consortium (NDEMC), a pilot program funded by the Department of Commerce’s Economic Development Administration. Led by the Council on Competitiveness, the project provided Midwestern small and mid-sized manufacturers with access to advanced modeling and simulation resources. Multinational semiconductor chip maker Intel is helping the AweSim team coordinate workforce education components.

To date, collaborating AweSim partner organizations have committed to a $3.4 million cost-share, and in June Ohio’s Department of Development Services invested an additional $3 million in AweSim through its Third Frontier Commission Innovation Platform Program.

The objectives of the AweSim program include: • Create a unified, innovative and commercially ready platform consisting of a web-based AweSim App Store, supporting infrastructure and appropriate app development tools.

• Initially populate the AweSim App Store with at least six apps of broad marketability, such as a virtual wind tunnel and a crush test, and lay the infrastructure for rapid development and deployment of future apps.

• Actively pilot and market the developed apps to small and mid-sized manufacturers.

• Create a broad-based training program to equip both novice employees and traditionally trained manufacturing specialists with the enhanced skills needed to master the capabilities and applications of this powerful technology.

The AweSim team anticipates numerous economic benefits for companies that participate in the program, including these outcomes: Developers creating new manufacturing apps will make them available to end-users through Nimbis’ e-commerce App Store, which in turn accesses OSC’s high performance computing infrastructure to run the apps, generating revenue to Nimbis, app developers and OSC. • App developers are expected to further substantial financial investments toward the creation of new apps in future years.

• OSC’s improved tools (App Kit, App Runtime) will provide engineering service providers, software companies and cloud providers the ability to rapidly and efficiently develop apps for new markets.

• Small and mid-sized manufacturers will benefit from a low-cost, accessible technical capability; this will help ensure their global competitiveness by supporting faster and cheaper creation of improved products. The apps are estimated to reduce time to market by 50 percent and decrease product development cost by 30 percent.

• Workers will receive training on these sophisticated new digital tools, ensuring full use of the new capabilities and significantly enhancing their employment opportunities.

Simulation-driven design offered through the AweSim program offers manufacturers a sustainable competitive advantage via a significantly lower cost than alternative technologies, productivity tools and built-up capabilities. Together these differentiate the platform from potential competitors and benefit the value chain of suppliers, large manufacturers and end users.

The Ohio Supercomputer Center recently partnered with the GE Global Research Center to convert the company’s welding simulation methodology into an online app. Based upon an earlier welding portal built for EWI, developers produced a new app, the Submerged Arc Weld Predictor, which provides a 3-D analytical assessment of weld penetration on thick plates. The app development project was funded through the federal (NDEMC) program, a public-private partnership funded by the U.S. Economic Development Administration to support and enhance the use of modeling and simulation among Midwestern small and mid-sized manufacturers.

Alan Chalker, Ph.D. OSC Director of Technology Solutions Director of AweSim (614) 247-8672 • alanc@awesim.org

AweSim

7

OSC Resources: Providing Essential Hardware, Software At the heart of the Ohio Supercomputer Center are the supercomputers, mass storage systems and software applications. Collectively, OSC supercomputers provide a peak computing performance of 214 Teraflops – that’s the equivalent to everyone on earth performing over 30,000 calculations every second.

Last year, more than 1,000 academic and healthcare researchers from across Ohio took advantage of OSC’s supercomputing and storage resources, consuming more than 85 million computing hours.

These users depend on four key systems that are available at OSC. (See system specs, page 9) • HP Intel Xeon Oakley Cluster, which provides clients with a total peak performance of 154 Teraflops of computing power, at 60 percent of the power consumption of previous systems. Oakley also has 4 gigabytes of memory per core, which is more than many national supercomputing centers.

• IBM AMD Opteron Glenn Cluster, which provides clients with a total peak performance of 60 Teraflops.

• Csuri Advanced GPU Environment, which leverages the unique computing properties of the Graphics Processing Units to provide a robust visualization environment. These GPUs are accessed through either the Oakley or Glenn clusters.

8

OHIO SUPERCOMPUTER CENTER

• Mass Storage Environment, containing more than 2 Petabytes of disk storage for a single, centralized point of control.

Additionally, knowing that any computer – supercomputer or otherwise – is only as useful as the software that runs on it, the center provides licenses for more than 30 software applications and access to more than 70 different software packages. Researchers can run software for which they provide the license, as well. OpenFOAM for computational fluid dynamics, LS-DYNA for structural mechanics and Parallel MATLAB for numeric computation and visualization were among the most used software codes this past year.

Beyond providing these shared statewide resources, OSC works to create a user-focused, user-friendly environment. For example, Doug Johnson, senior systems engineer in HPC operations, and his colleagues established the Ruby Development Cluster, in partnership with Intel and The Ohio State University, to test Intel Xeon processor cards. (See the sidebar, page 9.) A team led by David Hudak, Ph.D., program director for cyberinfrastructure and software development, created OnDemand, a web-based application that enables “point-and-click” access to the supercomputers. And, the user services support team regularly interacts with users, from sending system notifications to one-on-one coaching to improving specific codes.

SYSTEM SPECS

“We take great pride in meeting our users needs. We strive to help them be more successful in their research.”

—Doug Johnson Senior Systems Engineer

HP Intel Xeon Oakley Cluster (page 8, detail above) Base Configuration • 692 nodes/33TB memory/8,300+ cores total ››12 cores/node – 48 gigabytes (GB) of memory/node ››Intel Xeon x5650 CPUs (2.67 Ghz) ››HP SL390 G7 nodes • Large memory ››8 dual socket, 12-core nodes (192 GB of memory/node) • Huge memory ›› HP ProLiant DL580 G7 with 4 Intel Xeon E7 8837 processors/32 cores total, with 1TB memory • QDR Infiniband interconnect in all nodes (40 Gigabits/second) Graphic Processor Units (GPU) Configuration • 128 nVidia Tesla M2070 GPUs Ruby Development Cluster (detail above)

››64 nodes (2 GPUs/node)

Researchers at OSC and The Ohio State University have been actively involved in testing the new Intel Xeon Phi hardware, which makes up the bulk of the Ruby Development Cluster.

The innovative components of the cluster include the Intel Xeon Phi accelerator and 200 GB solid-state storage drives per node. The cluster features eight HP SL250 nodes, each with two Intel E5 2670 processors for a total of 16 cores, 128 gigabytes of RAM and 1 Terabyte disk drive.

“The Phi accelerators provide fast processing with a small amount of very fast memory,” said Doug Johnson, senior systems engineer in HPC operations. “This is important because memory performance is frequently what limits application performance on HPC systems.”

The quest for fast memory previously led researchers to GPUs, the computing card traditionally used in video game systems. However, programming for a GPU can be challenging.

“The Phi cards can work as fast as GPUs, but with a different – and in most cases, easier – programming model,” Johnson said. “The Ruby cluster gives us the opportunity to test this new programming model, learn how to manage and support large numbers of the Phi cards, and, ultimately, help us establish a path for building our next supercomputing system.”

IBM AMD Opteron Glenn Cluster (detail above) Base Configuration • 650 nodes/24TB memory/5300 cores ››650 dual socket, quad core 2.5 GHz Shanghai nodes (24GBs memory/node) ››8 quad socket, quad core 2.4 GHz Shanghai nodes (64GBs memory/node) • Infiniband interconnect (20 Gbps) GPU Configuration • 36 GPU-accelerated nodes, connected to 18 nVidia Quadro Plex S4’s for a total of 72 CUDA-enabled devices (2 GPUs/node) • Each Quad Plex S4 includes ››4 Quadro FX 5800 GPUs; 240 cores/GPU ››4 GB memory per card

Hardware and Software

9

Virtual Environments: Turning Results into Realities The Ohio Supercomputer Center’s Virtual Environments and Simulation Group involve an interdisciplinary team of research scientists, computer scientists and clinicians.

The team, which includes colleagues who have been working together for more than two decades, applies high performance computing and advanced interface technology to virtually explore complex computational data.

As advanced simulations integrate progressively larger computational data sets from multiple sources, staff members of the Ohio Supercomputer Center create intuitive methodologies to integrate these vast caches of multisensory data into a single coherent simulation that can facilitate a researchers explorations and interactions.

While these types of virtual environments were once seen only as a unique extension of gaming technology, today virtual environments and simulations are considered essential tools for competitiveness, from healthcare to education to manufacturing.

“The diversity of these projects shows that virtualization and simulations are becoming more commonplace,” said Don Stredney, director of the Interface Lab and senior research scientist, biomedical applications. “Not only have we been able to promote the use of interactive 3-D visualizations, we’ve developed ways to show that developments can often be extended to another, related area.”

The following examples illustrate the wide array of funded modeling and simulation projects OSC supports: • Geoscience students with mobility impairments can explore the geological structures of Mammoth Cave through an interactive virtual interface created by experts at OSC. (See story, page 11.)

10

OHIO SUPERCOMPUTER CENTER

• The National Institute for Occupational Safety and Health is in the process of funding a project to create virtual training environments for home health care providers. With more providers going into the field, it’s critical that they be able to identify household risks such as the presence of mold, dangerous use of equipment, or other threats to the health of either patients, their family, or the health care providers themselves. The effort is to reduce the need to use expensive theatrical models that are costly to maintain and that provide new methods to establish national standards for use in training.

• OSC’s Virtual Temporal Bone Project (left), developed in partnership with Nationwide Children’s Hospital – Columbus and OSU’s Department of Otolaryngology, allows future surgeons to practice delicate drilling techniques on a computerbased teaching system.

More than 10 remote institutions throughout the country are using the device, which now includes improved realism and automated assessments of the students’ performance.

• The Ohio State University Driving Simulation Laboratory (top) supports the design and implementation of vehicle instrument panels and in-vehicle information systems that can be safely used by drivers. A unique university and industrial partnership, the lab was established through collaborations with Honda R&D Americas, oversight from Ohio State’s Office of Research and technical expertise from OSC.

GEOSCIENCES: A MAMMOTH PROJECT

A virtual, former slave-turned-cave guide is partnering with visualization experts at OSC, a geoscience researcher at Georgia State University (GSU) and others to develop an interactive, simulated field trip through parts of Kentucky’s famous Mammoth Cave.

“We have developed a prototypical visual environment of a geological field study,” explained Christopher Atchison, Ph.D., a former graduate research assistant at OSC and now an assistant professor of geoscience education at GSU. “This project seeks to develop and test the efficacy of a simulated representation of cave and karst formations for use in geoscience education, with the intent of increasing interest and to support participation by students with mobility impairments.”

To help students find their way through the cave, Ohio State University graduate student Nikki Lemon created a virtual guide. The avatar was modeled on Materson “Mat” Bransford, a slave who led tours of the cave through the early-to-mid 1800s.

“The early guides provided the foundation of what we know about Mammoth Cave,” said Atchison. “Other than the Native Americans from hundreds of years earlier, the slave guides were the earliest explorers, who began constructing the first maps of the cave based on their personal experiences.”

Officials at Mammoth Cave National Park have expressed an interest in having the group design and install a version of the application at their visitor’s center.

“We have sought to integrate additional media to enrich the educational goals and impart a historical perspective to the project,” said Don Stredney, director of the OSC’s Interface Laboratory. “We’ve created the avatar with historical accuracy and provide visitors with an engaging and unique educational experience through virtual technologies.”

In addition to structural data, the team gathered high-precision data through laser remote-sensing technology, called LIDAR (Light Detection And Ranging), and high-resolution digital photography.

All the data was then integrated into a virtual environment.

The two-year National Science Foundation project is titled “Expanding Geoscience Diversity Through Simulated Field Environments for Students with Physical Disabilities.” The project is a collaboration of OSC (Stredney, Thomas Kerwin, Ph.D., Brad Hittle), GSU (Atchison), the Mammoth Cave International Center for Science and Learning (Rick Toomey, Ph.D.), Ohio State’s Advanced Computing Center for the Arts and Design (Lemon, Alan Price), Ohio State’s School of Teaching and Learning in the College of Education (Karen Irving, Ph.D.), the National Cave and Karst Research Institute, the Cave Research Foundation, Ohio’s STEM Ability Alliance and the International Association for Geoscience Diversity.

To read more about the project, visit osc.edu/research/Biomed/projects/OEDG.

Above: A virtual guide is visible in this synthetic view of Mammoth Cave, built for a project to advance geoscience education for students with disabilities.

Virtual Environments

11

Education and Training: Preparing the New Workforce OSC has earned a national reputation for exceptional training and education programs.

“We’ve created a pipeline to build, and sustain, a computational science workforce,” said Pankaj Shah, executive director, OSC and OARnet. “Our programs include undergraduate education and in-career certification programs, supplemented by outreach to high school and middle school age students, internships and user training.

A new program, to be funded by BP, creates an opportunity for a student to work in OSC’s HPC systems group and with the BP HPC team in Houston. Additionally, this past summer, OSC hosted two research interns funded by the National Science Foundation, and 11 summer interns, in areas ranging from HPC training to business analysis, sponsored by the Ohio Board of Regents.

RALPH REGULA SCHOOL OF COMPUTATIONAL SCIENCE

To encourage interest in the Science, Technology, Engineering and Mathematics (STEM) fields, OSC has offered summer programs for nearly 25 years.

Each year, Ohio’s brightest freshman, sophomore and juniors participate in OSC’s Summer Institute, a two-week summer camp where participants experience high performance computing and network firsthand. Likewise, the Young Women’s Summer Institute helps middle-school girls develop an interest in computer, math, science and engineering.

The center’s virtual Ralph Regula School of Computational Science coordinates computational science education, which is the use of computer modeling and simulation to solve complex business, technical and academic research problems. Students may take part in the computational science minor or associate degree program offered at eight Ohio colleges and universities; the stackable workforce certification program provides workers with an opportunity to learn computer modeling and the underlying mathematics and computer programming skills.

STUDENT INTERNSHIPS

More than 30 undergraduate and graduate students gain real-world experience through a variety of internships at OSC and its partner organizations in the Ohio Technology Consortium.

12

OHIO SUPERCOMPUTER CENTER

SUMMER PROGRAMS

USER TRAINING

OSC’s faculty and student researchers can take advantage of ongoing training for scientific computing workshops, one-on-one vendors and web-based portals, taught by the center’s instructors or in collaboration with the Extreme Science and Engineering Discovery Environment (XSEDE).

Biological Sciences

Ohio’s bioscience researchers are leveraging the resources of the Ohio Supercomputer Center to gather and analyze massive amounts of genetic, molecular and environmental data to better understand human physiology, individualize diagnoses and treat diseases. The research stories on the following pages illustrate a small sampling of these efforts.

Combating Nerve Agents Hadad models the structure of potential antidotes to deadly nerve agents At the Ohio State Center for Catalytic Bioscavenger Medical Defense Research II, chemists Thomas J. Magliery and Christopher M. Hadad lead a team that employs sophisticated methods of protein engineering, high-throughput screening and computational chemistry. Their goal: harness the body’s own defenses to counteract nerve agents and create new types of antidotes for exposure to pesticides and other poisons.

Nerve agents are chemicals that attack the nervous system, causing paralysis and seizures and – ultimately – killing the victim through asphyxiation. They do so by bonding with the enzyme acetylcholinesterase, thereby destroying its activity so that chemical messages from the brain to the rest of the body are transmitted without regulation.

Once attached to the enzyme, nerve agents can’t be removed. So the researchers are focusing on ways to stop the deadly chemicals before they can attach in the first place.

“Fortunately, there are enzymes already in human blood that can deactivate these agents,” said Magliery, co-leader of the Ohio State center.

“We just have to engineer them to be more efficient, and we have to be able to produce and formulate them as drugs.”

Hadad, who leads an effort to model the chemical structure of candidate enzymes on the powerful parallel supercomputer systems at OSC, described one of the main challenges. “In nature, each enzyme generally has only one function

Above and cover: Using computational chemistry to help develop new antidotes for nerve agents, Christopher Hadad leveraged Ohio Supercomputer Center resources to create a molecular dynamics model of paraoxonase bound to a ligand.

– one thing that it does very well,” he said. “But we need an enzyme that will deactivate many different nerve agents. We need one molecule that can do it all.”

The ideal enzyme would remain active for days or weeks at a time, pulling toxic agents from the body over and over again. It could be administered as an antidote immediately after an attack, or as an inoculation against future attacks.

Soldiers and first responders are among the likely recipients of such a preventive dose, but so are people whose jobs regularly expose them to nerve agents, such as pesticides. Household pesticides pose the same dangers, so an enzymatic drug could save lives in poison control centers as well.

The Center, funded by a $7.5 million award from the National Institutes of Health, is a collaboration between The Ohio State University, the U.S. Army Medical Research Institute of Chemical Defense at Aberdeen Proving Ground, Md., and the Weizmann Institute of Science in Israel.

PROJECT LEAD: Thomas Magliery and Christopher Hadad, The Ohio State University RESEARCH TITLE: Computational modeling of human paraoxonase activity FUNDING SOURCE: National Institutes of Health WEBSITE: https://hadad.group.chemistry.ohio-state.edu

14

OHIO SUPERCOMPUTER CENTER

Modeling Bone Nucleation and Growth Sahai uses computational methodology to study bone mineralization process By investigating the mechanisms of bone formation, researchers at the University of Akron may help develop treatments for bone-related diseases such as osteomalacia, more commonly known as rickets, and osteogenesis imperfecta, a genetic disorder in which bones break easily.

Osteomalacia is not the same as osteoporosis, another bone disorder that can also lead to bone fractures. Osteomalacia results from a defect in the bone-building process, while osteoporosis develops because of a weakening of previously constructed bone.

Up to 50 percent of bone is made of a modified form of the mineral hydroxyapatite; the highly ordered structure includes insoluble protein, collagen, and is associated with other soluble proteins and small molecules such as citrate. The nucleation and growth of hydroxyapatite mineral crystals play an important role in controlling the physical properties of bone, and its source of calcium.

“My research group is interested the fundamental mechanisms of bone biomineralization from the molecular- to the micron-scale,” said Nita Sahai, an Ohio research scholar and professor of polymer science at the University of Akron.

The mechanisms for bone mineralization are not clearly known so far; a major limitation to progress in this research area is the inability to sufficiently sample the bone’s nucleation phenomena at the Angstrom (0.1 nanometer) to nanometer length-scale and microsecond

Above: A series of light and dark bands is observed across the axis of the fibril when collagen molecules are stained with metal ions and then viewed in the electron microscope. The bands are designated sequentially c2– c3, as shown.

time-scale. This hurdle exists because regular molecular dynamics simulations cannot accurately sample the calcium and phosphate ion distributions in the nucleating cluster.

To overcome these limitations, Sahai and her team are using the Ohio Supercomputer Center to calculate the ion behavior in the presence of collagen fibril by using Hamiltonian ReplicaExchange Molecular Dynamics. (See illustration above.) These advanced computer simulations allow them to determine the mineralization mechanisms across a wide range of length-scales and time-scales, approaching 100 nanoseconds. This compares drastically to the current standard, which is one to a few nanoseconds. Experimental results obtained in the laboratory provide benchmarks for the computational results.

“By analyzing the computational results, we can establish detailed mechanisms for how bone forms,” Sahai said. “Not only does this represent a major advancement in the methodology for modeling biomineralization mechanisms, it will contribute to scientists designing drugs for treating debilitating bone diseases.”

PROJECT LEAD: Nita Sahai, University of Akron RESEARCH TITLE: Computational approach to bone biomineralization mechanisms: molecular-to-micron-scale hierarchical structure FUNDING SOURCE: University of Akron WEBSITE: http://www.uakron.edu/sahai-group

Biological Sciences

15

Aa @ A

a

A @

a a

a

A a a

A @ @

a

@ A @

A A Marker Discovery

Linkage

Association

RT-PCR

Personalized Medicine

Left: Christopher Bartlett and his team at Nationwide Children’s Hospital are using the Ohio Supercomputer Center to better understand genetic links between families with multiple people affected by specific language impairment.

Deciphering a Childhood Learning Disability Bartlett team seeks genetic coupling that causes specific language impairment As one of the most common childhood learning disabilities, specific language impairment – a delay in mastering language skills, despite normal hearing, education and intelligence – affects about 5 to 7 percent of all kindergartners.

The exact cause of specific language impairment, or SLI, is unknown, but recent discoveries suggest it has a strong genetic link.

Christopher Bartlett, an assistant professor at Nationwide Children’s Battelle Center for Mathematical Medicine and The Ohio State University, and his team, including Stephen Petrill from Ohio State’s Department of Psychology, have made significant strides in understanding heredity’s role.

“By using a complex statistical methodology, called posterior probability of linkage, we examined the DNA of families with multiple people affected

by SLI,” Bartlett said. “This type of analysis enables us to look at how two genes interact and many other possible genetic architectures.”

To date, the research team has made four important contributions to this research space: • They found strong evidence of a genetic variation associated with SLI on the human chromosome 13q21, and characterized how it affects language, reading or both.

• To examine the hypothesis that SLI and autism share genetic causes, they performed the first study that required both disorders be present in each family they studied.

• Bartlett’s team furthered the development of the underlying computational and statistical genetic methods for disease gene discovery.

• They discovered a modifier gene effect that implicated working memory in developmental language impairments.

Bartlett and his team are extending these successes through genome scans collected from new families. Additionally, they are developing and testing the multivariate statistical approaches to further understand the relationship between implicated genes and language ability – and will again be using the Ohio Supercomputer Center for the computationally intensive calculations. In fact, during the current two-year study, they will use 250,000 resource units – an aggregate measure of the use of CPU, memory, and file storage – to examine about 15,000 gene expression levels in 1,083 people.

“The memory and multicore processors available at the Ohio Supercomputer Center are a must,” Bartlett said, “as the scans typically involve millions of likelihood calculations at thousands of genetic positions across the genome. It would be too slow and, therefore, impossible to conduct this research without supercomputers.”

The goal is to translate this basic science into clinical applications, Bartlett added, in the hopes of earlier identification in patients, earlier intervention, and, ultimately, better outcomes for children affected by this disorder.

PROJECT LEAD: Christopher Bartlett, The Research Institute at Nationwide Children’s Hospital & The Ohio State University RESEARCH TITLE: Childhood language impairment and gene expression in the brain FUNDING SOURCE: National Institutes of Health WEBSITE: http://mathmed.org/#Bartlett_Research_Group

16

OHIO SUPERCOMPUTER CENTER

Discovering Keys to Controlling Malaria Serre investigates genetic diversity as key to drug resistance of Plasmodium vivax Malaria affected 219 million people around the globe in 2010, according to the World Health Organization. This life-threatening disease, caused by plasmodium parasites that are transmitted to people through the bites of infected mosquitoes, killed about 660,000 people in 2010 – mostly African children under the age of five.

David Serre, assistant staff, Genomic Medical Institute, Cleveland Clinic, is doing his part to understand how the genetic diversity of one particular parasite species, Plasmodium vivax, enables the species to develop resistance to treatment drugs. P. vivax is the most frequent cause of recurring malaria and can lead to severe disease and sometimes death.

“We know that the parasites’ genetic diversity affects their ability to resist eradication and acquire and develop new strategies for invasion,” Serre said. “However, little research has been done to understand the genetic diversity of P. vivax, in part because it cannot be propagated in continuous in vitro culture. This limits our understanding of the parasite’s biology.”

As an alternative to studying P. vivax in vitro, Serre and his colleagues used high-throughput sequencing to analyze the entire genomes of five P. vivax isolates. They were collected from the monkey-adapted Belem strain and from blood samples of infected patients in Madagascar, where P. vivax has invaded a population previously thought to be protected against this parasite, and Cambodia, where drug resistance is a growing problem.

“These analyses will provide valuable knowledge regarding genes potentially involved in the mechanisms underlying drug resistance and the biological mechanisms of red blood cell invasion.”

Their results identified more than 80,000 Single Nucleotide Polymorphisms, or DNA sequence variations, distributed throughout the genome, and revealed that each patient was infected with multiple P. vivax strains.

With these enormous data sets in hand, Serre and his colleagues now are using the Ohio Supercomputer Center to conduct rigorous population genetic analyses to better understand the history and organization of the P. vivax population. These studies notably include searching the genome for patterns of genetic diversity consistent with the effects of natural selection and identifying the genetic basis of disease-related traits by association.

“These analyses will provide valuable knowledge regarding genes potentially involved in the mechanisms underlying drug resistance and the biological mechanisms of red blood cell invasion,” Serre said. “Ultimately, the results from this effort will provide crucial information for understanding the spread of drug resistance phenotypes and help design more efficient malaria control programs.”

PROJECT LEAD: David Serre, Assistant Staff, Genomic Medical Institute, Cleveland Clinic RESEARCH TITLE: Investigation of the diversity, population and evolutionary history of Plasmodium vivax FUNDING SOURCE: National Institute for Allergy and Infectious Diseases (NIAID – NIH) WEBSITE: http://bit.ly/GSq61Y

Biological Sciences

17

This illustration, developed by Erich Grotewold at The Ohio State University, represents the distribution of binding sites for several transcription factors (TFs 1- 6) in the genome of maize.

Analyzing Plant Metabolism Grotewold, colleagues dissect gene regulatory networks in maize, cereals, crops Erich Grotewold, a professor of molecular genetics and horticulture and crop science at The Ohio State University, is leveraging the resources of the Ohio Supercomputer Center as part of his studies to address fundamentally important questions in plant research.

He and project co-principle investigators Andrea Doseff, an Ohio State Medical Center and Department of Molecular Genetics associate professor, and John Gray, an associate professor at the University of Toledo, are the recipients of a $4.23 million grant from the National Science Foundation Plant Genome Program to study “Systems Approaches to Identify Gene Regulatory Networks in the Grasses.”

“Establishing gene regulatory networks and linking system components to agronomic traits is an important emerging theme in plant systems biology,” Grotewold said. “This is the first concerted effort to comprehensively dissect the gene regulatory networks that target the metabolism of phenolic compounds, found in maize, other cereal crops and commonly-consumed vegetables.”

These compounds include phenylpropanoids, lignins, flavonoids and hydroxamic acids, all of fundamental agricultural importance. For example, lignin is a complex plant polymer important in biofuel considerations, which is gaining significance as a feedstock for the generation of biochemicals and biomaterials.

As a component of this project, Grotewold is integrating genetics, molecular biology, biochemistry, mathematics, and bioinformatics for the comparative transcriptional genomics of phenolic genes across grass crops.

Specifically, he’s combining the discovery of novel transcription factors that control maize phenolic accumulation with the identification of their interaction partners and of direct target genes for individual and combinations of transcription factors. In molecular biology and genetics, a transcription factor is a protein that binds to specific DNA sequences, thereby controlling the flow of genetic information from DNA to messenger RNA. “Some of the bioinformatics approaches involve alignment of hundreds of millions of biological sequences to reference genomes, as the first step in the analysis pipeline towards the discovery of the binding regions for often novel transcription factors,” said Grotewold. “Similarly, simulation and modeling approaches will be involved in prediction and inference of gene regulatory networks, which will be validated experimentally. Supercomputers are paramount for this complexity of analysis.”

These findings will accelerate the study of the regulation of other important metabolic and developmental pathways in maize and other grasses.

Results are being made widely available through GRASSIUS (grassius.org), a web-accessible knowledge base hosted by the Grotewold lab.

PROJECT LEAD: Erich Grotewold, The Ohio State University RESEARCH TITLE: Gene regulatory networks that target the metabolism of phenolic compounds in maize FUNDING SOURCE: National Science Foundation WEBSITE: http://eglab.osu.edu/home

18

OHIO SUPERCOMPUTER CENTER

Advanced Materials

Ohio researchers are conducting groundbreaking studies of various advanced materials. The creation and testing of computational models through Ohio Supercomputer Center systems continues to set the bar high for materials science research in Ohio.

The following pages are just a few of such projects supported by the Center.

Understanding Colloidal Suspension Maia research team analyzing intermediates found in food, cosmetics, drugs Colloidal suspension is the term for a substance that is microscopically dispersed throughout another substance and is found in many every day products – food, cosmetics, drugs. An intermediate between a true solution and a suspension, particles in a colloidal suspension are small, cannot be seen by the naked eye and can easily pass through filter paper, yet they also are large enough to be blocked by parchment paper or animal membrane.

Colloidal suspensions exhibit a transition from a thinning behavior to a thickening behavior as the rate of shearing forces increase. Despite all the experimental and computational studies, an understanding of the structure of suspensions in different flow regimes remains controversial.

A research team lead by João Maia, Ph.D., employed a dissipative particle dynamics (DPD) model on Ohio Supercomputer Center systems to conduct a comprehensive study of the flow and structure behaviors of monodisperse (having particles of similar size) and bimodal (having particles of two sizes) suspensions over a wide range of shear rates.

“Computational studies in general have successfully predicted the local dynamics of colloidal particles, but have been restricted to small-scale systems and failed in explaining large time and/or length scales,” said Maia, director of the Center for Advanced Polymer Processing at Case Western Reserve University. “Similarly, most numerical studies on suspensions have focused

Above: A research team led by Joao Maia of Case Western Reserve University created simulations of colloidal suspensions depicting particles contributing to hydroclusters (shown in red), which play a crucial role in the substance’s shear thickening behavior.

on either near-equilibrium flow conditions or on diluted suspensions, because of the difficulties in dealing with multi-body hydrodynamic interactions and their crucial role in such systems.”

Since the original DPD model at times does not account for the lubrication forces that define the fluid behavior, especially in high-shear rates, the team developed a modified DPD model that defined the suspended particles as a hard core covered by a soft shell, which more adequately accounted for the lubrication factors.

“The interplay between flow and structure indicates that hydroclusters are formed in the shear-thickening regime, whereas interparticle interaction is responsible for the shear-thinning response at low stresses,” said Maia. “The effect of particle size, ratio and combination in bimodal systems have also been investigated and quantitative agreement with existing experimental data was found.”

As a result, it was possible for Maia’s group to perform, for the first time, a comprehensive study on different aspects of the bimodal dispersions and correlate the macroscopic behavior with the microstructure in different flow regimes.

PROJECT LEAD: João Maia, Case Western Reserve University RESEARCH TITLE: Bridging the gap between microstructure and macroscopic behavior of monodisperse and bimodal colloidal suspensions FUNDING SOURCE: National Science Foundation WEBSITE: http://engineering.case.edu/centers/capp/director

20

OHIO SUPERCOMPUTER CENTER

Evaluating Silica Nanochannels Zambrano studies mechanisms to control the flow of fluids in microdevices Microdevices, such as Labs-On-a-Chip (LOC) systems, are used for biomolecular detection and custom chemical synthesis, among other applications. Over the last decade, LOC systems have evolved from a single channel to systems capable of integrating thousands of reaction vessels, conduits and valves.

As the miniaturization process of LOC devices is reaching the molecular level, nanofluidics, the study of flows in and around structures with at least one dimension between 1-100 nanometers, has become of broader interest. The potential to design and fabricate nanoscale LOC devices requires the rapid and inexpensive handling of minute samples confined in nanoconduits. At the nano-scale, the large surface-to-volume ratio results in deviations from the macroscale fluid behavior, the molecular size matters, the hydration effects are significant and electrostatic and other forces can determine the system behavior.

As an electrolyte solution is confined inside a silica channel, the wall surface becomes charged and will in turn be screened by counterions in a thin layer known as the electric double layer (EDL). If an axial electric field is applied, the motion of the counterions in the EDL will drag the solvent in the direction of the counterion migration, a phenomenon known as electroosmosis. Novel LOC applications employ electroosmotic transport as the driving mechanism to conduct fluids, because electroosmosis generates a uniform bulk flow,

Above: A snapshot from simulations developed by Harvey Zambrano of The Ohio State University shows the amorphous silica nanochannel (red and yellow) confining a stream (red and white) of electrolyte-water solution.

which gives higher flow rates and less dispersion than with pressure-driven flow.

Leveraging the resources of the Ohio Supercomputer Center, Zambrano, Ph.D., a postdoctoral researcher at the Computational Micro-Nanofluidics Lab and at the Microsystems and Nanosystems Engineering Lab at The Ohio State University, is evaluating two novel mechanisms to control electroosmotic transport through nanochannels. Zambrano is conducting massively parallel Non-Equilibrium Molecular Dynamics (NEMD) simulations of an electrolytewater solution. The first mechanism adds traces of divalent ions to an electrolyte solution confined in a nanochannels. The second modifies the EDL by implementing on the channel walls surface counter-charged patches.

“Our objective is to determine if surface-charged patches or traces of divalent ions can be used to control electroosmotic flows in nanodevices,” said Zambrano. “To that end, we’re performing NEMD simulations of 100 nanoseconds (one billionth of a second) on sets of all atoms within a channel, containing a different electrolyte concentration and using a time step of 2 femtoseconds (one quadrillionth of a second). The channel is 34.76 nanometers long and confines 20,500 water molecules.”

PROJECT LEAD: Harvey Zambrano, The Ohio State University RESEARCH TITLE: Controlling the electroosmotic flow in silica nanochannels: An atomistic study FUNDING SOURCE: Army Research Office WEBSITE: http://researchgate.net/profile/Harvey_Zambrano/

Advanced Materials

21

Investigating Nematic Vesicles Selingers evaluate how shape affects the behavior of liquid crystals Liquid crystals are at the heart of the technology inside most computer, tablet and smartphone displays today, and researchers are finding more applications for liquid crystals every day – in fields, such as advanced photonics, sensors, bio- and medical molecular devices, and smart materials for new energy applications.

Liquid crystals are chemical compounds that can behave like both conventional liquids and solid crystals. They can demonstrate vastly different properties, depending upon the position and orientation of its molecules in arrangements that researchers describe as phases. One of the most common is the nematic phase, where molecules align themselves parallel to each other but not in well-defined planes within tiny hollow spheres called vesicles. The long-axis orientation of a nematic molecule is referred to as its director.

“In membranes with nematic liquid-crystalline order, there is a geometric coupling between the nematic director and the shape: nonuniformity in the director induces curvature, and curvature provides an effective potential acting on the director,” said Jonathan Selinger, Ph.D., an Ohio Eminent Scholar at the Liquid Crystal Institute and a professor of chemical physics at Kent State University.

For a closed vesicle, there must be a total topological charge of +2, which normally occurs as four defects of charge +1/2 each. Previous research has suggested that these four defects will form a regular tetrahedron – a shape with four triangular faces – which leads to a tetrahedral-shaped

Above: These simulations depict two vesicles in equilibrium with surfactant molecules in solution. On the left, the vesicle has four defects with topological charge of +1/2 each; the inset shows an expanded view of one of these defects. On the right, the defects have merged into holes at the ends of the vesicles.

vesicle, a feature that may be useful in designing very small, colloidal particles for photonic applications.

Selinger’s group, also guided by Robin Selinger, Ph.D., a professor of chemical physics at KSU, took three approaches to investigating the behavior of a nematic vesicle: particle-based simulation, spherical harmonic expansion and finite-element modeling. When a liquid crystal has a purely 2-D intrinsic interaction, they found that the perfect tetrahedral shape is stable over a wide range of parameters. However, they also found that when it has a 3-D intrinsic and extrinsic interaction, the perfect tetrahedral shape is never stable, demonstrating the difficulty in designing tetrahedral structures for photonic crystals.

These studies of non-ideal behavior in nematic membranes led to the motivation for additional research: to understand what other types of non-ideality are important for orientational order on curved or deformable surfaces. In particular, when will researchers see the tetrahedral arrangement of defects that is useful for photonic crystals, and when will other structures occur?

PROJECT LEAD: Jonathan V. Selinger, Kent State University RESEARCH TITLE: Nematic order on a deformable vesicle: theory and simulation FUNDING SOURCE: National Science Foundation WEBSITE: http://www.lci.kent.edu/PI/SelingerJ.htm

22

OHIO SUPERCOMPUTER CENTER

Modeling Transition Metal Alloys Wilkins group developing methods to design, engineer new materials Developing new materials and engineering their novel properties have been the driving forces behind many revolutionary modern technologies.

The emerging capabilities in predictive modeling and simulation have created an opportunity to implement the “materials-by-design” paradigm.

A research group at The Ohio State University has been developing a systematic way to design accurate empirical inter-atomic potentials for many technologically important metals and metal alloys.

While quantum mechanical simulations are usually highly accurate and reliable, they are computationally expensive. Atomistic molecular dynamics simulations based on classical inter-atomic potentials, though, are much less computationally demanding and extend materials simulations beyond the reach of quantum mechanical simulations.

“Vital to the success of classical atomistic simulations are high-quality potentials capable of mimicking the quantum mechanical interactions between atoms,” said John Wilkins, Ph.D., Ohio Eminent Scholar and Ohio State professor of physics. “To bridge between accurate quantum mechanical interactions and fast classical potentials, we optimize the parameters of classical potentials to forces, energies and stresses computed by quantum mechanical simulations for representative atomic configurations.”

His research team leverages the embedded-atom method potential and its expanded versions as the formats for the potentials.

Above: Wilkins’ research group investigated key components in many technologically important alloys, constructing inter-atomic potentials that are accurate and reliable under a wide range of pressures and temperatures.

“What is unique about our group’s approach is that we use ‘splines’ to represent functional terms of potential,” said Hyoungki Park, Ph.D., a postdoctoral researcher on the team. “Splines are piecewise polynomials that have a high degree of smoothness at their connection points, which are called spline knots.”

Discretization of functional terms in the potential by spline knots yields a large number of parameters, and connecting these spline knots forms a hypersurface. The researchers “tame/ train” the hypersurface by fitting it to a quantum mechanical database. Developing complex splinebased potentials by finding a global minimum fit to large quantum mechanical databases has been too difficult for publicly available optimizers.

Utilizing Ohio Supercomputer Center systems, the team built a robust optimizer that uses a hybrid combination of the genetic algorithm and a least-squares minimization routine.

The researchers have constructed many interatomic potentials for vanadium, niobium, tantalum, molybdenum and tungsten. These potentials are accurate and reliable under a wide range of pressures and temperatures, making them very useful for studying mechanical and thermodynamic behaviors.

PROJECT LEAD: John Wilkins, The Ohio State University RESEARCH TITLE: Microscopic modeling of transition metals FUNDING SOURCE: Department of Energy WEBSITE: http://www.physics.ohio-state.edu/~wilkins/

Advanced Materials

23

Enhancing Pavement Engineering Ioannides models crack propagation to refine pavement performance predictions Most American highways are constructed as a Portland cement concrete (PCC) slabs that are poured and finished on a layered roadbed. Such pavement structures are subjected to millions of applications of traffic wheel-loads, as well as numerous cycles of temperature and moisture variations, and eventually succumb to cracking.

Both analytical and numerical techniques have been developed using engineering mechanics for establishing usable predictions of slab responses under typical operating conditions. Nonetheless, most pavement design guides available to engineers today also rely heavily on statistical/ empirical algorithms, in order to relate pavement responses to distresses or cracks.

“Such hybrid approaches employ the so-called Miner’s cumulative damage hypothesis and have been found produce unrealistic pavement performance predictions, because they decouple the responses and the distresses,” said Anastasios Ioannides, Ph.D., associate professor of civil engineering at the University of Cincinnati (UC).

“Such limitations can be addressed by venturing beyond the elastic limit of material behavior and developing a rational failure criterion that obeys instead the principles of fracture mechanics.”

Ioannides is using Ohio Supercomputer Center systems and a general-purpose finite element package, ABAQUS, to model crack propagation in concrete pavement slabs equipped with aggregate interlock and dowel bar load-transferring mechanisms and subjected to temperature and wheel

Above: Anastasios Ioannides at the University of Cincinnati leveraged Ohio Supercomputer Center resources and ABAQUS software to render these simulation of cracking concrete slabs.

loads. Traction separation-based cohesive elements are inserted along the crack plane to simulate the fracture process.

Parameters known to influence both pre-crack and post-crack responses are considered, including joint stiffness, joint opening, aggregate size, dowel-concrete interaction and dowel spacing. To ascertain the validity of results from the proposed finite element discretization, calculated responses are compared with data from previous experimental and numerical studies.

“This investigation is a continuation of the step-by-step development and application of fracture mechanics tools in pavement engineering initiated at UC in the late 1990s and relying on the Fictitious Crack Model,” Ioannides said. “I hope that a contribution will be made to the on-going effort for more mechanistic pavement design to obviate the need for the purely statistical transfer functions resulting from applications of Miner’s hypothesis.”

Efforts of the UC research team were boosted significantly by the 2005 release of ABAQUS Version 6.5, which for the first time included “a family of cohesive elements for modeling deformation and damage in finite-thickness adhesive layers between bonded parts.”

PROJECT LEAD: Anastasios Ioannides, University of Cincinnati RESEARCH TITLE: Three dimensional finite element simulation of crack propagation in jointed concrete pavements subjected to temperature and wheel loads FUNDING SOURCE: Army Research Office WEBSITE: http://sas.ceas.uc.edu/facultyandStaff/profiles/Anastasios_Ioannides.html

24

OHIO SUPERCOMPUTER CENTER

Energy & the Environment

Finding the solutions to significant interrelated global energy and sustainability issues, such as managing wildlife habitats, designing efficient engines and improving air quality, requires computational modeling, simulation and analysis. The Ohio Supercomputer Center provides researchers in these areas with those resources.

You will find a sample of these data-rich projects on the following pages.

Converting CO2 to Fuel Asthagiri, colleagues study feasibility of turning greenhouse gas to alcohol Carbon dioxide (CO2) is considered an atmospheric trace element, yet also is recognized as a greenhouse gas that has increased significantly since the advent of industrialization. The conversion of CO2 to alcohols is a potentially attractive way to translate intermittent sources of renewable energy, such as wind and solar, into a viable form of chemical energy for our existing transportation infrastructure. For this approach to be feasible, the CO2 conversion must be done efficiently and the fuel product selectivity optimized. Identifying such catalysts has been elusive, despite extensive experimental work over two decades.

As members of a DOE Energy Frontier Research Center project at Louisiana State University, Aravind Asthagiri, Ph.D., and his colleagues at The Ohio State University and Penn State University are examining why CO2 reduction on copper (Cu) electrodes produces methane and ethylene instead of methanol, as seen in heterogeneous methanol synthesis. Such work represents a first step in understanding the important factors that affect selectivity and activity of CO2 electroreduction catalysts. The research team leveraged Ohio Supercomputer Center resources to perform density functional theory (DFT) calculations to investigate both the thermodynamics and kinetics of CO2 reduction on Cu. “Our calculations identified the free energies and activation barriers of the elementary steps for CO2 electrochemical reduction on copper electrodes,” said Asthagiri, an Ohio State associate

Above: Optimized structures of the initial, transition, and final states associated with the elementary step of CO2 reduction to COOH with 1 H2O in the (a) watersolvated, and (b) water-assisted models. (Orange=copper, red=oxygen, gray=carbon, white=hydrogen.)

professor of chemical and biomolecular engineering.

“This approach represents a paradigm shift in computational electrocatalysis, as we are applying a novel method to evaluate potential-dependent activation barriers, which have thus far been rare in examining electrocatalytic reactions with DFT.” The activation barriers turn out to be critical to understanding the unique selectivity found in Cu electrodes. Based on these new DFT calculations, the team determined a reaction path that is able to reconcile a range of experimental data on CO2, carbon monoxide (CO) and formaldehyde (CH2O) reduction on Cu electrodes. Instead of proceeding through a CHO intermediate (aldehyde, which leads to methanol), methane formation goes through reduction of CO to COH (alcohol), which eventually leads to CHX species (hydrocarbons) that can produce both methane and ethylene, as has been observed experimentally.

“This analysis suggests that future design of electrocatalysts for CO2 electrochemical reduction should focus on the relative energetics of COH versus CHO formation to tailor the selectivity of the desired products towards liquid fuels, such as methanol,” said Asthagiri.

PROJECT LEAD: Aravind Asthagiri, The Ohio State University RESEARCH TITLE: Selectivity of CO2 reduction on copper electrodes: The role of the kinetics of elementary steps FUNDING SOURCE: U.S. Department of Energy WEBSITE: http://cbe.osu.edu/people/asthagiri.1

26

OHIO SUPERCOMPUTER CENTER

Increasing Engine Efficiency Ibrahim collaboration evaluating options for better turbine airflow management The flow in the endwall region of a compressor or turbine airfoil passage contains a complex system of vortices, which interact with each other and produce undesirable effects, including the disruption of cooling flows and the generation of aerodynamic losses. With unshrouded airfoils, the tip endwall region also includes leakage through the clearance between the rotor blade tips and the turbine casing. This tip gap leakage can cause undesirable heat transfer and interact with the rest of the endwall flow to increase secondary losses.

Endwall flows are further complicated by periodic unsteadiness as airfoils move through the wakes shed by upstream airfoils. The velocity deficit and elevated turbulence in the wakes affect the flow on the airfoil surfaces and at the endwalls.

“Many studies have considered endwall flows, and several methods have been proposed for flow modification and secondary loss reduction,” said Mounir Ibrahim, Ph.D., chair and professor of mechanical engineering at Cleveland State University (CSU) and director of the Power and Energy Systems Lab at CSU’s Fenn College of Engineering. “The flow is still very complex and not completely understood, particularly with regard to periodic unsteadiness. A more complete understanding allows for better flow management, leading to reduced losses and heat transfer. This will lead to increased engine efficiency, resulting in reduced emissions, fuel consumption and operating cost.”

Above: Simulations conducted by a research team led by Mounir Ibrahim at Cleveland State University depicts total pressure loss comparisons downstream of the trailing edge in an airfoil passage (left: experiment, right: CFD).

Ibrahim’s research team, in collaboration with Ralph Volino, Ph.D., a professor of mechanical engineering at the U.S. Naval Academy (USNA), is investigating the endwall flow in a turbine passage. Cases with and without tip gaps and tip flow control are being evaluated, including both experimental documentation (USNA) and computation modeling (CSU). In the experiments, flow visualization, detailed pressure loss and velocity field measurements are being made in the endwall region, and unsteady flow field measurements with ensemble averaging on the wake passing events are being included.

The research team’s efforts are: 1) fully documenting flow physics; 2) documenting the unsteady response of the endwall flow with and without tip gaps to wakes using PIV and other measurement techniques; 3) demonstrating and improving the understanding of how to control tip flows using passive techniques and active blowing to improve engine performance; 4) producing a new experimental database with which to guide and validate the development of new computational tools; and 5) testing computational models for better prediction of turbomachinery endwall flows.

PROJECT LEAD: Mounir Ibrahim, Cleveland State University RESEARCH TITLE: Computational investigation of unsteady endwall and tip gap flows in gas turbine passages FUNDING SOURCE: National Aeronautics and Space Administration WEBSITE: http://academic.csuohio.edu/ibrahimm/research.html

Energy & the Environment

27

Learning About the Universe Connolly study compares cosmic rays and neutrinos as information sources The earth and other celestial bodies are continually bombarded by extremely fast-moving, subatomic particles known as cosmic rays, gamma rays and neutrinos. Although the origin of ultra-high energy (UHE) particles (defined as above 1018 electronvolts or eV) is unknown, scientists believe that they could be created and accelerated by cataclysmic events, such as collisions between compact objects or a star falling into the black hole at the center of a galaxy.

Nearly all of the cosmic rays that originate outside of Earth’s atmosphere are the nuclei of well-known atoms, such as hydrogen and helium, stripped of their electron shells. Neutrinos also are subatomic particles, but they possess a neutral electrical charge, only interact weakly and, therefore, can pass through normal matter unimpeded.

Astrophysicists can use information gleaned from these elusive particles to learn more about the universe and even about particle physics. A study led by Amy Connolly, Ph.D., assistant professor of physics at The Ohio State University, leveraged Ohio Supercomputer Center resources to evaluate the complementarity of cosmic rays and neutrinos as a source of information about the universe’s most extreme particle accelerators.

“Cosmic rays above 1019.5 eV cannot have originated from more than approximately 100 megaparsecs from Earth due to the onset of the GZK process,” said Connolly. The Greisen– Zatsepin–Kuzmin (GZK) process represents the

Above: Charts created by Amy Connolly at The Ohio State University show the complementarity of cosmic rays and neutrinos according to which cosmic sources they probe. Both use a flat redshift evolution and E-1 injection spectrum.

slowing and bending of cosmic rays over long distances due to microwave background radiation.

“Neutrinos, however, can travel cosmological distances unabated and would be our only view of the distant universe above the GZK cutoff.”

Connolly’s group utilized open-source CRPropa, as well as other simulation and analysis software, to calculate the energy spectrum of protons, gamma rays and neutrinos observed at Earth. They analyzed protons produced at energies between 1016 eV and 1025 eV and at distances up to 3 gigaparsecs.

They quantified the implications of current cosmic ray measurements and neutrino constraints on parameters characterizing the UHE sources, subject to existing gamma ray measurements. The group measured the spectral index below 1020 eV and the overall normalization of source emission.

Neutrinos provide the ultimate acceleration energy (Emax) and are unique probes of redshift evolution, or the dependence of the density of sources on distance. For the first time, they constrained Emax based on the non-observation of UHE neutrinos by current experiments and quantified the sensitivity of future experiments to measuring Emax and the source evolution.

PROJECT LEAD: Amy Connolly, The Ohio State University RESEARCH TITLE: The complementary nature of cosmic rays and neutrinos in constraining UHE astrophysics FUNDING SOURCE: National Science Foundation WEBSITE: http://physics.ohio-state.edu/directory_pages/detail.php3?id=2676

28

OHIO SUPERCOMPUTER CENTER

Controlling Supersonic Airflow Gaitonde team researching use of heated plasma to control shock wave effects The interactions between shock waves from supersonic aerospace vehicles and the airflow immediately adjacent to their exterior surfaces, referred to as shock boundary layer interactions, can have significant effects on the design and performance of wings, control surfaces and propulsion systems. Unsteadiness caused by the shocks can lead to thermal and mechanical stresses that can lead to degradation of performance or outright failure of a vehicle.

Experimental research is underway to control the unsteadiness using plasma actuators to generate and introduce heated plasma into the airflow. At the same time, computational research related to these experiments is being conducted to generate a better understanding of these control mechanisms and determine how to improve them.

Large Eddy Simulations – mathematical models for turbulence – are needed to understand the physics of both baseline and controlled cases.

To develop this capability, Datta Gaitonde, Ph.D., a professor of mechanical and aerospace engineering at The Ohio State University, and his team, are conducting a systematic research program leveraging the computational power of the Oakley Cluster at the Ohio Supercomputer Center. The challenge of the first stage involved generating a supersonic turbulent boundary layer that demonstrated the expected mean and statistical properties and contained the proper coherent structures.

Above: Simulations developed by a research team led by Datta Gaitonde at The Ohio State University depict the effect of impinging and reflecting shocks on coherent structures in the boundary layer.

“It is computationally expensive and theoretically difficult to simulate natural transition, and often, the computational overhead of simulating the transition of the flow is larger than that of the main shock interaction,” said Gaitonde, the John Glenn Chair and an Ohio Research Scholar.

Simulations on fine, medium and coarse mesh resolutions show that the medium grid reproduces most of the behavior of the fine grid, making it a suitable grid for future shock/turbulent boundary-layer interactions (STBLI) studies, whereas the coarse grid has a relatively delayed transition, affecting its ability to have the same qualitative and quantitative results as the fine grid. The evidence suggests that the coarser mesh results are consistent with delayed transition and may, in fact, reproduce many of the key features of turbulence if the transition length can be shortened.

Recently, these results have been employed to perform the full STBLI, using a combination of OSC and Department of Defense machines. The results are very promising, in that the two major features of interest, flow separation and unsteadiness, are both successfully modulated. Current efforts are focusing on determining the “sweet spot” for control techniques.

PROJECT LEAD: Datta Gaitonde, The Ohio State University RESEARCH TITLE: Spatially developing supersonic turbulent boundary layer with a body-force-based method FUNDING SOURCE: U.S. Air Force Office of Scientific Research, Ohio Research Scholars Program WEBSITE: http://hfcmpl.osu.edu/

Energy & the Environment

29

Seeking Greener Energy Sources Botte research group investigating urea’s potential to power hydrogen production Coal is currently the largest source of electricity generation in the United States, while gasoline and diesel fuel power most vehicles. However, coal, gasoline and diesel fuel are non-renewable resources, and the combustion of these fossil fuels produces various pollutants. As a result, alternative, non-polluting energy sources such as hydrogen are desirable. Unfortunately, most hydrogen production processes also rely upon fossil fuels. Therefore, there is a clear need for renewable energy sources that can power hydrogen production, but at the same time mitigate pollution.

“Although electrolysis of water has been considered, a new technology to produce hydrogen by the electrochemical oxidation of urea from animals and humans is plausible and provides an opportunity to remediate waste,” said Gerardine Botte, Ph.D., Russ Professor of chemical and biomolecular engineering in the Russ College of Engineering and Technology at Ohio University.

“Using nickel oxyhydroxide as the catalyst, hydrogen and nitrogen can be produced in a basic medium, and the process should consume 70 percent less energy than water electrolysis.”

In the presence of hydroxide ions, urea is electrochemically converted to nitrogen and hydrogen using nickel-based electrodes. However, the mechanism of this reaction has not been fully understood as the conversion of urea typically decreases over time. Botte’s research team has been investigating the interaction of urea and

Above: Simulations created by Gerardine Botte’s research group at Ohio University display a side (left) and top (right) view of a possible unit cell of monodentate urea adsorption on Ni(1 0 0). The red oxygen atom is in the top position (T), one blue nitrogen atom is in the hollow position (H) and the other blue nitrogen atom is close to the bridge position (B).

hydroxide ions with nickel as an initial study in understanding the conversion of urea.

Botte’s team is using atomic-level modeling to obtain a better picture of the surface chemistry during urea oxidation. The modeling is based on Density Functional Theory (DFT) as applied in CRYSTAL, a localized atomic basis set code, and comparisons are being made to in-situ measurements using X-ray Diffraction, Raman and Infrared spectroscopy methods at Ohio University’s Center for Electrochemical Engineering Research (CEER).

Due to the large amount of parallel CPU time required for the calculations, Botte identified the need for supercomputing capabilities made available by the Ohio Supercomputer Center.

“We are performing geometry optimization and frequency calculations at varying adsorption geometries and coverage of urea and hydroxide on nickel using Crystal09,” explained Damilola Daramola, Ph.D., a multiscale-modeling scientist at CEER. “These results are providing information on the electronic, geometric and vibrational properties of the adsorbates and substrate and could help in furthering the knowledge of this urea-to-hydrogen technology.”

PROJECT LEAD: Gerardine Botte, Ohio University RESEARCH TITLE: Computational studies of urea and hydroxide adsorption on nickel surfaces FUNDING SOURCE: Army Construction Engineering Research Laboratory WEBSITE: http://www.ohio.edu/engineering/about/people/profiles.cfm?profile=64EA7FE7-5056-A800-489FB6C991848078

30

OHIO SUPERCOMPUTER CENTER

Research Landscape

Ohio’s strengths in basic and applied research are broad and deep, spanning a multitude of fields, such as economics, sociology, computer science, automotive design and signal processing. This spectrum of Ohio Supercomputer Center clients encompasses many fields of study, as highlighted on the following pages.

Optimizing 400 MPH Aerodynamics Clark, Rizzoni refine body shape of Venturi Buckeye Bullet 3 land-speed race car The Venturi Buckeye Bullet 3 (VBB3) is a streamlined electric land-speed race car designed and assembled by undergraduate and graduate students at The Ohio State University’s Center for Automotive Research (OSU CAR). After setting world and international speed records with previous vehicles, the team recently built a new streamliner with the latest electric drive technology from program partner Venturi Automotive, an electric car company based in Monaco. The new drive train is capable of more than 2,000 horsepower and is expected to eventually thrust the four-wheel drive VBB3 to speeds in excess of 400 mph, making it the first electric vehicle to surpass this mark. Its predecessors were the first (and are to date the only) electric vehicles to exceed 300 mph.

“At these high speeds, aerodynamics play a crucial role in vehicle and driver safety, as well as being one of the critical factors that dictate the peak performance of the vehicle,” said Giorgio Rizzoni, Ph.D., Ohio State professor of mechanical and aerospace engineering and director of OSU CAR. “From the start of the design process, the aerodynamics of each proposed body shape for the VBB3 was evaluated in Fluent and OpenFOAM using the computational resources available at the Ohio Supercomputer Center.”

In addition to optimizing the body shape to achieve minimum drag, packaging constraints and vehicle stability considerations played a major role in the final design. For example, to ensure

Above: Professor Giorgio Rizzoni, graduate student Casie Clark and the Venturi Buckeye Bullet 3. The VBB3 team recently tested the new streamliner at Wendover, Utah. Clark, the team aerodynamicist, used Ohio Supercomputer Center resources to make numerous studies of the vehicle, including a pressure contour (top) and a velocity contour (bottom) at 300 miles-per-hour.

(Photo credit: Denis Broussard, Venturi Automobiles)

the desired yaw stability at high speed, extensive simulations were run to evaluate the effectiveness of the vertical tail, which underwent several design iterations. Then, several new runs and studies were launched to evaluate the overall aerodynamic performance of the vehicle, with some solutions involving more than 42 million cells.

“Since it’s a new car with a new body, it has been interesting to look at the current aero, and to identify areas of improvement,” said Casie Clark, an Ohio State graduate student in aeronautical and astronautical engineering. “For example, I ran CFD jobs aimed at designing a wind deflector to deflect air around the tires, instead of allowing a large air mass inside the wheel wells, which creates substantial drag.”

The team recently transported the VBB3 to Wendover, Utah, where, despite flooding that forced cancellation of the international time trials, the team tested and refined the streamliner at the local airport’s 1.5-mile long runway.

PROJECT LEAD: Giorgio Rizzoni, The Ohio State University RESEARCH TITLE: CFD simulations of the Buckeye Bullet electric car FUNDING SOURCE: The Ohio State University, Venturi Automobiles WEBSITE: http://car.osu.edu/motorsports/buckeye-bullet

32

OHIO SUPERCOMPUTER CENTER

Enhancing Surveillance Techniques Rigling, Rovito, others employ Watson techniques to advance object recognition Object recognition is an important problem that has many applications that are of interest to the Air Force. Object recognition is a key enabler to autonomous exploitation of intelligence, surveillance and reconnaissance (ISR) data, which can make the automatic searching of millions of hours of video practical. Despite decades of research into the problem of object recognition, success in all operating conditions has been elusive.

Recently success in pattern recognition has been demonstrated by algorithms that make use of large amounts of training data, such as the algorithms employed by IBM’s Watson supercomputer, which defeated the world Jeopardy champion. Watson was preloaded with multiple terabytes of digital books that were indexed for quick retrieval.

Brian Rigling, Ph.D., is a member of a Wright State University research team that is working with colleagues at the Air Force Research Laboratory (AFRL) to leverage the technique in applying electro-optical exploitation to object recognition.

This approach requires making large sets of electro-optical training data available to researchers.

“Computer-modeled scenes are described by five pieces of information: scene object geometry, camera viewpoint, textures, lighting and shading,” explained Rigling. “Our goal was to create predictable, realistic-looking imagery, so we selected LuxRender, a free, unbiased physicsbased renderer that can solve the rendering equation for general lighting.”

Above: A research group led by Brian Rigling at Wright State University and Todd Rovito at the Air Force Research Lab developed a simple synthetic intersection and synthetic straight test track, based upon Sadr City, Iraq. The comparison view is a satellite image of the actual intersection.

However, this setup does not account for all lighting phenomena and camera viewpoints, so the team created a rendering pipeline that moves the simulated camera every 3 degrees over a hemisphere and used 17 lighting placements.

Each frame of data takes three minutes to process and each vehicle has 62,000 frames; on a single computer a single vehicle will take 92 days to compute. The team’s rendering pipeline can be run simultaneously; by leveraging computational resources at the Ohio Supercomputer Center the team could render 2,048 frames at the same time reducing the run time to a few hours.

“We used OSC’s Oakley cluster to quickly render large scenes with 100 million pixels that covered land segments of 28 KM by 20 KM,” said Todd Rovito, a research computer scientist at AFRL. “We have already generated twenty minutes of large-scale synthetic data and ten complete vehicle domes for a total of 620,000 frames. We plan to release all the generated data to the scientific research community via AFRL’s Sensor Data Management System.

PROJECT LEAD: Brian Rigling, Wright State University RESEARCH TITLE: Fiorano: Detection test bed and modeling FUNDING SOURCE: Air Force Research Laboratory WEBSITE: https://www.sdms.afrl.af.mil/index.php?collection=eo_synthetic_data

Research Landscape

33

Accelerating Computer Communication Panda team uses Oakley Cluster to test, optimize MVAPICH2-X performance Modern high performance computing systems allow scientists and engineers to tackle grand challenge problems in numerous fields, such as astrophysics, earthquake analysis, weather prediction, nanoscience modeling and biological computations. In concert with the many use cases, the field of computer architecture, interconnection networks and system design is undergoing rapid change. Advances in computing systems are coming mainly in the form of increased parallelism using multi-core processors, many-core accelerators and improved communication interfaces.

Dhabaleswar K. (DK) Panda, Ph.D., professor of computer science and engineering at The Ohio State University, leads a research group focused on designing better communication runtimes to take advantage of new network features so that applications can be developed and implemented using scalable, high performance MPI (message passing interface) or hybrid MPI models.

“Communication runtimes must also evolve to support emerging architecture trends and programming models. For example, it is widely believed that a hybrid programming model is optimal for many scientific computing problems, especially for exascale computing,” said Panda.

“Our library, MVAPICH2-X, is the first to provide a unified high-performance runtime that supports both MPI and PGAS programming models. This minimizes the development overheads that have been a substantial deterrent in porting MPI applications to PGAS models. The unified runtime also

Above: Dhabaleswar Panda at The Ohio State University led a group in studies of communication runtimes on systems, including the HP-Intel Xeon Oakley Cluster at the Ohio Supercomputer Center.

delivers superior performance compared to using separate MPI and PGAS libraries by optimizing use of both network and memory.”

Panda’s research team uses Ohio Supercomputer Center systems to test the performance of new communication algorithms, runtime designs and application evaluations. For a recent study, a team member reserved about a third of the more-than 8,300 nodes on OSC’s Oakley cluster for runs of a redesigned HPL-benchmark, which can more accurately measure system performance for mixed CPU/GPU node systems. This was made possible through the close collaboration between Panda’s team and OSC staff members Karen Tomko, Ph.D., and Doug Johnson.

The MVAPICH2 and MVAPICH2-X software packages, developed by his research group, are currently being used by more than 2,055 organizations in 70 countries. Panda held the first MVAPICH User Group meeting at OSC in August.

Panda and his students are also involved in designing a high performance and scalable version of Hadoop for Big Data to exploit remote direct memory access (RDMA) technology, as provided by InfiniBand on modern clusters. The Hadoop-RDMA version is publicly available and includes RDMA-based design for multiple components of Hadoop.

PROJECT LEAD: Dhabaleswar Panda, The Ohio State University RESEARCH TITLE: Research in communication runtimes for emerging HPC systems FUNDING SOURCE: National Science Foundation, Department of Energy WEBSITE: http://nowlab.cse.ohio-state.edu/

34

OHIO SUPERCOMPUTER CENTER

Exploiting Powerful Lasers Schumacher group models laser experiments to study interactions inside targets A new generation of powerful lasers has recently become operational, like the 400 Terawatt Scarlet laser at The Ohio State University (a Terawatt is equal to one trillion watts). These lasers can drive matter to extreme temperatures and densities, applying pressures well over a billion atmospheres.

There is a worldwide effort to use intense lasers to recreate conditions at the core of gas giant planets or stellar atmospheres in the laboratory where they can be studied. Moreover, intense lasers also can generate beams of electrons, ions, neutrons, x-rays, gamma rays and even positrons that might have properties useful for applications, ranging from medical treatments to testing airplane parts for corrosion.

However, the extreme laser-matter interactions possible with such intense light are beyond any known analytic theory, and numerical simulations are crucial for designing experiments and understanding the resulting data. Douglass Schumacher, Ph.D., associate professor of physics at Ohio State, leads a modeling group that performs simulations of experiments using Scarlet and other powerful lasers around the country.

“The experimental diagnostics in use can provide only an indirect measure of the incredible interactions taking place,” said Schumacher. “Our simulations allow us to determine what is going on inside the target, where the temperature can easily exceed one million degrees.”

Schumacher’s modeling team uses the computational resources of the Ohio Supercomputer

Above: A research group led by Douglass Schumacher at The Ohio State University are conducting simulations of experiments being conducted on powerful lasers, such as Ohio State’s 400 Terawatt Scarlet laser, shown here.

Center to perform “particle-in-cell” simulations that break the target into hundreds or thousands of tiny pieces, each modeled using a single processor. The processors communicate with each other, sharing information about particle motion and electric and magnetic fields. This way, the laser interaction and subsequent evolution (and destruction) of the target can be followed with high temporal and spatial resolution.

“We have been able to explain discrepancies between experimental results and other models by running simulations large enough to account for the many ways the laser energy can spread throughout the target,” Schumacher said.

His group is studying a number of different problems, including how to create regions of high density and temperature in solid targets using Scarlet. Little is known about matter under these conditions, but it is believed to be important for planetary evolution. His team also is exploring how to turn Scarlet into a proton or x-ray source with properties that are ideal for a range of uses, such as treating cancer.

PROJECT LEAD: Douglass Schumacher, The Ohio State University RESEARCH TITLE: Modeling intense laser plasma interactions driven by next generation short pulse lasers FUNDING SOURCE: Air Force Office of Scientific Research, Department of Energy WEBSITE: http://hedp.osu.edu

Research Landscape

35

Analyzing Group Behavior Models Lee, Yang studying the influence of incomplete information on social interactions In a social group, some information is shared by everyone and other information is known only to some members. For example, when analyzing the interactions of college students’ academic performance, it is not likely that a student knows the IQ and/or SAT scores of all the other students in the class. Therefore, it is practical to incorporate incomplete information into a model that is used to study the interactions of individuals within any social grouping. Moreover, under different information structures, an individual will form different conditional expectations about the behavior of another group member.

“On one hand, the more information an individual has, the more precise that person’s prediction will be,” noted Lung-Fei Lee, Ph.D., a professor of economics at The Ohio State University. “On the other, the correlation of that person’s behavior with those of other individuals depends on the realizations of the publicly known characteristics and the privately known features.

As a result, the intensity of interactions of behaviors of individuals in a social group will vary with the relevant information structure.”

To account for incomplete information in behavior models, researchers determine that the actions of one individual is influenced by that person’s expectations of behaviors of the rest of the people in a group, where the expectations are conditional upon the individual’s information set.

By using a variable to represent those expectations, researchers can model social interactions

Lung-Fei Lee at The Ohio State University is developing models of group behavior that include situations where some members have incomplete information.

of various types of behaviors. Moreover, since it can be assumed that it is the conditional expected behaviors that produce the variable for an individual’s behavioral responses, scientists can investigate the influence of incomplete information on social interactions.

Lee and doctoral student Chao Yang are analyzing the model as a simultaneous-move game with incomplete information. The observed data is viewed as the outcome of a Bayesian Nash Equilibrium (BNE) – a strategy profile and player beliefs about other players that maximizes the expected payoff.

“The equilibrium can be solved via the projection method, whereby we use linear combinations of the base functions to approximate the expectation functions and solve the coefficients of those linear combinations using the equilibrium condition,” explained Lee. “Although the projection method helps alleviate much of the computation burden, the model estimation is still computationally expensive, and we are using Ohio Supercomputer Center systems for Monte Carlo experiments to analyze various behaviors, information structures and network associations.”

PROJECT LEAD: Lung-Fei Lee, The Ohio State University RESEARCH TITLE: Social interactions and incomplete information: A game theoretical approach FUNDING SOURCE: The Ohio State University WEBSITE: http://www.econ.ohio-state.edu/lee/

36

OHIO SUPERCOMPUTER CENTER

Industrial Engagement

The Ohio Supercomputer Center has a long history of supporting industrial research, reaching back as far as the Center’s founding in 1987. As you will find described on the following pages, manufacturers have leveraged the Center’s computational and storage resources to design and test many products, such as electronics, fans, containers, fuel cells and wind deflectors.

Optimizing Plastic Containers Manufacturer uses modeling, simulation to lighten weight, maintain strength KLW Plastics, a leading designer, manufacturer and distributor of containers, recently partnered with Kinetic Vision and the Ohio Supercomputer Center to evaluate the effectiveness of advanced modeling and simulation technologies to optimize its container products by lightening their weight, while maintaining the required strength. The partnership arose through the federal National Digital Engineering and Manufacturing Consortium (NDEMC) program, a public-private partnership funded by the U.S. Economic Development Administration to support and enhance the use of modeling and simulation among America’s small and medium manufacturers.

“Pursuit of a higher standard in containers is part of our everyday work here,” said Tom Gruber, national sales manager for KLW, based in Monroe, Ohio. “Our state-of-the-art manufacturing technology, our research and development efforts, product design, raw material sourcing, our production process and testing contribute to a highly consistent and reliable final product.”

KLW was using 3-D CAD applications on desktop systems prior to the NDEMC program, but had not yet integrated modeling and simulation into their product development and testing workflow. Through the NDEMC partnership, Cincinnati product design and development firm Kinetic Vision worked with the engineering team at KLW to capture computerized tomography (CT) data of one of their signature blow-mold containers.

Above: Kinetic Vision engineers partnered with KLW Plastics to test integration of modeling and simulation into their product development and testing workflow.

The team then fed the data into OSC’s Oakley HP-Intel Xeon cluster to compute and evaluate finite element models of the tight-head container using the LS-DYNA software package. The project built upon and fully validated the company’s virtual top-load, pressure and drop tests using current production KLW packaging.

“Developing validated advanced simulation models through the collaboration of Ohio-based companies, the Ohio Supercomputer Center and technologies like industrial CT is making manufacturers more competitive in the global market place,” said Jim Topich, director of engineering of Kinetic Vision.

KLW identified two additional long-term benefits of the simulations: 1) a better understanding of how to utilize advanced modeling and simulation prototyping resources prior to cutting expensive new molds, and 2) improved predictions of container performance using different thickness distributions. Lessons learned from this project should lead to substantial cost savings for the company by reducing the weight of the containers, while maintaining reliability.

“Our idea is to be as advanced as we can and set good ways to doing business in an industry that is very stagnant,” Gruber said. “We’re trying to make waves.”

PROJECT LEAD: Jim Topich, Kinetic Vision RESEARCH TITLE: Design optimization of plastic containers FUNDING SOURCE: National Digital Engineering and Manufacturing Consortium WEBSITE: http://kinetic-vision.com/about-us/

38

OHIO SUPERCOMPUTER CENTER

Dissipating Thermal Energy Engineers evaluate behavior of circuit boards used in the process control industry Demand for electronic devices of increasingly smaller sizes and with substantially improved processor and graphics functionality has resulted in higher-density power requirements. Consequently, significant increases in heat generated are being registered at the component, board and system levels. These higher operating temperatures can significantly shorten the operational lifetime of an electronic device. As a result, managing the evolution, distribution and dissipation of thermal energy in electronic components and circuits is important for their long-term reliability.

AltaSim Technologies partnered with the Ohio Supercomputer Center to evaluate the thermal behavior of newly developed printed circuit boards designed for use in the process control industry. The partnership evolved last year through the federal National Digital Engineering and Manufacturing Consortium (NDEMC) program, a public-private partnership funded by the U.S. Economic Development Administration to support and enhance the use of modeling and simulation among America’s small and medium manufacturers.

“At AltaSim, we realize that customer needs vary depending on time in the product design cycle, availability of validated thermal models and selection of technology to meet cost, manufacturability and thermal design targets,” said Kyle C. Koppenhoefer, Ph.D., a principal at AltaSim. “Integration of strategically designed computational simulations allows us to quantify the effects

Above: Engineers at AltaSim Technologies leveraged Ohio Supercomputer Center resources to model thermal energy transfer from full, printed circuit boards.

of specific design changes and the interrelationship between multiple variables, thus providing a broad, yet metered view of the product design.”

To achieve the required scale and level of accuracy, AltaSim engineers needed to consider large numbers of components to achieve a comprehensive simulation of the full printed circuit board.

They also wanted to include a significant level of detail in both the board and surroundings, and consequently the size of the model escalated significantly. To allow the engineers to analyze the modeled behavior within a reasonable length of time, AltaSim accessed OSC’s flagship HP-Intel Xeon Oakley Cluster and the ANSYSICEPAK engineering simulation solution software package. ANSYS prototyping tools can be used to put a virtual product through a rigorous testing procedure before it is created as a physical object.

The decreased completion time for analyses allowed AltaSim engineers to consider different designs rapidly and make design recommendations for improved product performance that would otherwise not have been possible. As a consequence, they achieved final designs for operational components that have significantly improved product performance.

PROJECT LEAD: Kyle Koppenhoefer, AltaSim Technologies RESEARCH TITLE: Thermal energy management of electric circuits FUNDING SOURCE: National Digital Engineering and Manufacturing Consortium WEBSITE: http://altasimtechnologies.com

Industrial Engagement

39

Optimizing Fuel Cell Efficiency Enhancements to TMI product to “save lives and hundreds of millions of dollars” Since the early 1990s, the promise of fuel cells has been onsite power generation with the same round-the-clock availability that has long been the exclusive province of the electric utility industry, but without the cumbersome distribution grid.

Engineers at Technology Management, Inc. (TMI) have developed a solid oxide fuel cell (SOFC) system and are collaborating with Lockheed Martin, a major American defense contractor, and Stark State College of Technology to commercialize this technology project.

“Our system design specifically focuses on a simple, low cost platform that can be produced in volume by Ohio’s automotive supply chain and appliance manufacturers,” said Benson Lee, president of TMI. “The high fuel efficiency of fuel cells – consuming a third to a half the fuel of a comparable diesel genset – will save lives and hundreds of millions of dollars annually, the price being paid to get fuel to the front lines.”

TMI engaged with the Ohio Supercomputer Center to conduct SolidWorks geometry testing to optimize the efficiency of their fuel cell technology platform. The TMI-OSC partnership arose through the federal National Digital Engineering and Manufacturing Consortium (NDEMC) program, a public-private partnership funded by the U.S. Economic Development Administration to support and enhance the use of modeling and simulation among America’s small and medium manufacturers.

Above: Ohio Supercomputer Center engineers partnered with experts at Technology Management Inc. to model TMI’s fabrication process and component specifications of its highly efficient fuel cells.

“Models were prepared for both fuel cells and hot subassemblies to help optimize performance through iterative simulation of various flow and geometric factors associated fuel cell operation,” said Lee. “The net result we expect from the project will include improved product performance and reliability, resulting in decreased time to market introduction, with accelerated manufacturing job creation and accelerated market expansion.”

The models prepared and tested by OSC staff have been used to improve understandings that could not be obtained through experimental methods. Individual fuel cells were modeled to determine the impact of imperfections in the fabrication process on cell and stack performance.

The data from these simulations have been used to modify TMI’s fabrication process and component specifications, which has helped improve reproducibility and performance.

TMI is currently testing these improvements and preliminary findings have shown promising results. By utilizing OSC’s advanced computing resources through its solution partners to achieve both its technology and production goals, Lee is confident that TMI will remain competitive in its targeted global markets for years to come.

PROJECT LEAD: Benson P. Lee, Technology Management Inc. RESEARCH TITLE: Modeling of fuel cell stacks and systems FUNDING SOURCE: National Digital Engineering and Manufacturing Consortium WEBSITE: http://AnywherEnergy.com/

40

OHIO SUPERCOMPUTER CENTER

Improving Industry Collaborations Project develops “simulation-as-a-service” app for remote software access A research team recently sought to transform how professionals and students make and learn about advanced manufacturing components through a “simulation-as-a-service” app based on cloud resources and software access. Their application allows users to remotely access software and compute resources using a virtual desktop-asa-service system for advanced manufacturing processes.

Prasad Calyam, an assistant professor of Computer Science at the University of Missouri, led a team of specialists from the Ohio Supercomputer Center, the Ohio Academic Resources Network (OARnet), The Ohio State University, the City of Dublin, Ohio, and Metro Data Center (MDC), in partnership with TotalSim, VMware and HP. Dublin operates DubLink, a 96-strand fiber-optic data network that connects directly to MDC, a regional high-tier data center in Dublin, and to OARnet, Ohio’s 100 Gigabit per second statewide research and education network backbone. TotalSim, a computational fluid dynamics design firm, has partnered with OSC on multiple projects to help small and mid-sized businesses engage in modeling and simulation.

“The service allows for better real-time collaboration between TotalSim’s expertise and the customers, allowing more rapid iterations between user feedback and revised simulations,” explained Calyam. “This decreased time is thanks to the fact that large simulation data-sets are moved only across OARnet and DubLink, and the integrated, secure provisioning of desktop applications (such

Above: A screen shot of simulation results from an app developed by TotalSim LLC as a virtual desktop-as-aservice system for advanced manufacturing processes.

as Paraview and Microsoft Word) and cloud applications (such as WebEx and Box.net) in the simulation-as-a-service app. External users view the results through a thin-client connection to a virtual desktop, but the large data sets and provisioned applications never leave the cloud.”

The team presented their work at the Next Generation Application Summit in June, and won an award for the “Best Application for Advanced Manufacturing.” The challenge was sponsored by Mozilla Foundation and is part of the nonprofit US Ignite initiative, funded partly by the National Science Foundation.

To demonstrate the application, the research team featured the Truck Add-on Predictor, a computational tool that allows suppliers to model the airflow over a joint vehicle/trailer body and the corresponding aerodynamic forces. By understanding these elements, designers can reduce the weight of key structural components or replace them with other materials, while potentially increasing aerodynamic efficiency. The predictor was developed through the Department of Energy’s Lightweight Automotive Materials Program (LAMP), a partnership between the National Center for Manufacturing Sciences, OSC, TotalSim, Nimbis Services and SimaFore.

PROJECT LEAD: Prasad Calyam, University of Missouri RESEARCH TITLE: Simulation-as-a-Service for advanced manufacturing FUNDING SOURCE: U.S. Department of Energy WEBSITE: http://osc.edu/press/team-enters-national-us-ignite-innovation-competition-wins-award-for-best-advanced

Industrial Engagement

41

Prototyping Fan Designs CFD design firm develops online simulation portal for auto industry supplier Despite the broad reach and the growth in computational fluid dynamics (CFD) tools and methods over the past two decades, the ability to access this technology remains outside the reach of many small and medium manufacturers (SMM) – the so-called “missing middle.”

On the heels of several federal projects designed to encourage the use of high performance computing in industry, the CFD design firm TotalSim is piloting a vertical application approach to the virtual prototyping process through funding secured from the Small Business Innovation Research (SBIR) program at the Department of Energy. The vertical application approach recognizes that complex systems are often made of several simpler components, each of which needs to be modeled and treated appropriately to capture the performance of the complete system. The TotalSim team has templated the modeling approach appropriate to each of these classes of flow, enabling these to be applied in isolation or in combination.

The prototype application – the Fan Portal – evolved from conversations with Horton Inc., a leading provider of advanced airflow management solutions and one of TotalSim’s existing customers.

Meeting a critical requirement for the development process, Horton engineers fully described to TotalSim their current engineering design and testing processes and were intimately involved in helping to shape the tool and test the application during the development process. The web-based interface was connected to powerful Ohio

Above: TotalSim’s Fan Portal accepts user input on parameters, such as fan geometry, blade number, shroud diameter, fan immersion, flow rate and speed and rotational direction and provides the customer with visual and numerical results, such as pressure, torque, power and efficiency.

Supercomputer Center systems, where the simulations model was built, run and post processed, returning back to the user pertinent performance metrics and pre-defined analysis data.

“The app was developed to replace an existing simulation method at a lower cost and a higher throughput, but it could very well have introduced simulation to an environment where none existed,” said Ray Leto, president of TotalSim. While the current portal is very specific to Horton’s specifications, it is easily extensible to nearly any other type of fan. For instance, TotalSim could leverage the portal, with very minimal changes, to designing fans for various other applications, such as for cooling computer chips or powering a boat.

“The majority of the gain from the deployment of this technology is to allow initial or expanded access to expert-level simulation tools for SMMs,” said Leto. “The combination of easy-to-use web-based portals, proven and provable performance and low entry price appeals to a broad range of small and mid-sized companies.”

PROJECT LEAD: Raymond Leto, TotalSim RESEARCH TITLE: Web-based CFD vertical applications using cloud-based HPC FUNDING SOURCE: U.S. Department of Energy WEBSITE: http://totalsim.us/about-us

42

OHIO SUPERCOMPUTER CENTER

Contacts Pankaj Shah Executive Director, Ohio Supercomputer Center and OARnet (614) 292-1486 • pshah@oh-tech.org Jamie Abel OH-TECH Communications Director (614) 292-6495 • jabel@oh-tech.org Alan Chalker, Ph.D. OSC Director of Technology Solutions Director of AweSim (614) 247-8672 • alanc@osc.edu Steven Gordon, Ph.D. OSC Senior Education Specialist (614) 292-4132 • sgordon@osc.edu Brian Guilfoos OSC Client and Technology Support Manager (614) 292-2846 • guilfoos@osc.edu Dave Hudak, Ph.D. OSC Program Director for Cyberinfrastructure and Software Development (614) 247-8670 • dhudak@osc.edu Dwayne Sattler Associate Vice President for Policy Office of Research – OH-TECH The Ohio State University (614) 292-2207 • sattler.29@osu.edu Karen Tomko, Ph.D. OSC Senior Researcher in Computer Science (614) 292-1091 • ktomko@osc.edu Kevin Wohlever Interim Director, OSC Supercomputing Operations Director, OH-TECH Shared Infrastructure (614) 247-2061 • kevin@oh-tech.org To contact other OSC staff members, please refer to the online directory at osc.edu/staff-directory.

Ohio Supercomputer Center 1224 Kinnear Road, Columbus, Ohio 43212 P: (614) 292-9248 F: (614) 688-3184 oschelp@osc.edu www.osc.edu twitter.com/osc facebook.com/ohiosupercomputercenter youtube.com/oscnewmedia linkedin.com/company/ ohio-supercomputer-center The Ohio Technology Consortium (OH-TECH), the statewide technology division of the Ohio Board of Regents, seeks to propel Ohio’s knowledge economy through the creation and adoption of next-generation technology and information solutions. The consortium delivers world-class technologies, information and expertise to provide Ohioans with a strong foundation for education and workforce, scientific research and business innovation.

Consortium members include the Ohio Supercomputer Center, OARnet, eStudent Services, OhioLINK and the in-development Research and Innovation Center. OH-TECH leverages the strengths of each of these organizations, enabling each to concentrate on its core mission.

The 2013 Research Report was written and designed by the OH-TECH communications and creative teams: Ian MacConnell, Susan Mantey, Jamie Abel, Jaynie McCloskey, Raquib Ahmed, Jameson Keener and Charaun Little. Barb Woodall and Dr. Alan Chalker supplied invaluable assistance in identifying and developing statewide research stories.

Numerous other staff members provided assistance, including: Brian Guilfoos, Don Stredney, Dr. David Hudak, Dr. Steve Gordon, Doug Johnson, Dr. Karen Tomko, Kevin Wohlever, Linda Flickinger and Brad Hittle.

OSC extends its gratitude to all the researchers featured in the preceding pages for sharing their precious time, collaborative spirit and, most of all, fascinating scientific achievements.

1224 Kinnear Road Columbus, Ohio 43212 (614) 292-9248 www.osc.edu