Tested new software that they developed on Beagle. This software enables more than a thousand fold speedup of simulation of molecular processes in living systems.

Aaron Dinner’s group

Yves Lussier’s group have been using Beagle to develop high-throughput methods for identifying novel microRNA-related targets for cancer therapy, identifying the genetic underpinning of complex disease polymorphisms and computationally identifying novel unique personal variant polymorphisms associated to a disease.
Yves Lussier’s group

Epilepsy is a disease that is still poorly understood. To understand its mechanics, the Van Drongelen lab has been working at performing realistic computational simulations that bridge the gap between the experimentally accessible individual behavior (~100 cells) and the clinically recorded aggregate behavior (~ 1M).
Read More about Epilepsy

An important next step in host-pathogen evolution studies is to incorporate models of within-host dynamics into existing between-host models, which is being studied by Stefano Allesina and Greg Dwyer at the University of Chicago.

Host-Pathogen Evolution Studies

Taeyoon Kim’s group explore how a cell contracts against an external matrix using a computational model incorporating actin filaments, actin cross-linking proteins, and molecular motors. Results illustrate a novel mechanism for rigidity-sensing, which depends on the distance a motor walks along an actin filament to generate contractile force.
Taeyoon Kim’s group

Megan Puckelwartz & Lorenzo Pesce developed Megaseq which produces more usable sequence per genome from the same raw data. Megaseq relies on publicly-available software packages, and using MegaSeq WGA of as many as 240 genomes, from fastq extraction through variant calling, can be completed in approximately 50 hours.
Elizabeth McNally’s group

Complex diseases like cancer are rarely caused by an abnormality in a single gene, but rather reflect perturbations to intracellular molecular interaction networks that attract cells to new malignant and carcinogenic states.
Yuan Ji's group

Beagle Status
Director
Specs
Operations
Focus

	Jan 23	Jan 22	Jan 21	Jan 20	Jan 19
Current status
Jobs Running	68	69	69	77	69
Cores in Use	21952	22400	22400	22168	22400
/lustre/beagle Usage	70%	70%	69%	67%	65%
/lustre/beagle2 Usage	18%	18%	18%	18%	18%

	System is operating at peak performance.
	Scheduled maintenance: occurring at 7:00 AM on the first Tuesday of every month.
	Beagle Upgrade

Beagle2 News

All compute node CPUs upgraded from Magny-Cours to Abu Dhabi processors. With this upgrade, core count has been increased from 24 to 32 cores per node.
All compute node RAM DIMMs replaced. Available RAM has been increased from 32GB to 64GB per node.
Four nVidia GPU accelerator nodes have been added.
Two additional DDN storage cabinets installed to provide new Lustre scratch storage. An additional 1.4 PB of usable space has been added with that upgrade.
SMW and CLE system software upgraded from 7.0 to 7.2 and 4.2 to 5.2 respectively.
Development environment tools (Cray compiler, GNU complier) updated.
Lustre software updated from version 1.8 to 2.5. The issue that prevented group permissions from working properly on Lustre has now been fixed as well.

We had also planned to upgrade the network cards in the login nodes to 10Gbps connections, but have delayed that until January 2015. That upgrade will take place after ANL finishes planned upgrades to their network infrastructure taking place over the next few weeks.

With these changes, you will need to carefully reconsider and modify any and all submit scripts you have previous used on Beagle. These changes will affect the switches you use with aprun in your submit script, but you should spend some time generally thinking about how the increased hardware capabilities affect how you submit and run your jobs. Our wiki should now be updated to reflect these changes, but if you have any questions or concerns please let us know.

We’re also still in the process of moving data from the old Lustre volume to the new Lustre volume. By limiting access at first we’ll able to prioritize copying the data of users who need early access while we continue copying the data in the background.
If you want to be considered for early access or have any questions or concerns of any kind, please let us know at beagle-support@ci.uchicago.edu and we’ll do our best to answer ASAP.

As always, we appreciate your patience during this upgrade period.

Director

The acquisition of the Beagle supercomputer was made possible by a grant from the National Institutes of Health (NIH) National Center for Research Resources (NCRR).

Ian Foster, director of the Computation Institute at the University of Chicago and Argonne National Laboratory, is the PI for this project. Ian Foster, with UChicago’s team of technical and domain specialists, identified the need for a powerful computational environment that would serve the growing resource-intensive requirements of the biomedical research community.

Beagle’s “skin” was created by the Computation Institute’s Mark Hereld and Greg Cross. Beagle 2011 is built on three components: water and sky are divided by a wave. Moving to the right, the wave takes on the pitch of the double helix of DNA. The images of water and sky are generated by a stochastic, context-free grammar using a computer. This application of stochastic image generation gives Beagle 2011 a fractal aspect that combines visual elements inspired by biology and mathematics, disciplines at the heart of the research that Beagle will carry forward.

System Specifications

Ranks among 500 fastest machines (Dec. 2015)
Peak Performance: 212 TFlops/s
Cray XE6 system
CPU: AMD Operton 6300 series (model 6380)

64GB per node * 724 compute nodes = 46336GB

Max memory bandwidth for the Opteron 6380 is 102.4 GB/s
GPU: NVIDIA Tesla K20

32GB per node * 4 GPU compute nodes = 128GB

Total RAM on compute nodes: 46336 + 128 = 46464GB
728 compute nodes / 4 nodes per blade = 182 blades

Extreme Scalability Mode (ESM), which supports large scalable custom applications.
Cluster Compatibility Mode (CCM), which allows standard programs designed for smaller machines or clusters to run without modifications.
The nodes are connected in a 3D torus topology via the Cray Gemini interconnect.
A high-speed inter-processor connection network to support tightly coupled computational simulation and data-intensive analysis applications that involve frequent inter-process communication.
At least 32 Gigabyte memory per compute node, for applications that create large in-memory data structures or that will run many tasks on the same node.
The ability to easily and quickly schedule large jobs as data become available while being able to pursue a very large number of smaller tasks.

Beagle2 Operations

Beagle2 is generally available to its users 24 hours a day, seven days a week, excepting regularly scheduled preventative maintenance windows occurring at 7:00 AM on the first Tuesday of every month. The length of the outage window varies depending on the work to be done, but can be as long as 48 hours (though not typically). Two systems administrators provide immediate support during regular operating hours (9:00 AM – 5:00 PM, Monday through Friday), and provide emergency support during off-hours. We use the RT trouble ticket system that allows us to handle, track and audit user support questions. We also use other technologies such as blogs, wikis, and instant messaging to provide fast and up to date communication.

For account and project management, Beagle2 uses the Userbase system, a consolidated accounts and project management system used at both Argonne and the University. This system makes it easy to migrate users, groups and projects to the Beagle system already in use on other HPC resources at both sites.

We generate monthly internal usage reports and quarterly and annual usage reports for the user community. We provide, on request, weekly or monthly reports specific for each user, project, or PI for their own usage tracking. These targeted reports are useful for allocation forecasting as well as job optimization.

We handle configuration management using Cray’s xtopview utility, and we use Nagios combined with the HSS to monitor system health and alert the support staff of problems, and Nessus for periodic scanning of the system for vulnerabilities. We use Ganglia for performance and utilization monitoring allowing users to see in real time the cluster utilization and hot spots.

Focus Areas

Beagle2 will focus — but not exclusively — on biomedical research supported by NIH funding.
Some of the project areas include:

Quantitative determination of free energies associated with large conformational changes in cell membranes
Molecular structure and ligand interaction prediction in cellular networks
Whole-body model for studies of electrical and thermal injury
Computation of possible configurations of transcriptional networks
Data-mining of biomedical literature to understand regulatory networks in cancer and to understand complex disease processes

Mapping brain structure to human behavior
Quantitative medical-image analysis
High volume text-mining
Genomic and metagenomic data analysis
Modeling of economic impact of climate change
Large scale molecular dynamics
Model ion channels in nerve cells
Study transcriptional networks