Bridges User Guide: Data Collections

Community datasets are appropriate when data will be shared amongst Bridges’ groups.  Any data that should only be accessed by one group should be stored in that group’s pylon5 space.

Public datasets

Some data collections are available to anyone with a Bridges’ account.  They include:

2019nCoVR: 2019 Novel Coronavirus Resource

Last updated: August 27, 2020

The 2019 Novel Coronavirus Resource concerns the outbreak of novel coronavirus in Wuhan, China since December 2019. For more details about the statistics, metadata, publications, and visualizations of the data, please visit https://bigd.big.ac.cn/nco.

Available on Bridges at /pylon5/datasets/community/genomics/2019nCoVR.

COCO

COCO (Common Objects in Context) is a large scale image dataset designed for object detection, segmentation, person keypoints detection, stuff segmentation, and caption generation. Please visit http://cocodataset.org/ for more information on COCO, including details about the data, paper, and tutorials.

Available on Bridges at /pylon5/datasets/community/COCO

PREVENT-AD

The PREVENT-AD (Pre-symptomatic Evaluation of Experimental or Novel Treatments for Alzheimer Disease) cohort is composed of cognitively healthy participants over 55 years old, at risk of developing Alzheimer Disease (AD) as their parents and/or siblings were/are affected by the disease. These ‘at-risk’ participants have been followed for a naturalistic study of the presymptomatic phase of AD since 2011 using multimodal measurements of various disease indicators. Two clinical trials intended to test pharmaco-preventive agents have also been conducted. The PREVENT-AD research group is now releasing data openly with the intention to contribute to the community’s growing understanding of AD pathogenesis.

Available on Bridges at /pylon5/datasets/community/prevent_ad

ImageNet

ImageNet is an image dataset organized according to WordNet hierarchy.  See the ImageNet website for complete information.

Available on Bridges at /pylon5/datasets/community/imagenet

Natural Languge Tool Kit Data

NLTK comes with many corpora, toy grammars, trained models, etc. A complete list of the available data is posted at: http://nltk.org/nltk_data/

Available on Bridges at /pylon5/datasets/community/nltk

MNIST

Dataset of handwritten digits used to train image processing systems.

Available on Bridges at /pylon5/datasets/community/mnist

Genomics Data

Several genomics datasets are publicly available, in addition to the 2019nCoVR: 2019 Novel Coronavirus Resource listed above.

BLAST

The BLAST databases can be accessed through the environment variable $BLASTDB after loading the BLAST module.

CAMI

CAMI (Critical Assessment of Metagenome Interpretation) is a community-led initiative designed to help tackle challenges in metagenome assembly and analysis by aiming for an independent, comprehensive and bias-free evaluation of methods. Data from the first CAMI challenge is available at /pylon5/datasets/community/genomics/cami.

RepBase

Repbase is the most commonly used database of repetitive DNA elements. You must register with RepBase at http://www.girinst.org and send proof of registration to genomics@psc.edu in order to use the Repbase database.

UCSC

The University of California at Santa Cruz reference genomes are available at /pylon5/datasets/community/genomics/UCSC.  The collection includes human, mouse and drosophila genomes.

Other genomics datasets

Other available datasets are typically used with a particular genomics package.  These include:

Other useful datasets

A list of datasets that may be useful follows. These datasets are not currently installed on Bridges, but can be copied to your pylon5 space.

Deep Learning

Keras Datasets for Import

These datasets are available from https://keras.io/datasets/

• CIFAR10 small image classification
• CIFAR100 small image classification
• IMDB Movie reviews sentiment classification
• Reuters newswire topics classification
• MNIST database of handwritten digits
• Fashion-MNIST database of clothing
• Boston housing price regression dataset (from CMU)

Image Databases

• ImageNet
• MS-COCO
• The PASCAL Visual Object Classes Homepage
• Open Images Dataset V5
• The Street View House Numbers (SVHN)

Natural Language Processing

• Twenty Newsgroups
• Sentiment140
• WordNet
• Yelp Reviews
• The Wikipedia Corpus
• The Blog Authorship Corpus
• Natural Language Toolkit — NLTK 3.4.5
• DBpedia.org

Audio and Audio-Visual

• Free Spoken Digit Dataset
• Free Music Archive (FMA)
• Ballroom
• Million Song Dataset
• LibriSpeech
• VoxCeleb

Machine Learning

Scikit-Learn Datasets for Import

• Wisconsin Breast cancer – binary classification
• Iris – PCA, LDA, multi-class classification
• Wine – multi-class classification
• Boston house prices – regression
• Diabetes – regression
• Handwritten digits – image classification

Multi-class classification and clustering

• Abalone
• Car Evaluation
• Census Income
• Human Activity Recognition Using Smartphones

Regression

• Airport Delays
• Forest Fires – SVM
• Wine Quality

Binary Classification

• Heart Disease
• Bank Marketing

Univariate Time Series

• Monthly Sunspot
• Minimum Daily Temperatures

Multivariate Time Series

• EEG Eye State Data Set
• Occupancy Detection Data Set
• Ozone Level Detection Data Set