Bridges User Guide
Bridges User Guide: Data Collections
Community datasets are appropriate when data will be shared amongst Bridges’ groups. Any data that should only be accessed by one group should be stored in that group’s pylon5 space.
Public datasets
Some data collections are available to anyone with a Bridges’ account. They include:
2019nCoVR: 2019 Novel Coronavirus Resource
Last updated: August 27, 2020
The 2019 Novel Coronavirus Resource concerns the outbreak of novel coronavirus in Wuhan, China since December 2019. For more details about the statistics, metadata, publications, and visualizations of the data, please visit https://bigd.big.ac.cn/nco.
Available on Bridges at /pylon5/datasets/community/
COCO
COCO (Common Objects in Context) is a large scale image dataset designed for object detection, segmentation, person keypoints detection, stuff segmentation, and caption generation. Please visit http://cocodataset.org/ for more information on COCO, including details about the data, paper, and tutorials.
Available on Bridges at /pylon5/datasets/community/
PREVENT-AD
The PREVENT-AD (Pre-symptomatic Evaluation of Experimental or Novel Treatments for Alzheimer Disease) cohort is composed of cognitively healthy participants over 55 years old, at risk of developing Alzheimer Disease (AD) as their parents and/or siblings were/are affected by the disease. These ‘at-risk’ participants have been followed for a naturalistic study of the presymptomatic phase of AD since 2011 using multimodal measurements of various disease indicators. Two clinical trials intended to test pharmaco-preventive agents have also been conducted. The PREVENT-AD research group is now releasing data openly with the intention to contribute to the community’s growing understanding of AD pathogenesis.
Available on Bridges at /pylon5/datasets/community/
ImageNet
ImageNet is an image dataset organized according to WordNet hierarchy. See the ImageNet website for complete information.
Available on Bridges at /pylon5/datasets/community/imagenet
Natural Languge Tool Kit Data
NLTK comes with many corpora, toy grammars, trained models, etc. A complete list of the available data is posted at: http://nltk.org/nltk_data/
Available on Bridges at /pylon5/datasets/community/nltk
MNIST
Dataset of handwritten digits used to train image processing systems.
Available on Bridges at /pylon5/datasets/community/mnist
Genomics Data
Several genomics datasets are publicly available, in addition to the 2019nCoVR: 2019 Novel Coronavirus Resource listed above.
- BLAST
- The BLAST databases can be accessed through the environment variable $BLASTDB after loading the BLAST module.
- CAMI
- CAMI (Critical Assessment of Metagenome Interpretation) is a community-led initiative designed to help tackle challenges in metagenome assembly and analysis by aiming for an independent, comprehensive and bias-free evaluation of methods. Data from the first CAMI challenge is available at /pylon5/datasets/community/genomics/cami.
- RepBase
- Repbase is the most commonly used database of repetitive DNA elements. You must register with RepBase at http://www.girinst.org and send proof of registration to genomics@psc.edu in order to use the Repbase database.
- UCSC
- The University of California at Santa Cruz reference genomes are available at /pylon5/datasets/community/
genomics/UCSC. The collection includes human, mouse and drosophila genomes.
- Other genomics datasets
- Other available datasets are typically used with a particular genomics package. These include:
Barrnap | /pylon5/datasets/community/genomics/barrnap |
CheckM | /pylon5/datasets/community/genomics/checkm |
Dammit Dammit uniref90 |
/pylon5/datasets/community/genomics/dammit /pylon5/datasets/community/genomics/dammit_uniref90 |
Homer | /pylon5/datasets/community/genomics/homer |
Kraken | /pylon5/datasets/community/genomics/kraken |
Long Ranger | /pylon5/datasets/community/genomics/longranger |
MetaPhlAn2 | /pylon5/datasets/community/genomics/metaphlan2 |
Phylosift | /pylon5/datasets/community/genomics/phylosift |
Prokka | /pylon5/datasets/community/genomics/prokka |
Other useful datasets
A list of datasets that may be useful follows. These datasets are not currently installed on Bridges, but can be copied to your pylon5 space.
Deep Learning
Keras Datasets for Import
These datasets are available from https://keras.io/datasets/
- CIFAR10 small image classification
- CIFAR100 small image classification
- IMDB Movie reviews sentiment classification
- Reuters newswire topics classification
- MNIST database of handwritten digits
- Fashion-MNIST database of clothing
- Boston housing price regression dataset (from CMU)
Image Databases
- ImageNet
- MS-COCO
- The PASCAL Visual Object Classes Homepage
- Open Images Dataset V5
- The Street View House Numbers (SVHN)
Natural Language Processing
- Twenty Newsgroups
- Sentiment140
- WordNet
- Yelp Reviews
- The Wikipedia Corpus
- The Blog Authorship Corpus
- Natural Language Toolkit — NLTK 3.4.5
- DBpedia.org
Audio and Audio-Visual
- Free Spoken Digit Dataset
- Free Music Archive (FMA)
- Ballroom
- Million Song Dataset
- LibriSpeech
- VoxCeleb
Machine Learning
Scikit-Learn Datasets for Import
- Wisconsin Breast cancer – binary classification
- Iris – PCA, LDA, multi-class classification
- Wine – multi-class classification
- Boston house prices – regression
- Diabetes – regression
- Handwritten digits – image classification
Multi-class classification and clustering
Regression
Binary Classification
Univariate Time Series
Multivariate Time Series