T3/Breedbase Workshop

:calendar: September 8th - 9th, 2020
:email: David Waring: djw64@cornell.edu

T3/Breedbase Production Databases:


T3/Breedbase Sandbox Databases:


Phenotype Upload Instructions:


Germplasm Search Tool:

Blank Upload Templates

:paperclip: Location Template
:paperclip: Accession Template
:paperclip: Trial Template (trial information and plot layout)
:paperclip: Phenotype Template (trait observations)

Trait Lookup Tables

:paperclip: Wheat Trait Lookup Table
:paperclip: Barley Trait Lookup Table
:paperclip: Oat Trait Lookup Table

Day One

What is T3?

T3 is The Triticeae Toolbox:

Overview of T3/Breedbase

What is Happening?

Current Status

What is Breedbase?

Why is T3 transitioning?

Workshop Goals

Demonstrate how to upload field trials

Demonstrate Breedbase integration into a breeding program

Try uploading your own data

Wrap-Up Discussion | Day Two

Phenotype Submission Workflow

Database Organization

Each crop has a separate production and sandbox database.

Submission Overview

Two ways to submit data:

Instructions

Detailed upload instructions are available on the sandbox database for each crop:

Demo Data

Below is an excerpt from and a link to the demo trial data I’ll be adding to the database:

PlotLineRowColumnRepBlockgrain yield (kg/ha)test weight (g/L)FHB Incidence (%)FHB Severity (%)DON Level (ppm)
101VA14W-9911111984.58739.0153.158.911.13
102NC14-2337212112044.19756.1535.598.081.75
103VA13W-3813112356.95787.5636.0220.781.52
104NC15-2567214112061.00752.5469.9220.642.25
105NC14-2337315111535.94769.6078.599.401.38

:paperclip: Complete Demo Data

The demo data contains observations from two locations: Ithaca, NY and Geneva, NY. There are ten lines sampled in three reps at each location with five traits observed: grain yield, test weight, FHB incidence, FHB severity, and DON level.

Create an Account

Each crop has a separate production and sandbox database. These are separate websites and copies of the database and as such have separate account management systems. You’ll need to create an account on each database you want access to.

You can create an account by clicking the New User button in the top-right of the toolbar. After filling out the registration form you should receive an account confirmation email from noreply@graingenes.org (check your Junk/Spam folder if you don’t see it in your Inbox) with a link to confirm your account. Once your account has been confirmed you can log in to the website.

Accounts created on a sandbox database will be automatically granted ‘submitter’ privileges. Accounts created on a production database will have read-only access.

Locations

Use Case: You have trials that were located in two different towns. Are these locations already in the database? How do you add a new location?

View Existing Locations

You can view the locations that are already in the database by going to the Manage > Locations page. Here you’ll see a table and a map of all existing locations to search for your trial locations.

Add New Locations

If the location of your trial is not already in the database, there are two ways to add new locations from the Manage > Locations page:

Location Template

A location template includes the following information:

NameAbbreviationCountry CodeCountry NameProgramTypeLatitudeLongitudeAltitudeNOAA Station ID
Geneva, NYGENNYUSAUnited States of AmericaCornell UniversityField42.880173-77.009083187GHCND:USC00303177

:paperclip: Blank Location Template
:paperclip: Demo Location Template

:information_source: If you’re using R to extract your data to generate the breedbase templates, you can use the breedbase R package to query an API to get the lat, lon, altitude and NOAA Station ID of locations based on street addresses.

Accessions

Use Case: You have a list of lines that were phenotyped in your trials. Are these lines already in the database? How do you update existing and add new lines?

Germplasm Search Tool

The Germplasm Search Tool is a separate tool that queries the breedbase database for germplasm lines and matches your line names up with existing database entries using a number of search methods. It is available under the Search menu as Bulk Accession Search. Links for each crop are:

The Germplasm Search Tool can be used to check the experimental list of lines for existing matches in the database and identify lines that:

The ten lines in the demo data are named:

BESS
ERNIE
LES15-5499
LES15-5605
NC14-23372
NC14-23373
NC15-25672
VA13W-174
VA13W-38
VA14W-99

There are two lines that will need to be added as Accessions in the database:

NC15-25672
VA14W-99

Accession Template

The Accession upload template contains properties about the germplasm lines being added to the database. If an entry exists with the same name in the database, the properties will be updated with the ones in the template. At a minimum, the accesion_name and species_name properties are required.

An Accession template can include the following information:

accession_namespecies_namepopulation_nameorganization_name(s)synonym(s)variety(s)country_of_origin(s)notes(s)accession_number(s)purdy_pedigreefilial_generation
NC15-25672Triticum aestivumNorth Carolina State UniversityNC15_25672PI 1234NC15-1/NC15-29
VA14W-99Triticum aestivumVirginia TechVA14W 99CItr 5678VA14W-1/VA14W-29

:paperclip: Blank Accession Template
:paperclip: Demo Accession Template

Add New Accessions

To upload an Accession template:

  1. Go to the Manage > Accessions page
  2. Click the Add Accessions Or Upload Accession Info link near the top right corner of the page
  3. Select the Uploading a File tab
  4. Click the Choose File button to select your upload template
  5. Click the Continue button and follow the prompts

View Accessions

Once your upload has been processed, you can find your new Accession(s) by going to the Search > Accessions and Plots page and entering the name of one of your Accessions.

Click the name of your Accession in the results table to open the Accession detail page. Here the Additional Info section should include the property values from your upload template.

Trials

Use Case: You have one or more trials that were previously designed. You know the plot layout of the trials. How do you add the trials to the database?

There are multiple methods of adding a trial to the database. For this example, we’ll be adding an existing trial by uploading a trial template file (not using the trial design tool). There are two trial template types:

Trial Properties

The following properties relate to the trial:

Plot Properties

The following properties relate to a plot:

Trial Template

The trial upload template combines the trial properties (which are repeated for each plot) and the plot properties.

trial_namebreeding_programlocationyeardesign_typedescriptiontrial_typeplot_widthplot_lengthfield_sizeplanting_dateharvest_dateplot_nameaccession_nameplot_numberblock_numberis_a_controlrep_numberrange_numberrow_numbercol_numberseedlot_namenum_seed_per_plotweight_gram_seed_per_plot
AYT_2019_IthacaCornell UniversityIthaca, NY2019RCBD2019 Advanced Yield Trial @ IthacaAdvanced Yield Trial112019-06-152019-10-11AYT_2019_Ithaca-PLOT101VA14W-991011111
AYT_2019_IthacaCornell UniversityIthaca, NY2019RCBD2019 Advanced Yield Trial @ IthacaAdvanced Yield Trial112019-06-152019-10-11AYT_2019_Ithaca-PLOT102NC14-233721021112
AYT_2019_IthacaCornell UniversityIthaca, NY2019RCBD2019 Advanced Yield Trial @ IthacaAdvanced Yield Trial112019-06-152019-10-11AYT_2019_Ithaca-PLOT103VA13W-381031113
AYT_2019_IthacaCornell UniversityIthaca, NY2019RCBD2019 Advanced Yield Trial @ IthacaAdvanced Yield Trial112019-06-152019-10-11AYT_2019_Ithaca-PLOT104NC15-256721041114
AYT_2019_IthacaCornell UniversityIthaca, NY2019RCBD2019 Advanced Yield Trial @ IthacaAdvanced Yield Trial112019-06-152019-10-11AYT_2019_Ithaca-PLOT105NC14-233731051115

:paperclip: Blank Trial Template
:paperclip: Demo Trial Template

Add Trials

To add the trial(s) to the database:

  1. Go to the Manage > Field Trials page
  2. Click the Upload Existing Trial(s) button near the top right of the page
  3. On Step 2, select the Multiple Trial Designs tab
  4. Click the Choose File button to select your upload template
  5. Click the Upload Trial Designs button and follow the prompts

View Trial

You can view the trial(s) you added by finding them in the table on the Search > Field Trials page (you can filter the table by entering the trial name in the search box in the top right corner of the table).

You can open the trial detail page by clicking on the name of the trial in the table. From this page you can check the properties of the trial. The Field Layout Tools and Phenotype Heatmap section contains a generated field layout with the plot positions. The Experimental Design section contains information on the field design and the accessions used in the trial.

Traits

Use Case: You have a list of traits that were observed in your trials. Are these traits available in the database? How do you find the traits? What do you call the traits when uploading data?

Breedbase Trait Management

Breedbase manages traits differently than we did with the original version of T3. Trait management may seem more complicated, but this allows us to more accurately and more precisely define the traits. In addition, trait observations can now be more easily compared to data stored in other databases that use the same trait definitions.

On the original T3, there was a T3 trait dictionary that was a list of trait names with descriptions that defined each individual trait. Trait observations were added to the database by linking each trait value with the T3 trait name.

On breedbase, traits are organized in a trait ontology, which is a controlled vocabulary of trait terms and definitions. The ontology is managed in a collaborative effort through The Generation Challenge Programme’s Crop Ontology project and is made publicly available online on their website. T3 has contributed trait definitions for all three of our crops:

The trait ontology is organized in a tree-like hierarchical structure. Under the root of the tree are trait categories - these are broad categories for grouping similar traits (such as Agronomic, Biotic Stress, Quality, etc). Directly under the trait categories are ontology trait terms, where a trait (in ontology terms) is the entity that is being measured (such as grain yield) but does not include any information about how it is measured or with what units. Directly under the trait is one or more ontology variable terms. The variable term contains information for a single trait as well as information about the method in which the trait was measured and the units/scale the value is recorded in. Observations made from a phenotyping trial must always be associated with a trait ontology variable.

View Traits

To view the traits in the trait ontology, open the ontology browser by going to the Manage > Trait Ontology Browser page. This page will display all of the ontologies loaded into the database - the trait ontology will be labeled Wheat traits, Barley Traits, or Oat traits. Direct links to the trait ontology for each crop are:

Here you can navigate through the ontology tree to find the ontology variable that matches your trait.

Search Traits

Traits can also be searched by name by going to the Search > Traits page. Make sure the trait ontology is selected in the search settings and enter a term to search for.

From the results table, you can choose a term that has a type set to VARIABLE_OF to associate with your trait data.

Trait Usage

When loading your phenotype observations you’ll need to know both the trait name and trait id of the trait variable that matches your observed trait. For example, for wheat grain yield in kg/ha you’ll need to know:

Trait Lookup Tables

If you’re familiar with the old trait names used on T3/Classic, you can use the lookup tables that link the old trait names to their corresponding breedbase trait variable names and IDs. There is a trait lookup table available for each crop:

:paperclip: Wheat Trait Lookup Table
:paperclip: Barley Trait Lookup Table
:paperclip: Oat Trait Lookup Table

Requesting New Traits

If there is no corresponding trait variable that matches your observed trait, you can request to have a new trait added to the ontology. You can either contact us directly or use the trait request form.

If your trait only differs in the units/scale that was measured, we ask that you convert your values to match the scale of an existing trait variable.

Phenotypes

Use Case: You have phenotype observations from one or more trials that have already been added to the database. How do you add the observations to the trials?

We’ll be adding phenotype observations to the new trials we created by creating a phenotype upload template. We’ll be creating the template manually, but a blank template with the proper trait column headers can be created by the website (from the trial detail page) if you have the traits you observed already in a list.

Create Phenotype Upload Template

We’ll be using the simple phenotyping spreadsheet format. This format has one required column for the plot name followed by a column for each observed trait. The column headers are:

Below is an excerpt of the trait observations from the demo dataset:

observationunit_nameGrain yield - kg/ha|CO_321:0001218Grain test weight - g/l|CO_321:0001210FHB incidence - %|CO_321:0001149FHB severity - %|CO_321:0001440FHB DON content - ppm|CO_321:0001154
AYT_2019_Ithaca-PLOT1011984.58739.0153.158.911.13
AYT_2019_Ithaca-PLOT1022044.19756.1535.598.081.75
AYT_2019_Ithaca-PLOT1032356.95787.5636.0220.781.52
AYT_2019_Ithaca-PLOT1042061.00752.5469.9220.642.25
AYT_2019_Ithaca-PLOT1051535.94769.6078.599.401.38

:paperclip: Blank Trial Template
:paperclip: Demo Trial Template

:information_source: Multiple trials can be uploaded at once with the same phenotype upload template.

Upload Phenotype Template

To upload the phenotype template:

  1. Go to the Manage > Phenotyping Results page
  2. Click the Upload Spreadsheet link near the top right corner of the page
  3. Select Simple for the Spreadsheet Format
  4. Click the Choose File button to select your template
  5. Click the Verify button to check the format of the template
  6. Click the Store button to store the data in the database

View Phenotype Data

You can view the trait observations from the trial detail page (find your trial in the Search > Field Trials page and click the trial name to get to the trial detail page).

In the Phenotype Summary Statistics section you can view a table of trait means and summary statistics and a histogram of trait observations for the trial. Here you can also download a table of all of the trait observations for the trial.

Day Two

Discussion of Day One

Common Problems

Submitting Trials

Once you have your trial(s) up on the sandbox, you can submit them to be included on the production database.

From the trial detail page, there is a Submit Trial button near the top of the page. This will tell us that the trial is ready for submission. We will review it and add it to the production database.

Available Shortcuts

Website Tools

There are tools available on the website that can help generate some of the templates we created manually.

Android Fieldbook

Breedbase is tightly integrated with the Android Fieldbook app which can be used to collect phenotype data on an Android tablet or phone directly in the field. The website can be used to create the field layout files that are used to load your field trial into the app. The data files created by the app can then be loaded directly into Breedbase to store the phenotype observations.

Breedbase R Package

If you’re using R to interact with your data, you can use the Breedbase R package to generate the upload templates.

Analysis Tools Overview

Trial Summary Tool

This is a tool that was available on T3/Classic and ported over to breedbase. It can be used to generate LSMeans and LSDs for one or more traits across one or more trials. It is accessible from the Analyze > Summarize Trials menu.

To use the tool, you will first need to have the trials you are interested in summarizing in a list.

Trial Analysis Tools

These tools are included with breedbase and are available in the Analysis Tools section of a trial detail page.

Breedbase Analysis Tools

These tools are built in to breedbase and could be made available.

Some of these tools are not yet available because they require some work from us to get them up and running. If you are interested in using any of these tools, please reach out to us and let us know. This will allow us to prioritize setting up tools that we know people are interested in using.