United States agricultural researchers have many options for making their data available online. This dataset aggregates the primary sources of ag-related data and determines where researchers are likely to deposit their agricultural data. These data serve as both a current landscape analysis and also as a baseline for future studies of ag research data.

Purpose

As sources of agricultural data become more numerous and disparate, and collaboration and open data become more expected if not required, this research provides a landscape inventory of online sources of open agricultural data.

An inventory of current agricultural data sharing options will help assess how the Ag Data Commons, a platform for USDA-funded data cataloging and publication, can best support data-intensive and multi-disciplinary research. It will also help agricultural librarians assist their researchers in data management and publication. The goals of this study were to

establish where agricultural researchers in the United States-- land grant and USDA researchers, primarily ARS, NRCS, USFS and other agencies -- currently publish their data, including general research data repositories, domain-specific databases, and the top journals
compare how much data is in institutional vs. domain-specific vs. federal platforms
determine which repositories are recommended by top journals that require or recommend the publication of supporting data
ascertain where researchers not affiliated with funding or initiatives possessing a designated open data repository can publish data

Approach

The National Agricultural Library team focused on Agricultural Research Service (ARS), Natural Resources Conservation Service (NRCS), and United States Forest Service (USFS) style research data, rather than ag economics, statistics, and social sciences data. To find domain-specific, general, institutional, and federal agency repositories and databases that are open to US research submissions and have some amount of ag data, resources including re3data, libguides, and ARS lists were analysed. Primarily environmental or public health databases were not included, but places where ag grantees would publish data were considered.

Search methods

We first compiled a list of known domain specific USDA / ARS datasets / databases that are represented in the Ag Data Commons, including ARS Image Gallery, ARS Nutrition Databases (sub-components), SoyBase, PeanutBase, National Fungus Collection, i5K Workspace @ NAL, and GRIN. We then searched using search engines such as Bing and Google for non-USDA / federal ag databases, using Boolean variations of agricultural data /ag data / scientific data + NOT + USDA (to filter out the federal / USDA results). Most of these results were domain specific, though some contained a mix of data subjects.

We then used search engines such as Bing and Google to find top agricultural university repositories using variations of agriculture, ag data and university to find schools with agriculture programs. Using that list of universities, we searched each university web site to see if their institution had a repository for their unique, independent research data if not apparent in the initial web browser search. We found both ag specific university repositories and general university repositories that housed a portion of agricultural data. Ag specific university repositories are included in the list of domain-specific repositories. Results included Columbia University International Research Institute for Climate and Society, UC Davis Cover Crops Database, etc. If a general university repository existed, we determined whether that repository could filter to include only data results after our chosen ag search terms were applied. General university databases that contain ag data included Colorado State University Digital Collections, University of Michigan ICPSR (Inter-university Consortium for Political and Social Research), and University of Minnesota DRUM (Digital Repository of the University of Minnesota). We then split out NCBI (National Center for Biotechnology Information) repositories.

Next we searched the internet for open general data repositories using a variety of search engines, and repositories containing a mix of data, journals, books, and other types of records were tested to determine whether that repository could filter for data results after search terms were applied. General subject data repositories include Figshare, Open Science Framework, PANGEA, Protein Data Bank, and Zenodo.

Finally, we compared scholarly journal suggestions for data repositories against our list to fill in any missing repositories that might contain agricultural data. Extensive lists of journals were compiled, in which USDA published in 2012 and 2016, combining search results in ARIS, Scopus, and the Forest Service's TreeSearch, plus the USDA web sites Economic Research Service (ERS), National Agricultural Statistics Service (NASS), Natural Resources and Conservation Service (NRCS), Food and Nutrition Service (FNS), Rural Development (RD), and Agricultural Marketing Service (AMS). The top 50 journals' author instructions were consulted to see if they (a) ask or require submitters to provide supplemental data, or (b) require submitters to submit data to open repositories.

Data are provided for Journals based on a 2012 and 2016 study of where USDA employees publish their research studies, ranked by number of articles, including 2015/2016 Impact Factor, Author guidelines, Supplemental Data?, Supplemental Data reviewed?, Open Data (Supplemental or in Repository) Required? and Recommended data repositories, as provided in the online author guidelines for each the top 50 journals.

Evaluation

We ran a series of searches on all resulting general subject databases with the designated search terms. From the results, we noted the total number of datasets in the repository, type of resource searched (datasets, data, images, components, etc.), percentage of the total database that each term comprised, any dataset with a search term that comprised at least 1% and 5% of the total collection, and any search term that returned greater than 100 and greater than 500 results.

We compared domain-specific databases and repositories based on parent organization, type of institution, and whether data submissions were dependent on conditions such as funding or affiliation of some kind.

Results

A summary of the major findings from our data review:

Over half of the top 50 ag-related journals from our profile require or encourage open data for their published authors.
There are few general repositories that are both large AND contain a significant portion of ag data in their collection. GBIF (Global Biodiversity Information Facility), ICPSR, and ORNL DAAC were among those that had over 500 datasets returned with at least one ag search term and had that result comprise at least 5% of the total collection.
Not even one quarter of the domain-specific repositories and datasets reviewed allow open submission by any researcher regardless of funding or affiliation.

See included README file for descriptions of each individual data file in this dataset.

US Department of Agriculture

Department of Agriculture Congressional Logs for Fiscal Year 2014

Department of Agriculture Enterprise Data Inventory

Department of Agriculture Secretary's Calendar Schedule

Realized Cost Savings and Avoidance

USDA Active Purchase Card Holders

USDA Annual FOIA Report

USDA Bureau IT Leadership Directory

USDA Governance Boards

USDA Help Desk Support Data Asset

USDA IT Policy Archive

USDA Purchase Card Transaction Report

ARMS Farm Financial and Crop Production Practices

Adoption of Genetically Engineered Crops in the U.S.

Ag and Food Statistics: Charting the Essentials

Agricultural Baseline Database

Agricultural Exchange Rate Data Set

Agricultural Productivity in the U.S.

Agricultural Research Funding in the Public and Private Sectors

Agricultural Trade Multipliers

Aquaculture Data

Atlas of Rural and Small-Town America

Best of Charts of Note 2013

Best of Charts of Note 2014

Best of Charts of Note 2015

Best of Charts of Note 2016

Best of Charts of Note 2017

Chart Gallery

Charts of Note

China Agricultural and Economic Data

Commodity Consumption by Population Characteristics

Commodity Costs and Returns

Commodity and Food Elasticities

Commuting Zones and Labor Market Areas

Cost Estimates of Foodborne Illnesses

Cotton and Wool Chart Gallery

Cotton, Wool, and Textile Data

County Typology Codes

County-level Data Sets

County-level Oil and Gas Production in the U.S.

Creative Class County Codes

Dairy Data

Data Visualizations

ERS Data APIs

Eating and Health Module (ATUS)

Editors' Pick Charts of Note 2018

Editors' Pick: Charts of Note 2019

Farm Household Income and Characteristics

Farm Income and Wealth Statistics

Federal Funds

Feed Grains Database

Fertilizer Imports/Exports

Fertilizer Use and Price

Food Access Research Atlas

Food Availability (Per Capita) Data System

Food Consumption and Nutrient Intakes

Food Dollar Series

Food Environment Atlas

Food Expenditure Series

Food Price Outlook

Food Security in the United States

Food and Nutrition Assistance Research Reports Database

FoodAPS National Household Food Acquisition and Purchase Survey

Foreign Agricultural Trade of the United States (FATUS)

Frontier and Remote Area Codes

Fruit and Tree Nuts Data

Fruit and Vegetable Prices

International Agricultural Productivity

International Baseline Data

International Food Consumption Patterns

International Food Security

International Macroeconomic Data Set

Irrigated Agriculture in the United States

Livestock & Meat Domestic Data

Livestock and Meat International Trade Data

Major Land Uses

Meat Price Spreads

Milk Cost of Production Estimates

Natural Amenities Scale

Normalized Prices