Accessing ERDDAP data with R#
The following notebook is a quick demostration on how to use R to access the ERDDAP server data. This is designed for users that prefer the programing interface to be R. There are already couple of great resource related to preprocessing and visualizing data in R. Therefore, the notebook will not repeat that part of the code but focusing on how to accessing the data from a R interface. The resources is listed below
http://cran.nexr.com/web/packages/rerddap/vignettes/Using_rerddap.html (A detail introduction of rerddap package and show the visualization in many different datasets)
https://ioos.github.io/ioos_code_lab/content/code_gallery/data_access_notebooks/2017-08-01-xtractoR.html (use a different package called xtractomatic to access the ERDDAP data)
Launch R in Jupyterlab
In addition to the standard R code demonstrated below, our current interface leverages JupyterLab to execute R code using the R kernel. To enable this functionality, we utilize the environment.yml file to create a Conda/Mamba environment. The primary objective is to install the r-irkernel package within this environment.This installation of r-irkernel through Conda/Mamba ensures that the R kernel becomes available as an option when launching JupyterLab. Selecting the R kernel empowers users to utilize the JupyterLab interface for running R code effortlessly.
Packages used#
To use R to access the ERDDAP server directly from your R script or R-notebook, you will need
rerddap
There are couple of ways to install the rerddap package, here we provide two popular ways below
In the R environment,
install.packages('rerddap')In the conda environment for different package version control,
conda install r-rerddap(many r package can be install this way by adding “r-” in front of the package name)
Import R packages#
The way to import packages to the current R environment is to use the require or library functions
Both functions will import the package but with subtle different of
The require() and library() functions can both be used to load packages in R, but they have one subtle difference:
require() will output a warning if a package is not installed and then continue to execute the code.
library() will output an error and stop the execution of the code.
Detail explanation cna be find here
library("rerddap")
Direct ERDDAP server based search of datasets#
The rerddap::ed_search helps user search for data directly on a ERDDAP server. It give user an output of all dataset full names and the associated dataset_id which is important for the downloading. Here we demostrate the ed_search function with query of sea surface temperature (sst) dataset.
whichSST <- rerddap::ed_search(query = "sst")
whichSST
# A tibble: 980 × 2
title dataset_id
<chr> <chr>
1 COBE SST Analysis (sst.mon.ltm.1981-2010), 1.0°, 0001 noaa_psl_…
2 COBE SST Analysis (sst.mon.ltm.1991-2020), 1.0°, 0001 noaa_psl_…
3 COBE SST Analysis (sst.mon.mean), 1.0°, 1891-present noaa_psl_…
4 HadISST Average Sea Surface Temperature, 1°, Global, Monthly, 187… erdHadISST
5 HadISST Average Sea Surface Temperature, 1°, Global, Monthly, 187… erdHadISS…
6 HadISST Sea Ice Component, 1°, Global, Monthly, 1870-present erdHadISS…
7 HadISST Sea Ice Component, 1°, Global, Monthly, 1870-present, Lon… erdHadISS…
8 NOAA ERSSTv4 (in situ only), 2°, Global, Monthly, 1854-2020 nceiErsst…
9 NOAA ERSSTv4 (in situ only), 2°, Global, Monthly, 1854-2020, Lon+… nceiErsst…
10 NOAA ERSSTv5 (in situ only) (sst.mnmean), 2.0°, 1854-present noaa_psl_…
# ℹ 970 more rows
If the url argument is not specified, the search will be the default ERDDAP server which is located at NOAA NMFS SWFSC ERD. To switch from the default server to a specific server, one can set the server url manually in one of the argument as the following.
whichSST2 <- rerddap::ed_search(query = "sst", url = "https://erddap.ifremer.fr/erddap/")
Once the url is changed to other server, a different set of SST dataset can been seen below.
whichSST2
# A tibble: 1 × 2
title dataset_id
<chr> <chr>
1 Daily MUR SST, Final product SST_Anomalies_Caledonie
Information about the dataset#
For a quick peek of the data meta data, one can also use the info function
info('SST_Anomalies_Caledonie',url = "https://erddap.ifremer.fr/erddap/")
<ERDDAP info> SST_Anomalies_Caledonie
Base URL: https://erddap.ifremer.fr/erddap
Dataset Type: griddap
Dimensions (range):
time: (2014-01-16T09:00:00Z, 2021-12-16T09:00:00Z)
latitude: (-27.0, -14.01)
longitude: (155.01, 175.01)
Variables:
analysed_sst:
Units: kelvin
Subsetting and download to local memory#
Here we focus on downloading a gridded dataset in the default ERDDAP server and subsetting it in the notebook.
sstInfo <- info('nceiErsstv5_LonPM180')
erSST <- griddap(sstInfo, latitude = c(22., 51.), longitude = c(-140., -105), time = c("2017-01-01", "2017-01-02"), fields = 'ssta')
info() output passed to x; setting base url to: https://upwell.pfeg.noaa.gov/erddap
erSST
The data can be viewed or used in the form of a data frame in R as following
erSST$data
| longitude | latitude | depth | time | ssta |
|---|---|---|---|---|
| <dbl[1d]> | <dbl[1d]> | <dbl[1d]> | <chr> | <dbl> |
| -140 | 22 | 0 | 2017-01-15T00:00:00Z | 0.5141125 |
| -138 | 22 | 0 | 2017-01-15T00:00:00Z | 0.5668831 |
| -136 | 22 | 0 | 2017-01-15T00:00:00Z | 0.6465950 |
| -134 | 22 | 0 | 2017-01-15T00:00:00Z | 0.7129841 |
| -132 | 22 | 0 | 2017-01-15T00:00:00Z | 0.7436638 |
| -130 | 22 | 0 | 2017-01-15T00:00:00Z | 0.7662201 |
| -128 | 22 | 0 | 2017-01-15T00:00:00Z | 0.8154736 |
| -126 | 22 | 0 | 2017-01-15T00:00:00Z | 0.8591919 |
| -124 | 22 | 0 | 2017-01-15T00:00:00Z | 0.8391647 |
| -122 | 22 | 0 | 2017-01-15T00:00:00Z | 0.7444363 |
| -120 | 22 | 0 | 2017-01-15T00:00:00Z | 0.6146202 |
| -118 | 22 | 0 | 2017-01-15T00:00:00Z | 0.4916267 |
| -116 | 22 | 0 | 2017-01-15T00:00:00Z | 0.3766174 |
| -114 | 22 | 0 | 2017-01-15T00:00:00Z | 0.3021393 |
| -112 | 22 | 0 | 2017-01-15T00:00:00Z | 0.2422123 |
| -110 | 22 | 0 | 2017-01-15T00:00:00Z | 0.2063828 |
| -108 | 22 | 0 | 2017-01-15T00:00:00Z | 0.2279358 |
| -106 | 22 | 0 | 2017-01-15T00:00:00Z | 0.2696514 |
| -104 | 22 | 0 | 2017-01-15T00:00:00Z | NaN |
| -140 | 24 | 0 | 2017-01-15T00:00:00Z | 0.5489960 |
| -138 | 24 | 0 | 2017-01-15T00:00:00Z | 0.5842934 |
| -136 | 24 | 0 | 2017-01-15T00:00:00Z | 0.6270676 |
| -134 | 24 | 0 | 2017-01-15T00:00:00Z | 0.6492863 |
| -132 | 24 | 0 | 2017-01-15T00:00:00Z | 0.6305580 |
| -130 | 24 | 0 | 2017-01-15T00:00:00Z | 0.6110039 |
| -128 | 24 | 0 | 2017-01-15T00:00:00Z | 0.6338577 |
| -126 | 24 | 0 | 2017-01-15T00:00:00Z | 0.6734467 |
| -124 | 24 | 0 | 2017-01-15T00:00:00Z | 0.6693363 |
| -122 | 24 | 0 | 2017-01-15T00:00:00Z | 0.6090870 |
| -120 | 24 | 0 | 2017-01-15T00:00:00Z | 0.5179253 |
| ⋮ | ⋮ | ⋮ | ⋮ | ⋮ |
| -124 | 50 | 0 | 2017-01-15T00:00:00Z | 0.3403406 |
| -122 | 50 | 0 | 2017-01-15T00:00:00Z | NaN |
| -120 | 50 | 0 | 2017-01-15T00:00:00Z | NaN |
| -118 | 50 | 0 | 2017-01-15T00:00:00Z | NaN |
| -116 | 50 | 0 | 2017-01-15T00:00:00Z | NaN |
| -114 | 50 | 0 | 2017-01-15T00:00:00Z | NaN |
| -112 | 50 | 0 | 2017-01-15T00:00:00Z | NaN |
| -110 | 50 | 0 | 2017-01-15T00:00:00Z | NaN |
| -108 | 50 | 0 | 2017-01-15T00:00:00Z | NaN |
| -106 | 50 | 0 | 2017-01-15T00:00:00Z | NaN |
| -104 | 50 | 0 | 2017-01-15T00:00:00Z | NaN |
| -140 | 52 | 0 | 2017-01-15T00:00:00Z | 0.1521754 |
| -138 | 52 | 0 | 2017-01-15T00:00:00Z | 0.1631517 |
| -136 | 52 | 0 | 2017-01-15T00:00:00Z | 0.1961660 |
| -134 | 52 | 0 | 2017-01-15T00:00:00Z | 0.2636414 |
| -132 | 52 | 0 | 2017-01-15T00:00:00Z | 0.3504291 |
| -130 | 52 | 0 | 2017-01-15T00:00:00Z | 0.4085178 |
| -128 | 52 | 0 | 2017-01-15T00:00:00Z | 0.4275451 |
| -126 | 52 | 0 | 2017-01-15T00:00:00Z | NaN |
| -124 | 52 | 0 | 2017-01-15T00:00:00Z | NaN |
| -122 | 52 | 0 | 2017-01-15T00:00:00Z | NaN |
| -120 | 52 | 0 | 2017-01-15T00:00:00Z | NaN |
| -118 | 52 | 0 | 2017-01-15T00:00:00Z | NaN |
| -116 | 52 | 0 | 2017-01-15T00:00:00Z | NaN |
| -114 | 52 | 0 | 2017-01-15T00:00:00Z | NaN |
| -112 | 52 | 0 | 2017-01-15T00:00:00Z | NaN |
| -110 | 52 | 0 | 2017-01-15T00:00:00Z | NaN |
| -108 | 52 | 0 | 2017-01-15T00:00:00Z | NaN |
| -106 | 52 | 0 | 2017-01-15T00:00:00Z | NaN |
| -104 | 52 | 0 | 2017-01-15T00:00:00Z | NaN |
Getting all dataset information in a ERDDAP server#
The following steps show the way of listing out all dataset that is available in a ERDDAP using the tabledap function.
urlBase <- "https://gliders.ioos.us/erddap/"
dataInfo <- rerddap::info("allDatasets", url = urlBase)
allGlider <- tabledap(dataInfo)
info() output passed to x; setting base url to: https://gliders.ioos.us/erddap
Warning message in set_units(temp_table, dds):
“NAs introduced by coercion”
Warning message in set_units(temp_table, dds):
“NAs introduced by coercion”
allGlider[0:10,0:3]
| datasetID | accessible | institution | |
|---|---|---|---|
| <chr> | <chr> | <chr> | |
| 2 | allDatasets | public | Axiom Docker Install |
| 3 | amelia-20180501T0000 | public | Virginia Institute of Marine Science - William & Mary |
| 4 | amelia-20200825T1929 | public | Virginia Institute of Marine Science - William & Mary |
| 5 | amelia-20201015T1436 | public | Virginia Institute of Marine Science - William & Mary |
| 6 | amlr01-20181216T0641-delayed | public | NOAA SWFSC Antarctic Ecosystem Research Division |
| 7 | amlr01-20191206T0452-delayed | public | NOAA SWFSC Antarctic Ecosystem Research Division |
| 8 | amlr02-20181211T1233-delayed | public | NOAA SWFSC Antarctic Ecosystem Research Division |
| 9 | amlr02-20191206T1236-delayed | public | NOAA SWFSC Antarctic Ecosystem Research Division |
| 10 | amlr03-20191206T0529-delayed | public | NOAA SWFSC Antarctic Ecosystem Research Division |
| 11 | angus-20190522T0000 | public | Skidaway Institute of Oceanography |