| Title: | R Interface to the USGS/EROS Machine-to-Machine (M2M) Application Programming Interface (API) |
|---|---|
| Description: | The USGS/EROS Machine-to-Machine (M2M) Application Programming Interface (API) provides a method of searching and acquiring data from the USGS/EROS data inventories. The M2M API provides the same options and functions as EarthExplorer (EE) including searching for data collections and imagery, requesting metadata, and accessing all datasets in the EE catalog. Note that this package is not affiliated with the USGS/EROS, but is a community-developed interface to the M2M API for use in R. |
| Authors: | Steven Pawley [aut, cre] |
| Maintainer: | Steven Pawley <[email protected]> |
| License: | MIT + file LICENSE |
| Version: | 0.2.0 |
| Built: | 2026-10-05 12:50:32 UTC |
| Source: | https://github.com/stevenpawley/usgsm2m |
Create a cloud cover filter for use in a scene search
filter_cloud(min = 0, max = 100)filter_cloud(min = 0, max = 100)
min |
Minimum cloud cover percentage (0-100) |
max |
Maximum cloud cover percentage (0-100) |
A named list with min and max cloud cover percentages
filter_cloud(min = 0, max = 30)filter_cloud(min = 0, max = 30)
Combine metadata filters with a logical operator
filter_metadata_and(...) filter_metadata_or(...)filter_metadata_and(...) filter_metadata_or(...)
... |
Metadata filters to combine, as built by [filter_metadata_value()] or [filter_metadata_between()]. |
A named list describing the combined filter.
filter_metadata_and( filter_metadata_value("5e83d14fb9436d88", "045"), filter_metadata_value("5e83d14ff1eda1b8", "027") )filter_metadata_and( filter_metadata_value("5e83d14fb9436d88", "045"), filter_metadata_value("5e83d14ff1eda1b8", "027") )
Create a metadata filter matching a range of values
filter_metadata_between(filter_id, first, second)filter_metadata_between(filter_id, first, second)
filter_id |
The field's filter id, from the 'id' column of 'M2MDataset$filters()'. |
first |
The lower bound. |
second |
The upper bound. |
A named list describing the filter.
filter_metadata_between("5e83d14fb9436d88", 40, 45)filter_metadata_between("5e83d14fb9436d88", 40, 45)
Metadata filters test dataset-specific fields. Get the available 'filter_id' values, and what they mean, from 'M2MDataset$filters()'.
filter_metadata_value(filter_id, value, operand = c("=", "like"))filter_metadata_value(filter_id, value, operand = c("=", "like"))
filter_id |
The field's filter id, from the 'id' column of 'M2MDataset$filters()'. |
value |
The value to match. |
operand |
One of "=" or "like". |
A named list describing the filter.
filter_metadata_value("5e83d14fb9436d88", "045")filter_metadata_value("5e83d14fb9436d88", "045")
Create a spatial filter for use in a scene search
filter_spatial(ll_lon, ll_lat, ur_lon, ur_lat)filter_spatial(ll_lon, ll_lat, ur_lon, ur_lat)
ll_lon |
Lower left longitude |
ll_lat |
Lower left latitude |
ur_lon |
Upper right longitude |
ur_lat |
Upper right latitude |
A named list with spatial filter parameters
filter_spatial(ll_lon = -120, ll_lat = 40, ur_lon = -119, ur_lat = 41)filter_spatial(ll_lon = -120, ll_lat = 40, ur_lon = -119, ur_lat = 41)
Create a temporal filter for use in a scene search
filter_temporal(start, end)filter_temporal(start, end)
start |
Start date in "YYYY-MM-DD" format |
end |
End date in "YYYY-MM-DD" format |
A named list with start and end dates
filter_temporal("2020-07-01", "2020-07-31")filter_temporal("2020-07-01", "2020-07-31")
Authenticates against the M2M API and returns a session object, which is the entry point for everything else in this package. From a session you reach a dataset, from a dataset a scene search, and so on - each step returns an object whose methods are the valid next steps, so the download pipeline is discoverable by tab-completing '$' in the console.
m2m_session( username = Sys.getenv("M2M_USERNAME"), token = Sys.getenv("M2M_TOKEN") )m2m_session( username = Sys.getenv("M2M_USERNAME"), token = Sys.getenv("M2M_TOKEN") )
username |
ERS username. This is the same username that is used to log into Earth Explorer. Defaults to the 'M2M_USERNAME' environment variable. |
token |
Earth Explorer M2M application token. To generate a token, go to https://ers.cr.usgs.gov/. The token is different from the password used to log into Earth Explorer. Defaults to the 'M2M_TOKEN' environment variable. |
The usual pipeline is:
“' sess <- m2m_session() queue <- sess$ dataset("landsat_ot_c2_l2")$ search(spatial = filter_spatial(-120, 40, -119, 41), temporal = filter_temporal("2020-07-01", "2020-07-31"))$ products()$ select_bands(c("B4", "B5"))$ request(label = "my_order")
queue$retrieve("data/") “'
An [M2MSession] object.
## Not run: # Login using environment variables sess <- m2m_session() # Login using arguments sess <- m2m_session(username = "your_username", token = "your_token") ## End(Not run)## Not run: # Login using environment variables sess <- m2m_session() # Login using arguments sess <- m2m_session(username = "your_username", token = "your_token") ## End(Not run)
Returned by ‘M2MSession$dataset()'. Holds a dataset’s metadata and is the starting point for a scene search. Not created directly.
infoA one-row tibble of the dataset's metadata.
M2MDataset$new()Wrap dataset metadata. Use 'M2MSession$dataset()' instead.
M2MDataset$new(session, info)
sessionThe parent [M2MSession].
infoA one-row tibble of dataset metadata.
M2MDataset$alias()The dataset's system-friendly alias, e.g. '"landsat_ot_c2_l2"'.
M2MDataset$alias()
A string.
M2MDataset$id()The dataset's identifier.
M2MDataset$id()
A string.
M2MDataset$filters()Metadata filter fields available for this dataset. Each row's 'id' is the 'filterId' used when building a metadata filter to pass to '$search(metadata = )'.
M2MDataset$filters()
A tibble of filter fields.
M2MDataset$search()Search this dataset for scenes. All filters are optional; with none supplied every scene in the dataset matches.
M2MDataset$search( spatial = NULL, temporal = NULL, cloud = NULL, metadata = NULL, max_results = NULL )
spatialA spatial filter, see [filter_spatial()].
temporalA temporal filter, see [filter_temporal()].
cloudA cloud cover filter, see [filter_cloud()].
metadataA metadata filter built from '$filters()' field ids.
max_resultsMaximum number of scenes to return. If 'NULL' (the default) all matching scenes are retrieved by paging through results.
An [M2MSceneSearch] object.
M2MDataset$related_scenes()Find the scenes related to a given scene, via the 'scene-search-secondary' endpoint.
Only datasets that define a secondary relationship support this; others raise an 'm2m_error' with code 'DATASET_ERROR'. The related scenes belong to a different dataset, so the returned search is bound to that dataset rather than this one - meaning '$products()' on the result acts on the correct dataset.
M2MDataset$related_scenes(entity_id, max_results = NULL)
entity_idThe 'entityId' of the scene to find relatives of.
max_resultsMaximum number of scenes to return. If 'NULL' (the default) all are retrieved by paging through results.
An [M2MSceneSearch] over the secondary dataset.
M2MDataset$print()Print a summary of the dataset.
M2MDataset$print(...)
...Ignored.
M2MDataset$clone()The objects of this class are cloneable with this method.
M2MDataset$clone(deep = FALSE)
deepWhether to make a deep clone.
Returned by 'M2MSceneSearch$products()'. Holds what the M2M API will let you download for a set of scenes, and lets you narrow it before placing a request. Not created directly.
The API offers downloads at two granularities, and this object exposes both:
* **Individual files** - the bands and per-file products nested inside a product's 'secondaryDownloads', listed by '$bands()'. This is the default selection. * **Whole products** - the scene-level entries the API lists, such as a Level-1 Product Bundle (a single '.tar' for the scene) or a full-resolution browse image, listed by '$scene_products()'. Some of these have no constituent files at all and so never appear in '$bands()'.
Both are requestable. Use '$select_bands()' to queue individual files, or '$select_products()' to queue whole products; '$selected()' always shows what '$request()' will actually queue.
productsA scene-level tibble of products, with the individual bands nested in a 'secondaryDownloads' list-column.
M2MDownloadOptions$new()Wrap download options. Use 'M2MSceneSearch$products()' instead.
M2MDownloadOptions$new(session, products, selected = NULL)
sessionThe parent [M2MSession].
productsA tibble of products from the download-options endpoint.
selectedAn optional pre-narrowed band tibble.
M2MDownloadOptions$selected()The rows '$request()' will queue. Individual files by default, or whole products after '$select_products()'.
M2MDownloadOptions$selected()
A tibble of downloadable items.
M2MDownloadOptions$bands()The currently selected downloads. An alias for '$selected()', kept because the selection is individual bands in the common case.
M2MDownloadOptions$bands()
A tibble of downloadable items.
M2MDownloadOptions$scene_products()The whole-product entries the API lists for these scenes, one row each, such as a Level-1 Product Bundle or a full-resolution browse image.
These are requestable in their own right and are a different unit from '$bands()': a bundle is a single '.tar' holding the whole scene, where '$bands()' would list its contents as separate files. Products with no constituent files, such as browse imagery, appear only here.
M2MDownloadOptions$scene_products()
A tibble of products, without the 'secondaryDownloads' column.
M2MDownloadOptions$select_bands()Select the individual files whose 'displayId' matches any of the given patterns.
Each value is matched as a literal substring of 'displayId', which is how '"B4"' picks out '..._SR_B4' without you writing the whole name. Nothing is treated as pattern syntax, so a value containing punctuation matches itself. See '$bands()' for the values. Matching nothing warns and lists what was available.
Like '$select_products()', this selects afresh from everything available at that granularity rather than narrowing an existing selection - use '$filter()' to narrow. It also does not filter on availability; chain '$filter(bulkAvailable)' if you want only bulk-downloadable files.
M2MDownloadOptions$select_bands(patterns)
patternsA character vector of patterns, e.g. 'c("B4", "B5")'.
A new [M2MDownloadOptions] with those files selected.
M2MDownloadOptions$select_products()Switch the selection to whole products rather than the individual files inside them, optionally keeping only those whose 'productName' matches one of the given patterns.
Values are matched **exactly**, against either 'productName' or 'productCode', so a value copied out of '$scene_products()' selects what you copied. Names contain characters such as parentheses that would otherwise be read as pattern syntax, and they differ between datasets - the bundle is "Landsat Collection 2 Level-2 Product Bundle" for 'landsat_ot_c2_l2' but "Standard Format" for 'corona2' - so read them off '$scene_products()' rather than assuming. Matching nothing warns and lists what was available.
For anything looser, chain '$filter()', which takes arbitrary expressions: '$select_products()$filter(grepl("Browse", productName))'.
As with '$select_bands()' this does not filter on availability; chain '$filter(available)' to drop products the API has marked unavailable.
M2MDownloadOptions$select_products(products = NULL)
productsAn optional character vector of 'productName' or 'productCode' values, matched exactly.
A new [M2MDownloadOptions] with the products selected.
M2MDownloadOptions$filter()Narrow the current selection using 'dplyr::filter()' semantics against the tibble returned by '$selected()'.
M2MDownloadOptions$filter(...)
...Expressions passed to 'dplyr::filter()'.
A new [M2MDownloadOptions] with the narrowed selection.
M2MDownloadOptions$request()Place a download request for the current selection. Files the distribution system can serve immediately come back with URLs; anything needing preparation is collected later via 'M2MDownloadQueue$refresh()'.
M2MDownloadOptions$request(label)
labelA label identifying this order in the download queue. Required, and used to find the order again later.
An [M2MDownloadQueue] object.
M2MDownloadOptions$print()Print a summary of the available and selected downloads.
M2MDownloadOptions$print(...)
...Ignored.
M2MDownloadOptions$clone()The objects of this class are cloneable with this method.
M2MDownloadOptions$clone(deep = FALSE)
deepWhether to make a deep clone.
Returned by 'M2MDownloadOptions$request()', or reconnected to with 'M2MSession$download_queue(label)'. Tracks which files are ready to download and which the distribution system is still preparing.
labelThe label identifying this order.
M2MDownloadQueue$new()Attach to a download order. Use 'M2MDownloadOptions$request()' or 'M2MSession$download_queue()'.
M2MDownloadQueue$new(session, label, available = NULL, n_preparing = NULL)
sessionThe parent [M2MSession].
labelThe order's label.
availableOptionally seed the ready-to-download files.
n_preparingNumber of records the API accepted for preparation.
M2MDownloadQueue$refresh()Re-poll the M2M queue for this label, picking up files that have finished preparing since the last check.
Unlike the other methods here, this updates the object in place rather than returning a new one, because it reflects changing server-side queue state.
M2MDownloadQueue$refresh()
The queue, invisibly.
M2MDownloadQueue$ready()The files that can be downloaded now. Readiness is based on having a URL, regardless of which internal API bucket supplied it.
M2MDownloadQueue$ready()
A tibble of downloadable files.
M2MDownloadQueue$pending()The files still being prepared, which have no URL yet.
M2MDownloadQueue$pending()
A tibble of pending files.
M2MDownloadQueue$is_ready()Whether every file in the order can be downloaded now.
M2MDownloadQueue$is_ready()
'TRUE' if nothing is still being prepared.
M2MDownloadQueue$retrieve()Download every file that has a URL to disk. Call '$refresh()' first if '$is_ready()' is 'FALSE'.
Files are named as the server names them - the '.tar' of a product bundle, the '.TIF' of a band - rather than after the 'entityId' the queue lists them under, which for a scene-level product is the scene id and carries no extension.
A file already in 'out_dir' under that name is left alone unless 'overwrite' is 'TRUE', so re-running this after an interrupted order only fetches what is missing. Since the name usually only becomes known once the server has answered, the request is still made for each file, but none of the body is transferred for one that is already there.
Proxied downloads are reported back to the API afterwards, since it does not serve them itself and would otherwise leave them in the queue indefinitely. Files skipped as already present are reported too - they are on disk, so the order is fulfilled.
M2MDownloadQueue$retrieve(out_dir, report_proxied = TRUE, overwrite = FALSE)
out_dirDirectory to write files into. Created if missing.
report_proxiedWhether to mark proxied downloads complete.
overwriteWhether to download files that are already in 'out_dir', replacing them.
A tibble with one row per file: 'entityId', 'downloadId', 'url', 'path', 'size' and 'status'. A 'status' of '"expired"' means the signed URL is no longer valid - '$refresh()' and retry; '"skipped"' means the file was already in 'out_dir'.
M2MDownloadQueue$summary()Summarize this order by dataset, via the 'download-summary' endpoint.
M2MDownloadQueue$summary(download_application = "M2M", send_email = FALSE)
download_applicationThe application the downloads were requested under. Required by the API; the counts come back as zero if it does not match the one used at request time.
send_emailWhether the API should also email the summary.
A list with 'label', 'download_count', 'scene_count', 'total_estimated_size' and a 'collections' tibble.
M2MDownloadQueue$prepare()Move this order's scenes into the queue for processing, via the 'download-order-load' endpoint.
Unlike the other methods here this changes server-side state: it is what starts a staged order being prepared. Follow it with '$refresh()' to pick up URLs as they become ready.
M2MDownloadQueue$prepare(download_application = NULL)
download_applicationOptional application name to scope the order to.
The queue, invisibly.
M2MDownloadQueue$remove_items()Remove individual files from this order, leaving the rest of it in place. '$cancel()' drops the whole order instead.
Ids come from the 'downloadId' column of '$ready()' or '$pending()'.
M2MDownloadQueue$remove_items(download_id, quiet = FALSE)
download_idA vector of download ids.
quietSuppress the message shown before a large batch.
The queue, invisibly.
M2MDownloadQueue$cancel()Cancel this order, removing it from the M2M queue.
M2MDownloadQueue$cancel()
The queue, invisibly.
M2MDownloadQueue$print()Print a summary of the order.
M2MDownloadQueue$print(...)
...Ignored.
M2MDownloadQueue$clone()The objects of this class are cloneable with this method.
M2MDownloadQueue$clone(deep = FALSE)
deepWhether to make a deep clone.
Scene lists are the M2M API's way of naming a set of scenes so that other endpoints can act on them in bulk: 'download-options' and 'scene-metadata-list' will not take scene ids inline, only a 'listId'.
'M2MSceneSearch$products()' creates and uses one for you, so the usual pipeline never mentions scene lists. Reach for this class when the list itself is what you want - '$metadata()' fetches metadata for every scene in one call rather than one call per scene, and ‘$summary()' reports the set’s combined extent.
Reattach to an existing list with 'M2MSession$scene_list()'. Searches create the lists required for '$products()' internally. Not created directly.
list_idThe scene list's identifier.
dataset_nameThe dataset alias the list belongs to, or 'NA' if not yet known. Reattaching to a list by id leaves this 'NA' until ‘$dataset()' resolves it, since the API does not report a list’s dataset except through its summary.
removedWhether '$remove()' has deleted this list. Lists also expire on their own, which this does not track.
M2MSceneList$new()Attach to a scene list. Use 'M2MSession$scene_list()' instead.
M2MSceneList$new(session, list_id, dataset_name = NA_character_)
sessionThe parent [M2MSession].
list_idThe scene list identifier.
dataset_nameThe dataset alias the list belongs to, or 'NA' if unknown.
M2MSceneList$dataset()The dataset alias this list belongs to, looked up from the list's summary if it is not already known and cached thereafter.
A scene list can span several datasets, in which case there is no single answer and this errors - pass 'dataset_name' to 'M2MSession$scene_list()' to say which one you mean.
M2MSceneList$dataset()
The dataset alias, as a string.
M2MSceneList$scenes()The entity ids currently in the list.
M2MSceneList$scenes()
A tibble of scene entity ids.
M2MSceneList$metadata()Full metadata for every scene in the list.
M2MSceneList$metadata()
A tibble of scene metadata.
M2MSceneList$summary()Summarize the list's spatial and temporal extent, and the datasets it spans.
M2MSceneList$summary()
A tibble, with overall bounds in the 'spatialBounds' attribute.
M2MSceneList$products()Discover the products available for the scenes in this list.
M2MSceneList$products(band_group = TRUE)
band_groupWhether to include secondary file groups (bands).
An [M2MDownloadOptions] object.
M2MSceneList$remove()Delete the scene list from the M2M server. Lists also expire on their own after a period of inactivity.
M2MSceneList$remove()
The scene list, invisibly.
M2MSceneList$print()Print a summary of the scene list.
M2MSceneList$print(...)
...Ignored.
M2MSceneList$clone()The objects of this class are cloneable with this method.
M2MSceneList$clone(deep = FALSE)
deepWhether to make a deep clone.
Returned by 'M2MDataset$search()'. Holds the matching scenes and is the bridge to discovering downloadable products. Not created directly.
scenesA tibble of the matching scenes.
total_hitsThe total number of scenes the API reported as matching, which exceeds 'nrow(scenes)' when 'max_results' was set.
M2MSceneSearch$new()Wrap search results. Use 'M2MDataset$search()' instead.
M2MSceneSearch$new(session, dataset, results, total_hits)
sessionThe parent [M2MSession].
datasetThe [M2MDataset] that was searched.
resultsA tibble of matching scenes.
total_hitsTotal matches reported by the API.
M2MSceneSearch$dataset()The dataset these scenes came from.
M2MSceneSearch$dataset()
An [M2MDataset] object.
M2MSceneSearch$filter()Subset the scenes before requesting products, using 'dplyr::filter()' semantics against the 'scenes' tibble.
M2MSceneSearch$filter(...)
...Expressions passed to 'dplyr::filter()'.
A new [M2MSceneSearch] with the subset applied.
M2MSceneSearch$combine()Combine these scenes with those of other searches, so one '$products()' call and one download order covers all of them.
Scenes appearing in more than one search are kept once, so searches that overlap do not double-count. All the searches must be of the same dataset, since a set of products is discovered per dataset.
M2MSceneSearch$combine(...)
...Other [M2MSceneSearch] objects.
A new [M2MSceneSearch] holding every scene.
M2MSceneSearch$products()Discover the products (bands and related files) available for these scenes. This registers a scene list server-side first, which the M2M API requires before download options can be listed.
M2MSceneSearch$products(band_group = TRUE)
band_groupWhether to include secondary file groups, i.e. the individual bands. 'TRUE' by default.
An [M2MDownloadOptions] object.
M2MSceneSearch$print()Print a summary of the search results.
M2MSceneSearch$print(...)
...Ignored.
M2MSceneSearch$clone()The objects of this class are cloneable with this method.
M2MSceneSearch$clone(deep = FALSE)
deepWhether to make a deep clone.
Represents an authenticated connection to the USGS/EROS M2M API. Create one with [m2m_session()] rather than calling 'M2MSession$new()' directly.
M2MSession$new()Authenticate and open a session. Both arguments are required; [m2m_session()] is the documented way in and is what supplies the 'M2M_USERNAME' / 'M2M_TOKEN' defaults.
M2MSession$new(username, token)
usernameERS username.
tokenEarth Explorer M2M application token.
M2MSession$dataset()Look up a dataset by alias or id. This is the usual starting point for a search.
M2MSession$dataset(name = NULL, id = NULL)
nameThe system-friendly dataset alias, e.g. '"landsat_ot_c2_l2"'. Use this or 'id', not both.
idThe dataset identifier. Use this or 'name', not both.
An [M2MDataset] object.
M2MSession$find_datasets()Search the catalog for datasets matching a name pattern. Use this when you don't yet know a dataset's alias; wildcards are applied automatically around 'pattern'.
M2MSession$find_datasets(pattern, spatial = NULL, temporal = NULL)
patternSearch pattern for the dataset name.
spatialOptional spatial filter, see [filter_spatial()].
temporalOptional temporal filter, see [filter_temporal()].
A tibble of matching datasets. Pass a 'datasetAlias' value from the result to '$dataset()' to continue.
M2MSession$download_queue()Reconnect to an existing download order by its label, for example to resume collecting a large order in a later R session.
M2MSession$download_queue(label)
labelThe label the downloads were requested under.
An [M2MDownloadQueue] object.
M2MSession$downloads()List every download in the queue, regardless of status or label.
M2MSession$downloads()
A tibble of queued downloads.
M2MSession$remove_items()Remove individual downloads from the queue, across any label. Use this to clear out completed or unwanted entries without dropping a whole order the way 'M2MDownloadQueue$cancel()' does.
Ids come from the 'downloadId' column of '$downloads()':
“' done <- dplyr::filter(sess$downloads(), statusText == "Complete") sess$remove_items(done$downloadId) “'
The API removes one item per request and answers the same way for an id that does not exist, so a successful call is not evidence that anything was removed - check '$downloads()' afterwards.
M2MSession$remove_items(download_id, quiet = FALSE)
download_idA vector of download ids.
quietSuppress the message shown before a large batch.
The number of ids submitted, invisibly.
M2MSession$download_labels()List the distinct order labels in the download queue, one row each. Use this to find an order to reconnect to with '$download_queue()' when you no longer remember its label.
M2MSession$download_labels(download_application = NULL)
download_applicationOptional application name to scope the listing to.
A tibble with 'label', 'downloadCount', 'totalComplete', 'downloadSize' and 'dateEntered' (epoch milliseconds).
M2MSession$eula()Retrieve the text of one or more End User License Agreements. Some datasets require a EULA to be accepted (once, through the EarthExplorer website) before downloads will succeed.
M2MSession$eula(code = NULL, codes = NULL)
codeA single EULA code. Use this or 'codes', not both.
codesA character vector of EULA codes. Use this or 'code'.
A tibble with 'eulaCode' and 'agreementContent'.
M2MSession$scene_list()Attach to a scene list that already exists server-side.
M2MSession$scene_list(list_id, dataset_name = NULL)
list_idThe scene list identifier.
dataset_nameThe dataset alias the list belongs to. Optional - it is looked up from the list's summary when needed. Supply it to save that lookup, or to pick one when the list spans several datasets.
An [M2MSceneList] object.
M2MSession$ers_session()Return the small session value used by the package's internal HTTP functions. This method is intended for package internals.
M2MSession$ers_session()
An internal 'ers_session' object.
M2MSession$logout()End the session, invalidating its API key. M2M sessions also expire on their own after a period of inactivity.
M2MSession$logout()
The session, invisibly.
M2MSession$print()Print a summary of the session.
M2MSession$print(...)
...Ignored.
M2MSession$clone()The objects of this class are cloneable with this method.
M2MSession$clone(deep = FALSE)
deepWhether to make a deep clone.