diff --git a/.github/dictionary.txt b/.github/dictionary.txt index d7b6f5607..072ba55c3 100644 --- a/.github/dictionary.txt +++ b/.github/dictionary.txt @@ -111,6 +111,7 @@ dem_matching dem_name DEMs differenced +dir DockerizedTopsApp docstring DOI @@ -195,6 +196,7 @@ HyP3 hyp3 hyp3-docs hyp3-gamma +HyP3-generated HyP3-ISCE2 hyp3_sdk IaC @@ -208,6 +210,7 @@ inc_map.tif incidenceAngle include_dem include_rgb +initializer InSAR INSAR_ISCE_BURST INSAR_ISCE_MULTI_BURST @@ -301,6 +304,7 @@ non-geocoded NoSQL novertex OpenAPI +OpenSARLab OpenStreetMap OPERA_L2_RTC-S1 OPERA_L2_RTC-S1_V1 @@ -474,6 +478,7 @@ xx Yukon-Kuskokwim Yunjun yy +YYYY-MM-DD yyyymmdd YYYYMMDDTHHMMSSZ zz diff --git a/CHANGELOG.md b/CHANGELOG.md index 283afd670..51d360b78 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,19 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [PEP 440](https://www.python.org/dev/peps/pep-0440/) and uses [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [0.10.11] + +### Added +* A page describing [Download](https://hyp3-docs.asf.alaska.edu/using/downloading.md) options for On Demand Products in the [Using](https://hyp3-docs.asf.alaska.edu/using/) section + +### Changed +* Updated [ARIA S1 GUNW Product Guide](https://hyp3-docs.asf.alaska.edu/guides/gunw_product_guide.md) to include: + * information about date-based ARIA S1 GUNW job submissions + * supported polarizations + * SLC coverage constraints + +* Updated [OPERA RTC Product Guide](https://hyp3-docs.asf.alaska.edu/guides/opera_rtc_product_guide.md) to reflect that we now allow any IW burst SLC north of -60° latitude to be processed + ## [0.10.10] ### Changed diff --git a/docs/guides/burst_insar_product_guide.md b/docs/guides/burst_insar_product_guide.md index 6f9e6fdff..b7414d248 100644 --- a/docs/guides/burst_insar_product_guide.md +++ b/docs/guides/burst_insar_product_guide.md @@ -158,7 +158,7 @@ There are several options users can set when ordering Burst InSAR On Demand prod 2. There is an option to apply a **water mask**. This mask includes coastal waters and most inland waterbodies. Masking waterbodies can have a significant impact during phase unwrapping, as water can sometimes exhibit enough coherence between acquisitions to allow for unwrapping to occur over waterbodies, which is invalid. Refer to our -[InSAR Water Masking Tutorial](https://storymaps.arcgis.com/stories/485916be1b1d46889aa436794b5633cb "InSAR Water Masking StoryMap" ){target=_blank} +[InSAR Water Masking tutorial](https://storymaps.arcgis.com/stories/485916be1b1d46889aa436794b5633cb "InSAR Water Masking StoryMap" ){target=_blank} for more information. - Water masking is turned off by default. @@ -320,7 +320,7 @@ detrimental to phase unwrapping than the exclusion of some land pixels, so as of the water mask used for this option is no longer buffered. Visit our -[InSAR Water Masking Tutorial](https://storymaps.arcgis.com/stories/485916be1b1d46889aa436794b5633cb "InSAR Water Masking StoryMap" ){target=_blank} +[InSAR Water Masking tutorial](https://storymaps.arcgis.com/stories/485916be1b1d46889aa436794b5633cb "InSAR Water Masking StoryMap" ){target=_blank} for more information about how different water masking approaches can impact the quality of an interferogram. ### Post-Processing @@ -419,7 +419,7 @@ and datasets. Refer to the [Water Masking Processing Option](#applying-a-water-mask) section and our -[InSAR Water Masking Tutorial](https://storymaps.arcgis.com/stories/485916be1b1d46889aa436794b5633cb "InSAR Water Masking StoryMap" ){target=_blank} +[InSAR Water Masking tutorial](https://storymaps.arcgis.com/stories/485916be1b1d46889aa436794b5633cb "InSAR Water Masking StoryMap" ){target=_blank} for more information about water masking. For jobs processed using `INSAR_ISCE_BURST`, there are also four non-geocoded images that remain in their native @@ -527,7 +527,16 @@ files. | Radar sensing stop | Last date and time for data collection | 2020-06-04T02:23:16.030988 | *Table 6: List of additional InSAR parameters included in the parameter text file `INSAR_ISCE_BURST` job types.* +{% endblock %} + +{% block download_tutorial %} +Step-by-step instructions for finding and downloading RTC On Demand products in Vertex are available in the +[Downloading On Demand Products](https://storymaps.arcgis.com/stories/191bf1b6962c402086807390b3ce63b0#ref-n-4rF6zW "Burst-Based InSAR for Sentinel-1 On Demand StoryMap Tutorial - Downloading On Demand Products"){target=_blank} +section of the +[Burst-Based InSAR for Sentinel-1 On Demand](https://storymaps.arcgis.com/stories/191bf1b6962c402086807390b3ce63b0 "Burst-Based InSAR for Sentinel-1 On Demand StoryMap Tutorial" ){target=_blank} interactive StoryMap tutorial. +{% endblock %} +{% block merging_bursts %} ## Merging Sentinel-1 Single-Burst InSAR Products Burst InSAR products generated using the `INSAR_ISCE_BURST` job type can be merged together manually using the diff --git a/docs/guides/gunw_product_guide.md b/docs/guides/gunw_product_guide.md index c6c575567..47602120a 100644 --- a/docs/guides/gunw_product_guide.md +++ b/docs/guides/gunw_product_guide.md @@ -4,7 +4,7 @@ This document is a guide for users of [ARIA Sentinel-1 Geocoded Unwrapped (GUNW) Interferograms](https://www.earthdata.nasa.gov/data/catalog/asf-aria-s1-gunw-1 "ASF-ARIA-S1-GUNW Products" ){target=_blank}. The -[ARIA Sentinel-1 Geocoded Unwrapped Interferogram (ARIA-S1-GUNW)](https://aria.jpl.nasa.gov/products/standard-displacement-products.html "ARIA Standard Displacement Products" ){target=_blank} +[ARIA Sentinel-1 Geocoded Unwrapped Interferogram (ARIA-S1-GUNW)](https://aria.jpl.nasa.gov/products/standard-displacement-products.html "aria.jpl.nasa.gov/products/standard-displacement-products.html" ){target=_blank} product is a standardized interferometric SAR (InSAR) dataset that enables rapid analysis of surface deformation using Sentinel-1 SAR data. Produced by [JPL’s ARIA](https://aria.jpl.nasa.gov/ "aria.jpl.nasa.gov" ){target=_blank} @@ -20,11 +20,12 @@ to centimeter-scale ground displacement measurements. Generated through an open- these products support applications such as earthquake impact assessment, volcanic monitoring, and long-term land motion studies, with ongoing improvements enhancing their accuracy and usability. -The ARIA project also maintains the [ARIA-tools](https://doi.org/10.1029/2020GL090013){target=_blank} software, -which is a suite of open-source tools which automates the seamless download, post-processing manipulation, -aggregation, and management of ARIA-S1-GUNW products. Users may refer to the dedicated -[GitHub](https://github.com/aria-tools/ARIA-tools "ARIA-tools GitHub repository" ){target=_blank} -page for a more exhaustive overview and installation instructions, and +The ARIA project also maintains the +[ARIA-tools](https://doi.org/10.1029/2020GL090013 "Toward Sustained Monitoring of Subsidence at the Coast Using InSAR and GPS" ){target=_blank} +software package, which is a suite of open-source tools that allows users to automate the seamless download, +post-processing manipulation, aggregation, and management of ARIA-S1-GUNW products. Refer to the +[ARIA-tools GitHub page](https://github.com/aria-tools/ARIA-tools "ARIA-tools GitHub repository" ){target=_blank} +for a more thorough overview and installation instructions, and [tutorials led by EarthScope Consortium](https://www.youtube.com/watch?v=_a9T59VTz7Q&t=12734s "EarthScope Tutorials on YouTube" ){target=_blank}, which demonstrate practical applications. @@ -61,14 +62,15 @@ search portal by following these steps: ![Vertex ARIA S1 GUNW Dataset Selection](../images/vertex-GUNW-dataset-selection.png) 3. **Preview and Select Products** – Click on individual results to view metadata, including coverage area and acquisition details. -4. **Download Data** – To download, first add ARIA-S1-GUNW products to your download queue using the shopping - cart icon next to each product, then download your selected products using the “download” panel. +4. **Download Data** – To download, first add ARIA-S1-GUNW products to your Download Queue using the + shopping cart icon next to each product, then download your selected products using the + [options available in the Download Queue interface](https://docs.asf.alaska.edu/vertex/manual/#downloads-queue "docs.asf.alaska.edu/vertex/manual/#downloads-queue" ){target=_blank}. ## Ordering On-Demand Products If the ARIA-S1-GUNW products you need are not available in the archive, you can use ASF's On Demand platform to submit custom ARIA-S1-GUNW jobs for processing. Once processing is complete, you can access them as you would any other -[On-Demand](#accessing-on-demand-aria-s1-gunw-products "Jump to the Accessing On Demand ARIA S1 GUNW Products section of this document" ) +[On-Demand](#data-access "Jump to the Data Access section of this document") products from ASF. The download links provided will be active for 14 days. !!! warning "Sentinel-1C acquisitions not yet supported" @@ -107,9 +109,11 @@ create a new ARIA-S1-GUNW product (see figure below). #### ARIA Frame ID Maps -It can be tricky to find all of the appropriate granules for a given ARIA Frame ID for both the reference and -secondary acquisition dates. In the future, ASF plans to create utilities to simplify this process. In the meantime, -there are geojson files indicating the extent of each ARIA Frame ID that can be downloaded and used for reference. +The ARIA project provides geojson files indicating the extent of each ARIA Frame ID. These files can be +downloaded and used for reference. You may find it helpful to extract the polygon for a specific frame +from the geojson and use the +[Import AOI functionality in Vertex](https://docs.asf.alaska.edu/vertex/manual/#area-of-interest-options "docs.asf.alaska.edu/vertex/manual/#area-of-interest-options" ){target=_blank} +to search for Sentinel-1 acquisitions over that frame. There are different ARIA Frame ID maps for the ascending and descending orbit directions. Make sure that you are using the appropriate geojson file. @@ -117,111 +121,88 @@ using the appropriate geojson file. - [Ascending ARIA Frame IDs](https://d3g9emy65n853h.cloudfront.net/ARIA_S1_GUNW/ascending.geojson "ARIA-S1-GUNW Ascending Frame IDs GeoJSON" ){target=_blank} - [Descending ARIA Frame IDs](https://d3g9emy65n853h.cloudfront.net/ARIA_S1_GUNW/descending.geojson "ARIA-S1-GUNW Descending Frame IDs GeoJSON" ){target=_blank} -### Search for Sentinel-1 SLCs for an ARIA Frame ID +### Search for Sentinel-1 SLC Acquisition Dates for an ARIA Frame ID -Use a [Geographic Search](https://docs.asf.alaska.edu/vertex/manual/#geographic-search-options){target=_blank} for -Sentinel-1 SLC IW products in your area of interest in [Vertex](https://search.asf.alaska.edu/#/){target=_blank}, -setting the Area of Interest to the desired ARIA Frame ID, as delineated in the -[ARIA Frame ID maps](#aria-frame-id-maps). You may want to apply a search filter for the orbit direction that -matches the ARIA Frame ID extent that you are using. - -For each footprint that intersects the ARIA Frame ID, use the -[SBAS](https://docs.asf.alaska.edu/vertex/sbas/){target=_blank} or -[Baseline](https://docs.asf.alaska.edu/vertex/baseline/){target=_blank} tool in Vertex -to find other acquisitions to pair with the reference acquisition. - - - You will need to repeat the process of finding pairs for each footprint along the Sentinel-1 orbit path - that intersects the ARIA Frame ID extent. - -#### Sentinel-1 SLC Selection Constraints - -There are a number of conditions that must be met when selecting suitable sets of Sentinel-1 IW SLCs for -processing to ARIA-S1-GUNW: - - ***1. All scenes must be in VV polarization*** - - - ARIA S1 GUNW jobs do not support acquisitions in HH polarization - - cross-polarized returns (VH or HV) are not supported - - ***2. All scenes (reference and secondary) must be from the same relative orbit*** - - - they must all have the same path number, which matches the path of the extent of the desired ARIA Frame ID - - note that the ARIA frames are each constrained to a single path - - consider adding a filter to your geographic search to limit the returns to acquisitions with the same path - number as the ARIA Frame ID - - ***3. All scenes must have the same orbit direction (ascending/descending)*** - - - the orbit direction must match the orbit direction of the ARIA Frame ID you are using - - consider adding a filter to your geographic search to limit the returns to acquisitions with the same orbit - direction as the ARIA Frame ID - - ***4. All reference scenes must be from the same absolute orbit*** - - - they must all be from the same pass of the satellite - - acquisitions from different dates cannot be combined - - ***5. All secondary scenes must be from the same absolute orbit*** - - - they must all be from the same pass of the satellite - - acquisitions from different dates cannot be combined +The ARIA processing code takes a list of reference and secondary Sentinel-1 IW SLC granules as input, but +it can be tricky to find all of the necessary granules for a given ARIA Frame ID. To ensure that there is +full coverage over the desired ARIA Frame, **users will just pass the ARIA Frame ID and the dates of the +desired primary and secondary passes over that frame into the On-Demand job specification** rather than +assembling lists of primary and secondary SLCs. - ***6. Reference scenes must be acquired after the secondary scenes*** - - - the list of reference scenes are from the most recent pass, and the secondary scenes are from the earlier - pass that will be compared to the reference scenes - - ***7. Reference and secondary scenes must overlap the frame geometry*** - - - all of the scenes listed must overlap the ARIA Frame ID extent - - do not include any acquisitions where valid pixel data is wholly outside the extent of the ARIA frame, - even if the no-data padding around the edges overlaps the frame extent - -#### Compile a List of Sentinel-1 SLCs - -Based on your search results, create a list of the reference and secondary Sentinel-1 IW SLCs that intersect -with the ARIA Frame ID extent. - - Example: - - "reference": [ - "S1A_IW_SLC__1SDV_20250127T010136_20250127T010203_057623_07199D_4B63", - "S1A_IW_SLC__1SDV_20250127T010111_20250127T010138_057623_07199D_4E88", - "S1A_IW_SLC__1SDV_20250127T010045_20250127T010113_057623_07199D_4D3B" - ], - "secondary": [ - "S1A_IW_SLC__1SDV_20250103T010137_20250103T010204_057273_070BB6_CD45", - "S1A_IW_SLC__1SDV_20250103T010113_20250103T010140_057273_070BB6_1133", - "S1A_IW_SLC__1SDV_20250103T010047_20250103T010115_057273_070BB6_99C5" - ], +To find suitable primary and secondary acquisition dates to use for a specific ARIA Frame ID, use a +[Geographic Search](https://docs.asf.alaska.edu/vertex/manual/#geographic-search-options "Vertex Manual: Geographic Search Options" ){target=_blank} +for Sentinel-1 SLC IW products in +[Vertex](https://search.asf.alaska.edu/#/ "search.asf.alaska.edu" ){target=_blank}, +setting the Area of Interest to the desired ARIA Frame ID, as delineated in the +[ARIA Frame ID maps](#aria-frame-id-maps "Jump to the ARIA Frame ID Maps section of this document"). + +Applying appropriate filters to the +[Geographic Search](https://search.asf.alaska.edu/#/ "search.asf.alaska.edu" ){target=_blank} +will help ensure that you find results that match the desired [ARIA Frame ID](#aria-frame-id-maps "Jump to the ARIA Frame ID Maps section of this document"): + +- **Area of Interest**: use the extent of the desired ARIA Frame from the reference geojson, or drop a point + in the middle of the ARIA Frame location +- **Start Date** / **End Date**: restrict the date range as desired +- **File type**: `L1 Single Look Complex (SLC)` +- **Beam Mode**: `IW` +- **Polarization**: select both `VV+VH` and `VV` (HH ARIA S1 GUNW products are not supported) + - Note that only the VV polarization will be processed, but this will include VV SLCs acquired in both + dual-pol and single-pol modes in the search results +- **Direction**: match the orbit direction of the [ARIA Frame ID reference geojson](#aria-frame-id-maps "Jump to the ARIA Frame ID Maps section of this document") + used to select the desired ARIA Frame (or reference the `dir` attribute from the ARIA Frame ID geojson file) +- **Subtype**: select both `SA` and `SB` (ARIA S1 GUNW products from Sentinel-1C acquisitions are not currently + supported) +- **Path Start** / **Path End**: path of the desired ARIA Frame (`path` attribute from the ARIA Frame ID geojson file) + +Select an acquisition that intersects the ARIA Frame ID for a date you want to include in your InSAR pair, then use the +[Baseline](https://docs.asf.alaska.edu/vertex/baseline/ "docs.asf.alaska.edu/vertex/baseline" ){target=_blank} or +[SBAS](https://docs.asf.alaska.edu/vertex/sbas/ "docs.asf.alaska.edu/vertex/sbas" ){target=_blank} +tool to find an appropriate date to pair with it. + +Common area covered by the selected reference and secondary scenes must cover at least 90% of the ARIA Frame. If +the SLCs available for the submitted dates do not meet this coverage constraint, the job will fail to process. ### Submit On-Demand ARIA-S1-GUNW Jobs !!! warning "On Demand support not currently available in Vertex for ARIA-S1-GUNW products" - On-demand ARIA S1 GUNW products cannot currently be submitted directly from Vertex, but we plan to make - this feature available in the second half of 2025. Vertex is still very useful for selecting Sentinel-1 SLC - pairs to submit for processing, but once you identify scene pairs, you will need to submit them using the - [HyP3 Python SDK](../using/sdk.md){target=_blank} or [HyP3 API](../using/api.md){target=_blank}. + On-demand ARIA S1 GUNW products cannot currently be submitted directly from Vertex, but we plan to + make this feature available in the second half of 2025. + + Vertex is still very useful for selecting Sentinel-1 SLC acquisition dates to submit for processing, + but once you identify reference and secondary dates for the desired ARIA Frame ID, you will need to submit + the job for processing using the [HyP3 Python SDK](../using/sdk.md){target=_blank} or + [HyP3 API](../using/api.md){target=_blank}. On-Demand ARIA-S1-GUNW jobs can be submitted using the `ARIA_S1_GUNW` job type via the -[HyP3 API](../using/api.md#submitting-aria-s1-gunw-jobs){target=_blank}, or via the -[HyP3 Python SDK](../using/sdk.md){target=_blank} +[HyP3 API](../using/api.md#submitting-aria-s1-gunw-jobs "hyp3-docs.asf.alaska.edu/using/api/#submitting-aria-s1-gunw-jobs" ){target=_blank}, +or via the [HyP3 Python SDK](../using/sdk.md "hyp3-docs.asf.alaska.edu/using/sdk" ){target=_blank} using the `submit_aria_s1_gunw_job` method of the `HyP3` class. -Unlike our other On-Demand InSAR workflows, customizable processing options (multilooking, filter strength, etc.) +Unlike ASF's other On-Demand InSAR workflows, customizable processing options (multilooking, filter strength, etc.) are not available for ARIA-S1-GUNW jobs. -### Accessing On-Demand ARIA-S1-GUNW Products +To submit an ARIA_S1_GUNW job, all you need is: + +- the ARIA Frame ID number +- the reference date, which is the **more recent pass** over the ARIA Frame +- the secondary date, which is the **earlier pass** over the ARIA Frame + +The dates must be in YYYY-MM-DD format. + +#### Reference and Secondary Dates -To access the products you submitted On Demand, use the -[On Demand Search Type in Vertex](https://search.asf.alaska.edu/#/?maxResults=1000&searchType=On%20Demand "On Demand Vertex Search" ){target=_blank}. -Once processing is complete, the download links available in this interface are active for 14 days. +ARIA S1 GUNW products use the SLCs from the more recent pass as reference, while secondary scenes are from the +earlier pass in the date pair. **When submitting a job using the HyP3 API or SDK, the date passed as the reference +date *must* be more recent than the secondary date. If they are in the opposite order, an error will be raised.** -You can also search for your On Demand products programmatically, using the -[HyP3 API](https://hyp3-docs.asf.alaska.edu/using/api/#querying-jobs "hyp3-docs.asf.alaska.edu/using/api/#querying-jobs" ){target=_blank} -or the -[HyP3 Python SDK](https://hyp3-docs.asf.alaska.edu/using/sdk/#finding-existing-jobs "hyp3-docs.asf.alaska.edu/using/sdk/#finding-existing-jobs" ){target=_blank}. +Note that ***this order is opposite of the other On-Demand InSAR products available from ASF***. Both the +[InSAR GAMMA](insar_product_guide.md "Jump to InSAR Product Guide") products and the +[Burst InSAR](burst_insar_product_guide.md "Jump to Burst InSAR Product Guide" ) products use the +earlier acquisition as reference, and the more recent acquisition as secondary. This means that the ARIA unwrapped +interferograms have the *opposite* sign from the unwrapped interferograms generated by the other ASF On-Demand InSAR +workflows. In the ARIA S1 GUNW products, negative phase differences indicate movement away from the sensor and +positive phase differences indicate movement towards the sensor. ## Product Packaging @@ -247,10 +228,13 @@ GUNW naming convention includes: The product is packaged as a NetCDF4 file, with its top-level group named `science`. Within the science group, there is a `grids` group, which is further divided into three subgroups: `data`, `imagingGeometry`, and `corrections`. -The `data` group contains 2D datasets at a resolution of 3 arc-seconds (~90 m) and the `imagingGeometry` group -includes 3D datasets posted laterally at 0.1-degree intervals (~11 km). The `corrections` group provides ionospheric -and solid Earth corrections, and if a weather model is available, the corresponding tropospheric correction layer -(`HRRR/reference/troposphereWet`) will be included here. All 2D and 3D datasets are in the EPSG:4326 projection. + +- The `data` group contains 2D datasets at a resolution of 3 arc-seconds (~90 m). +- The `imagingGeometry` group includes 3D datasets posted laterally at 0.1-degree intervals (~11 km). +- The `corrections` group provides ionospheric and solid Earth corrections, and if a weather model is available, + the corresponding tropospheric correction layer (`HRRR/reference/troposphereWet`) will be included here. + +All 2D and 3D datasets are in the EPSG:4326 projection. The output netCDF file will include the layers listed in the table below. @@ -275,20 +259,21 @@ The output netCDF file will include the layers listed in the table below. | | parallelBaseline | 3D parallel baseline grid | meter | | | perpendicularBaseline | 3D perpendicular baseline grid | meter | -### Ionospheric Correction Layers +### Ionospheric Correction Layer Although the ionospheric effects for C-band SAR are only about one-sixteenth of those at L-band, the measurement accuracy of Sentinel-1 C-band SAR data can still be degraded by long-wavelength ionospheric signals. Utilizing the -[range-split spectrum methodology](https://doi.org/10.1109/TGRS.2019.2908494){target=_blank} available within ISCE2, -ARIA-S1-GUNW products include ionospheric correction layers for both the reference and secondary input data. +[range-split spectrum methodology](https://doi.org/10.1109/TGRS.2019.2908494 "doi.org/10.1109/TGRS.2019.2908494" ){target=_blank} available within ISCE2, +ARIA-S1-GUNW products include an ionospheric correction layer packaged as a differential field between the +secondary and reference input data, which can be directly subtracted from the unwrappedPhase field. ### Solid Earth Tides Correction Layers -[Solid Earth tides](https://doi.org/10.1109/TGRS.2022.3168509){target=_blank} (SET) are periodic deformations of the -Earth's crust caused by gravitational forces from the Moon and Sun, resulting in surface displacements of up to -several centimeters. Correcting for SET in InSAR is crucial to prevent these predictable, cyclic motions from being -misinterpreted as real ground deformation. ARIA-S1-GUNW products include an SET correction layer for both the -reference and secondary input data that are created using the +[Solid Earth tides](https://doi.org/10.1109/TGRS.2022.3168509 "doi.org/10.1109/TGRS.2022.3168509" ){target=_blank} +(SET) are periodic deformations of the Earth's crust caused by gravitational forces from the Moon and Sun, +resulting in surface displacements of up to several centimeters. Correcting for SET in InSAR is crucial to prevent +these predictable, cyclic motions from being misinterpreted as real ground deformation. ARIA-S1-GUNW products +include an SET correction layer for both the reference and secondary input data that are created using the [PySolid](https://github.com/insarlab/PySolid?tab=readme-ov-file "PySolid GitHub repository" ){target=_blank} python package. @@ -307,7 +292,13 @@ HRRR weather model is not available for a location of interest, (i.e. outside of tropospheric delay correction layer will not be included in the ARIA-S1-GUNW product. The wet and hydrostatic tropospheric delay correction are provided for both the reference and secondary input data. -### References +## Data Access + +Refer to the [Downloads](../using/downloading.md "hyp3-docs.asf.alaska.edu/using/downloading") +page for more information on viewing and downloading ARIA S1 GUNW On Demand products in Vertex or programmatically. +Once processing is complete, download links for On Demand products are valid for 14 days. + +## References Bekaert, David, et al. "The ARIA-S1-GUNW: The ARIA Sentinel-1 Geocoded Unwrapped Phase Product for Open InSAR Science and Disaster Response." IGARSS 2023-2023 IEEE International Geoscience and Remote Sensing Symposium. IEEE (2023). diff --git a/docs/guides/insar_product_guide.md b/docs/guides/insar_product_guide.md index d75699a4c..47d6121a2 100644 --- a/docs/guides/insar_product_guide.md +++ b/docs/guides/insar_product_guide.md @@ -357,6 +357,12 @@ Those not using ArcGIS will still find the contents of these XML files useful, b Geolocation XML files (aux files) are included for each of the PNG browse images to allow for proper display in GIS platforms. {% endblock %} +{% block download_tutorial %} +Step-by-step instructions for finding and downloading RTC On Demand products in Vertex are available in the +[Submit/Download Jobs](https://storymaps.arcgis.com/stories/68a8a3253900411185ae9eb6bb5283d3#ref-n-NRdP9N "InSAR On Demand! StoryMap Tutorial - On Demand Search"){target=_blank} section of the +[InSAR On Demand!](https://storymaps.arcgis.com/stories/68a8a3253900411185ae9eb6bb5283d3 "InSAR On Demand! StoryMap Tutorial" ){target=_blank} interactive StoryMap tutorial. +{% endblock %} + {% block line_of_sight %} ### Line-of-Sight Measurements When looking at a single interferogram, the deformation measurements in the line-of-sight orientation of the sensor indicate relative motion towards or away from the sensor. InSAR is not sensitive to motion in the azimuth direction of the satellite, so motion that occurs in the same direction as the satellite's direction of travel will not be detected. diff --git a/docs/guides/insar_product_guide_template.md b/docs/guides/insar_product_guide_template.md index b8cc40b7f..374c428c0 100644 --- a/docs/guides/insar_product_guide_template.md +++ b/docs/guides/insar_product_guide_template.md @@ -107,6 +107,16 @@ To analyze deformation caused by a single discrete event, such as an earthquake, {% block packaging %}{% endblock %} +## Data Access + +Refer to the [Downloads](../using/downloading.md "hyp3-docs.asf.alaska.edu/using/downloading") +page for more information on viewing and downloading On Demand InSAR products in Vertex or programmatically. +Once processing is complete, download links for On Demand products are valid for 14 days. + +{% block download_tutorial %}{% endblock %} + +{% block merging_bursts %}{% endblock %} + ## Limitations ### Baseline Calculation The baseline is defined as the difference of the platform positions when a given area is imaged. HyP3 baselines are calculated using the best state vectors available. If precise orbits are not yet available for the input granules, restituted orbits will be used. The original predicted orbits are not used for InSAR processing in HyP3. **If no restituted or precise state vectors are available, the process will not run.** diff --git a/docs/guides/opera_rtc_product_guide.md b/docs/guides/opera_rtc_product_guide.md index 29d3fa041..b562041e2 100644 --- a/docs/guides/opera_rtc_product_guide.md +++ b/docs/guides/opera_rtc_product_guide.md @@ -51,9 +51,9 @@ Most Sentinel-1 acquisitions are collected in two [polarizations](introduction_to_sar.md#polarizations), and the OPERA project generates RTC-S1 products for all available polarizations. -Sentinel-1 acquisitions over land generally have a vertical primary polarization, meaning that the SAR signal is sent out in -a vertical orientation, but both the co-polarized returns (also vertically polarized) and the cross-polarized returns -(horizontally polarized) are measured. This results in both VV and VH RTC-S1 products. +Sentinel-1 acquisitions over land generally have a vertical primary polarization, meaning that the SAR signal is sent +out in a vertical orientation, but both the co-polarized returns (also vertically polarized) and the cross-polarized +returns (horizontally polarized) are measured. This results in both VV and VH RTC-S1 products. In some areas, particularly remote islands and regions close to the Arctic Ocean, acquisitions are collected with a horizontal primary polarization. This results in HH and HV RTC-S1 products. @@ -78,7 +78,7 @@ RTC-S1 products generated by the OPERA project are all archived by ASF and can b interface, which provides access to all of NASA's Earth Science datasets For more information on options for accessing archived OPERA RTC-S1 products, refer to ASF's -[OPERA Sentinel-1 RTC Tutorial](https://storymaps.arcgis.com/stories/dc2807b444924fc3a76c117a2c909f8b "OPERA Sentinel-1 RTC StoryMap Tutorial" ){target=_blank}. +[OPERA Sentinel-1 RTC tutorial](https://storymaps.arcgis.com/stories/dc2807b444924fc3a76c117a2c909f8b "OPERA Sentinel-1 RTC StoryMap Tutorial" ){target=_blank}. For more information on the technical specifications of the OPERA RTC-S1 products, refer to JPL's [RTC Product Documentation](https://www.jpl.nasa.gov/go/opera/products/rtc-product/ "OPERA RTC Product" ){target=_blank}. @@ -130,10 +130,8 @@ section of the ## Ordering On-Demand OPERA RTC-S1 Products -On-Demand OPERA RTC-S1 products can be requested for any Sentinel-1 SLC burst acquired on or after April 14, 2016, -and before January 1, 2022, in the processing area where -[OPERA RTC-S1 Static Layer Files](#l2-radiometric-terrain-corrected-static-layer-rtc-static-files "Jump to the Static Layer Files section of this document" ) -are available in the ASF archive. +On-Demand OPERA RTC-S1 products can be requested for any Sentinel-1 IW SLC burst acquired on or after April 14, 2016, +and before January 1, 2022, for locations north of -60° latitude (i.e. all global landmasses except for Antarctica). ### Submitting On-Demand OPERA RTC-S1 Jobs @@ -169,45 +167,12 @@ or after that date will also return an error. #### Spatial Coverage for On-Demand OPERA RTC-S1 Products -On-Demand OPERA RTC-S1 Products can only be ordered for bursts that have associated -[OPERA RTC-S1 Static Layer Files](#l2-radiometric-terrain-corrected-static-layer-rtc-static-files "Jump to the Static Layer Files section of this document" ) -available in ASF's archive. The OPERA project supports processing over all global landmasses except for Antarctica. -Only bursts from +On-Demand OPERA RTC-S1 Products can only be ordered for bursts that are north of -60° latitude, +which includes all global landmasses except for Antarctica. Only bursts from [Sentinel-1 IW SLC](https://sentiwiki.copernicus.eu/web/s1-products "Sentinel-1 Products" ){target=_blank} products are supported as input. -Any jobs submitted where the Sentinel-1 burst is from an EW SLC or does not have associated static layers available -will return an error. - -!!! warning "OPERA RTC-S1 On-Demand coverage not available in some arctic regions" - - The static layers used to process acquisitions over some parts of Greenland and Arctic Canada are currently - unavailable in ASF's archive. On-Demand jobs submitted for these areas will return an error, as the static - layers are required for processing the OPERA RTC-S1 products. We are working with the OPERA team to add these - files to the ASF archive. - - This graphic illustrates the current coverage of OPERA RTC-S1 Static Layers. - ![ExtraFigure](../images/opera-rtc-static-layer-coverage.png "Current coverage of OPERA RTC-S1 Static Layers") - -## Accessing On-Demand OPERA RTC-S1 Products - -On-Demand OPERA RTC-S1 products can be accessed programmatically using the -[HyP3 API](../using/api.md "Using HyP3 API") -or -[Python SDK](../using/sdk.md "Using HyP3 Python SDK"). - -Users can also access the products using the -[On Demand Search](https://docs.asf.alaska.edu/vertex/manual/#on-demand-products-search-options "Vertex Manual" ){target=_blank} -interface in -[Vertex](https://search.asf.alaska.edu/#/?maxResults=1000&searchType=On%20Demand "Vertex On Demand Search" ){target=_blank}. - - - -**As with other On-Demand products available from ASF, these products are only available for 14 days once -processing is complete, so make sure that you download the files within that two-week window.** - -We plan to support adding standard OPERA RTC-S1 products generated On Demand to the OPERA_L2_RTC-S1_V1 collection, -expanding the archive based on user needs. Stay tuned for that functionality! +Any job submitted where the Sentinel-1 burst is from an EW SLC or located south of -60° latitude will return an error. ## Product Packaging @@ -242,9 +207,9 @@ The files available for download include: ### L2 Radiometric Terrain Corrected Static Layer (RTC-STATIC) Files -There are some ancillary products required for RTC processing that change very little through time. Instead of -re-generating these reference products for each Sentinel-1 burst acquisition, they are generated once for each -Sentinel-1 burst ID used for OPERA processing, and archived as `Static Layer` files. +There are some ancillary files generated during the RTC processing workflow that change very little through time. +Instead of including these ancillary products with each OPERA RTC product, they are generated once +for each Sentinel-1 burst ID and archived as `Static Layer` files. Refer to OPERA's [Product Specification Document for the OPERA Radiometric Terrain Corrected SAR Backscatter from Sentinel-1 Static Layers](https://d2pn8kiwq2w21t.cloudfront.net/documents/ProductSpec_RTC-S1-STATIC.pdf "ProductSpec_RTC-S1-STATIC.pdf" ){target=_blank} @@ -281,6 +246,15 @@ search results using the OPERA Burst ID. results for the RTC-STATIC products to be included in the search results, open the Search Filters panel again and select only the RTC-STATIC product type to return only the static layers that correspond to that RTC footprint. +!!! warning "OPERA RTC-STATIC products not available for HH and HH+HV polarized scenes" + + The static layers associated with HH and HH+HV polarized acquisitions, located over some parts of Greenland and + Arctic Canada, are currently unavailable in ASF's archive. We are working with the OPERA team to add these + files to the ASF archive. + + This graphic illustrates the current coverage of OPERA RTC-S1 Static Layers. + ![ExtraFigure](../images/opera-rtc-static-layer-coverage.png "Current coverage of OPERA RTC-S1 Static Layers") + ### Duplicate layer names There are **Local Incidence Angle** and **Mask** files listed in association with both the `RTC` search results @@ -370,6 +344,18 @@ OPERA_L2_RTC-S1_T069-147170-IW1_20210205T163901Z_20220101T140222Z_S1A_30_v1.0_VH OPERA_L2_RTC-S1_T069-147170-IW1_20210205T163901Z_20220101T140222Z_S1A_30_v1.0_mask.tif +## Data Access + +Refer to the [Downloads](../using/downloading.md) page for more information on viewing and downloading +OPERA RTC-S1 On Demand products in Vertex or programmatically. Once processing is complete, download links +for On Demand products are valid for 14 days. + + + + +We plan to support adding standard OPERA RTC-S1 products generated On Demand to the OPERA_L2_RTC-S1_V1 collection, +expanding the archive based on user needs. Stay tuned for that functionality! + ## On-Demand RTC Product Options from ASF In addition to OPERA RTC-S1 products, which use an RTC algorithm available in JPL's open-source @@ -390,7 +376,7 @@ most appropriate for your particular application. ### Spatial Extent OPERA RTC-S1 products are processed on the basis of an individual -[radar burst](https://storymaps.arcgis.com/stories/88c8fe67933340779eddef212d76b8b8 "Sentinel-1 Bursts StoryMap Tutorial" ){target=_blank} +[radar burst](https://storymaps.arcgis.com/stories/88c8fe67933340779eddef212d76b8b8 "Sentinel-1 Bursts StoryMap tutorial" ){target=_blank} extracted from a Sentinel-1 Interferometric Wide-Swath (IW) Single Look Complex (SLC) file, while ASF's RTC GAMMA On Demand products are generated using the Sentinel-1 Level 1 IW SLC or Ground Range Detected (GRD) file. diff --git a/docs/guides/rtc_product_guide.md b/docs/guides/rtc_product_guide.md index 1e17e43d3..29c4a1af6 100644 --- a/docs/guides/rtc_product_guide.md +++ b/docs/guides/rtc_product_guide.md @@ -375,6 +375,16 @@ A shapefile indicating the extent of the RTC data coverage is included in the pa *Table 6: Shapefile files and their extensions* +## Data Access + +Refer to the [Downloads](../using/downloading.md "hyp3-docs.asf.alaska.edu/using/downloading") page for more information on viewing and downloading +Sentinel-1 RTC On Demand products in Vertex or programmatically. Once processing is complete, download links +for On Demand products are valid for 14 days. + +Step-by-step instructions for finding and downloading RTC On Demand products in Vertex are available in the +[Access Products](https://storymaps.arcgis.com/stories/2ead3222d2294d1fae1d11d3f98d7c35#ref-n-xrlWHw "RTC On Demand! StoryMap Tutorial - Data Access"){target=_blank} section of the +[RTC On Demand!](https://storymaps.arcgis.com/stories/2ead3222d2294d1fae1d11d3f98d7c35 "RTC On Demand! StoryMap Tutorial" ){target=_blank} interactive StoryMap tutorial. + ## SAR Scales ### Power Scale @@ -494,15 +504,3 @@ Twele, A., Cao, W., Plank, S. and Martinis, S. 2016. Sentinel-1-based flood mapp Muro, J., Canty, M., Conradsen, K., Hüttich, C., Nielsen, A.A., Skriver, H., Remy, F., Strauch, A., Thonfeld, F. and Menz, G. 2016. Short-Term change detection in wetlands using Sentinel-1 time series. *Remote Sensing*, **8**(10):795. DOI: [10.3390/rs8100795](https://doi.org/10.3390/rs8100795 "https://doi.org/10.3390/rs8100795" ){target=_blank} Rüetschi, M., Schaepman, M.E., Small, D. 2018. Using Multitemporal Sentinel-1 C-band backscatter to monitor phenology and classify deciduous and coniferous forests in Northern Switzerland. *Remote Sensing*, **10**(1):55. DOI: [10.3390/rs10010055](https://doi.org/10.3390/rs10010055 "https://doi.org/10.3390/rs10010055" ){target=_blank} - -## Data Access - -To view or download Sentinel-1 RTC products, please see the links below: - -**Vertex:** [https://search.asf.alaska.edu/](https://search.asf.alaska.edu/ "https://search.asf.alaska.edu" ){target=_blank} - -**API:** [https://asf.alaska.edu/api/](https://asf.alaska.edu/api/ "https://asf.alaska.edu/api" ){target=_blank} - -For details on accessing data, including other SAR datasets, see ASF’s Get Started guide: [https://asf.alaska.edu/getstarted/](https://asf.alaska.edu/getstarted/ "https://asf.alaska.edu/getstarted/" ){target=_blank} - -To access data recipes, which are step-by-step tutorials for processing and working with SAR data, see ASF’s tutorials page: [https://asf.alaska.edu/how-to/data-recipes/data-recipe-tutorials/](https://asf.alaska.edu/how-to/data-recipes/data-recipe-tutorials/ "https://asf.alaska.edu/how-to/data-recipes/data-recipe-tutorials" ){target=_blank} diff --git a/docs/images/download-vertex-multiple.png b/docs/images/download-vertex-multiple.png new file mode 100644 index 000000000..0d02eb81f Binary files /dev/null and b/docs/images/download-vertex-multiple.png differ diff --git a/docs/images/download-vertex-single.png b/docs/images/download-vertex-single.png new file mode 100644 index 000000000..c2e58bf6e Binary files /dev/null and b/docs/images/download-vertex-single.png differ diff --git a/docs/images/on-demand-filters.png b/docs/images/on-demand-filters.png new file mode 100644 index 000000000..1558b4f1e Binary files /dev/null and b/docs/images/on-demand-filters.png differ diff --git a/docs/using/api.md b/docs/using/api.md index eedffb583..34939940b 100644 --- a/docs/using/api.md +++ b/docs/using/api.md @@ -108,10 +108,8 @@ and [spatial extent](../guides/opera_rtc_product_guide.md#spatial-coverage-for-on-demand-opera-rtc-s1-products "OPERA RTC-S1 Product Guide" ){target=_blank} constraints for processing. -Sentinel-1 Burst SLCs submitted for processing must have been acquired between April 14, 2016, and December 31, 2021, -and must have -[OPERA RTC-S1 Static Layer files](../guides/opera_rtc_product_guide.md#l2-radiometric-terrain-corrected-static-layer-rtc-static-files "OPERA RTC-S1 Product Guide" ){target=_blank} -associated with that burst ID. +Sentinel-1 IW Burst SLCs submitted for processing must have been acquired between April 14, 2016, and December 31, 2021, +and must be north of -60° latitude. ```json { @@ -192,23 +190,16 @@ You can submit InSAR jobs using the `INSAR_ISCE_BURST` job type, which takes a s ``` ## Submitting ARIA-S1-GUNW jobs -The ARIA-S1-GUNW job type takes a reference [ESA granule ID](https://sentiwiki.copernicus.eu/web/s1-products#S1-Products-SAR-Naming-Convention){target=_blank} set, a secondary ESA granule ID set, and an ARIA-S1-GUNW Frame ID as input. See the [ARIA-S1-GUNW Product Guide Frame ID section](../guides/gunw_product_guide.md#aria-frame-ids){target=_blank} for more details on these inputs. +The ARIA-S1-GUNW job type takes a reference date, a secondary date, and an ARIA-S1-GUNW Frame ID as input. +See the [ARIA-S1-GUNW Product Guide Frame ID section](../guides/gunw_product_guide.md#aria-frame-ids){target=_blank} for more details on these inputs. ```json { "job_type": "ARIA_S1_GUNW", "name": "gunw-example", "job_parameters": { - "reference": [ - "S1A_IW_SLC__1SDV_20250127T010136_20250127T010203_057623_07199D_4B63", - "S1A_IW_SLC__1SDV_20250127T010111_20250127T010138_057623_07199D_4E88", - "S1A_IW_SLC__1SDV_20250127T010045_20250127T010113_057623_07199D_4D3B" - ], - "secondary": [ - "S1A_IW_SLC__1SDV_20250103T010137_20250103T010204_057273_070BB6_CD45", - "S1A_IW_SLC__1SDV_20250103T010113_20250103T010140_057273_070BB6_1133", - "S1A_IW_SLC__1SDV_20250103T010047_20250103T010115_057273_070BB6_99C5" - ], - "frame_id": 23474 + "reference_date": "2019-12-31", + "secondary_date": "2018-12-12", + "frame_id": 11040 } } ``` diff --git a/docs/using/downloading.md b/docs/using/downloading.md new file mode 100644 index 000000000..7a8c10f05 --- /dev/null +++ b/docs/using/downloading.md @@ -0,0 +1,257 @@ +# Downloading HyP3 Products + +There are a number of interfaces available for downloading products generated On Demand using the HyP3 platform. + +- [On Demand Search](#on-demand-search-in-vertex) interface in Vertex +- [Programmatically](#programmatic-access) using the HyP3 API or HyP3 Python SDK + +## On Demand Search in Vertex + +The [On Demand Search](https://search.asf.alaska.edu/#/?maxResults=1000&searchType=On%20Demand "Vertex On Demand Search" ){target=_blank} +in Vertex allows you to view the status of any job you have submitted for processing and download any product +that has been successfully processed. You will need to sign in with +[Earthdata Login credentials](https://urs.earthdata.nasa.gov/ "urs.earthdata.nasa.gov" ){target=_blank} +before results will display. + +Refer to the +[On Demand Search section of the Vertex User Manual](https://docs.asf.alaska.edu/vertex/manual/#on-demand-products-search-options "docs.asf.alaska.edu" ){target=_blank} +for more information. + +### Downloading Individual Products + +Click on an item in the search results to view download options. You can add the product to the Download Queue using +the **cart icon** in either the Search Result (left) or File (right) pane, or launch a direct download in your +browser window by clicking the **cloud download icon** in the File pane. + +![Downloading Individual Products](../images/download-vertex-single.png "Download Single On Demand Product") + +To view the download queue, click the **Downloads** icon in the top right of the Vertex web interface. +The Download Queue interface provides options for direct download of individual items along with bulk download options. + +### Downloading Multiple Products + +You can add products to the Download Queue one by one using the cart icon, or you can search your On Demand +products for a specific set of products and add them all to the Download Queue using the **Queue** button at the +top of the results list. + +![Downloading Multiple Products](../images/download-vertex-multiple.png "Download Multiple On Demand Products") + +There are a number of options available for filtering your On Demand products, with the most useful being the +**Project Name** field. You can assign a project name when submitting jobs for processing to easily group items +together that are used for the same project. + +Once you've added products to the Download Queue, either by using the individual cart icons or the bulk Queue +button, click the **Downloads** icon in the top right of the Vertex web interface. When you open the Download Queue, +you have the option to launch direct downloads of individual items in the list, or you can choose to remove +individual items from the queue. + +To download all of the products listed in the Download Queue, click the **Data Download** button at the bottom of +the queue window and choose from the available options: + +#### Download Python Script + +The most robust approach for downloading very long lists of products is the +**Download Python Script** option. This downloads a python script that you can launch on your computer. + +- You will be prompted for your Earthdata Login credentials if necessary, then the script will work through the list + of download URLs, downloading them one by one until all of the items have been downloaded. +- If the script is interrupted during the download, you can simply re-run the same script; it will recognize + any products that have already been successfully downloaded and continue with the remaining items. +- To use this option, you must have a Python installation available on your computer. + +#### Download All + +Chrome users may find the **Download All** option useful. This option takes advantage of the multi-threading +capability in Chrome to download several items at a time. + +#### Copy URLs +You can also click on **Copy URLs** (located next to the Data Download button) to copy a list of the download URLs +for the files in your Download Queue, which you can then paste into your own download script. + +## Programmatic Access + +The +[HyP3 API](#accessing-products-using-the-hyp3-api "Jump to the Accessing Products Using the HyP3 API section of this document") +and the +[HyP3 Python SDK](#accessing-products-using-the-hyp3-python-sdk "Jump to the Accessing Products Using the HyP3 Python SDK section of this document") +provide programmatic access to On Demand products. + +The `name` parameter is referred to as "Project Name" in the Vertex interface, but is often referred to as "Job Name" +in documentation for the programmatic interfaces. This parameter is helpful in grouping together jobs submitted as +part of a particular analysis effort. It facilitates management and download of groups of jobs, makes it easier to +share products with colleagues, and allows users to access their HyP3-generated products using the notebooks available +on ASF's [OpenSARLab](https://opensarlab-docs.asf.alaska.edu/ "opensarlab-docs.asf.alaska.edu" ){target=_blank}. + +The +[HyP3 API](#accessing-products-using-the-hyp3-api "Jump to the Accessing Products Using the HyP3 API section of this document") +allows easy access to job URLs through the Swagger UI, but the +[HyP3 Python SDK](#accessing-products-using-the-hyp3-python-sdk "Jump to the Accessing Products Using the HyP3 Python SDK section of this document") +is better suited for scripting search and download workflows for On Demand products. + +### Accessing Products Using the HyP3 API + +The [HyP3 API](../using/api.md "hyp3-docs.asf.alaska.edu/using/api") +provides the ability to +[Query Submitted Jobs](../using/api.md#querying-jobs "hyp3-docs.asf.alaska.edu/using/api/#querying-jobs"). +To look up your On Demand jobs, you will need to have a valid Earthdata Login (asf-urs) session cookie, +which you can get by signing in to +[Vertex](https://search.asf.alaska.edu/ "search.asf.alaska.edu" ){target=_blank} or +[Earthdata Login](https://urs.earthdata.nasa.gov/ "urs.earthdata.nasa.gov" ){target=_blank} +with your Earthdata Login Credentials. + +Note that the parameter fields in the UI are populated with defaults. You will need to edit or delete any of the +default values that do not align with your desired search parameters. The start and end date fields reference +the date/time the jobs were submitted, not the date/time of the acquisitions used to generate the products. + +The returns from your +[Get Jobs API request](https://hyp3-api.asf.alaska.edu/ui/#/default/get_jobs "hyp3-api.asf.alaska.edu/ui/#/default/get_jobs" ){target=_blank} +include download links for the browse images and thumbnails used to display the product contents in Vertex, +as well as the link to the complete product package with a `.zip` extension. + +Here is an example of a response: +``` +{ + "jobs": [ + { + "processing_times": [ + 836.557 + ], + "browse_images": [ + "https://d3gm2hf49xd6jj.cloudfront.net/6f917fec-9c2f-42fb-a55c-3c4fe2e0520b/S1A_IW_20250420T135654_DVP_RTC30_G_gpuned_7959.png", + "https://d3gm2hf49xd6jj.cloudfront.net/6f917fec-9c2f-42fb-a55c-3c4fe2e0520b/S1A_IW_20250420T135654_DVP_RTC30_G_gpuned_7959_rgb.png" + ], + "credit_cost": 5, + "priority": 9960, + "execution_started": true, + "job_id": "6f917fec-9c2f-42fb-a55c-3c4fe2e0520b", + "name": "DEVELOP", + "thumbnail_images": [ + "https://d3gm2hf49xd6jj.cloudfront.net/6f917fec-9c2f-42fb-a55c-3c4fe2e0520b/S1A_IW_20250420T135654_DVP_RTC30_G_gpuned_7959_thumb.png", + "https://d3gm2hf49xd6jj.cloudfront.net/6f917fec-9c2f-42fb-a55c-3c4fe2e0520b/S1A_IW_20250420T135654_DVP_RTC30_G_gpuned_7959_rgb_thumb.png" + ], + "request_time": "2025-06-10T07:47:28+00:00", + "logs": [], + "user_id": "hjkristenson", + "status_code": "SUCCEEDED", + "job_parameters": { + "speckle_filter": false, + "include_inc_map": false, + "dem_name": "copernicus", + "radiometry": "gamma0", + "granules": [ + "S1A_IW_GRDH_1SDV_20250420T135654_20250420T135723_058841_074ABD_E7BB" + ], + "scale": "power", + "dem_matching": false, + "resolution": 30, + "include_rgb": true, + "include_dem": false, + "include_scattering_area": false + }, + "files": [ + { + "s3": { + "bucket": "hyp3-edc-prod-contentbucket-1fv14ed36ifj6", + "key": "6f917fec-9c2f-42fb-a55c-3c4fe2e0520b/S1A_IW_20250420T135654_DVP_RTC30_G_gpuned_7959.zip" + }, + "filename": "S1A_IW_20250420T135654_DVP_RTC30_G_gpuned_7959.zip", + "size": 656581903, + "url": "https://d3gm2hf49xd6jj.cloudfront.net/6f917fec-9c2f-42fb-a55c-3c4fe2e0520b/S1A_IW_20250420T135654_DVP_RTC30_G_gpuned_7959.zip" + } + ], + "expiration_time": "2025-06-25T00:00:00+00:00", + "job_type": "RTC_GAMMA" + }, + ], +} +``` + +The download URL for the full product package is provided near the end of the response for each job in the `jobs` +array of the response JSON. The value is paired with the `url` key in the `files` array. + +You can copy and paste each product URL directly into a browser window, or script a workflow to pull all the +product URLs from the response JSON into a bulk download function. It may be more convenient to use the +[HyP3 Python SDK](#accessing-products-using-the-hyp3-python-sdk "Jump to the Accessing Products Using the HyP3 Python SDK section of this document") +to script bulk download functionality. + +### Accessing Products using the HyP3 Python SDK + +The [HyP3 Python SDK](../using/sdk.md "hyp3-docs.asf.alaska.edu/using/sdk") is a wrapper around the HyP3 API, +and provides convenient search and download functionality for On Demand products (HyP3 jobs). +[This example notebook](https://github.com/ASFHyP3/hyp3-sdk/blob/main/docs/sdk_example.ipynb "SDK Example Notebook" ){target=_blank} +demonstrates how to use the SDK for a range of workflows. + +To access product information using the SDK, you will need to authenticate using the +[HyP3 initializer method](https://hyp3-docs.asf.alaska.edu/using/sdk_api/#hyp3_sdk.HyP3.__init__ "HyP3 SDK API Reference" ){target=_blank}. +You can either add your credentials to your local `netrc` file, or enter your credentials manually. +Refer to the ***Authenticating to the API*** section in the +[SDK Example Notebook](https://github.com/ASFHyP3/hyp3-sdk/blob/main/docs/sdk_example.ipynb "SDK Example Notebook" ){target=_blank} +for guidance. + +Use the `find_jobs` method from the `HyP3` class to generate a list of products to download (batch), then use the +`download_files` method from the `Batch` class to download all the products in the list. Refer to the +[HyP3 SDK API Reference](https://hyp3-docs.asf.alaska.edu/using/sdk_api/#hyp3_sdk "hyp3-docs.asf.alaska.edu/using/sdk_api/#hyp3_sdk" ){target=_blank} +for more information. + +## Product Packaging and Extraction + +On Demand products from ASF are delivered as zip files. The files contained in the zip archive vary by product type, +but the zip archive always includes an internal directory containing all the individual files. The directory names +can be quite long, and some users (particularly those using a Windows operating system) will need to make +accommodations in order to successfully extract the contents. + +### Extracting Product Packages + +When extracting the contents of a HyP3-generated zip file, you may need to specify a destination directory to prevent +the extraction of the internal directory to a directory named with the full zip file name. For many of the products, +this combination of directories would result in paths that are longer than can be used with Windows operating systems. + +### Downloading Individual Files + +Downloading the full zip file ensures that you have all of the data products as well as auxiliary files and +relevant metadata, but some users may not require all of the files included in the product zip archive. + +The contents of the zip files can all be accessed directly. Simply replace the `.zip` at the end of the download URL +with the tag for the specific file you want to download. + +For example, for the following download URL for an RTC On Demand product: + + https://d3gm2hf49xd6jj.cloudfront.net/76b1a849-c826-428a-966c-55f8bb88f814/ + S1A_IW_20250502T135654_DVP_RTC30_G_gpuned_70DD.zip + +simply replace the `.zip` with the desired product extension, such as `_VV.tif` for the RTC GeoTIFF in VV polarization: + + https://d3gm2hf49xd6jj.cloudfront.net/76b1a849-c826-428a-966c-55f8bb88f814/ + S1A_IW_20250502T135654_DVP_RTC30_G_gpuned_70DD_VV.tif + +You can then paste that URL into a browser window, or use it in a download script, to download only the designated +product rather than the full zip archive. + +*The zip archive contains valuable metadata products, including a readme file that provides information +about the workflow used to generate the product and the files included in the product package.* +***New users are advised to download the full archive*** *to ensure they have access to this information +and can determine what individual products are required for their application.* + +## Downloading Products Submitted by Other Users + +You can search for On Demand products processed under a different username. This functionality is a convenient +way to share products when collaborating with others, and can be accessed both in +[Vertex](#using-vertex "Jump to the Using Vertex section of this document") and the +[HyP3 Python SDK](#using-the-hyp3-python-sdk "Jump to the Using the HyP3 Python SDK section of this document"). + +### Using Vertex + +Click the **Filters** button to open the On Demand Search **Filters** window, and use +the **User ID** filter to enter the username under which the desired job was submitted. + +![On Demand Filters](../images/on-demand-filters.png) + +If the user who submitted the job also provides you with the **Project Name**, you can apply that search filter, +as well. The drop-down list in the **Project Name** field only displays the list for the user who is logged in, so you +will not be able to look up another user's list of Project Names using this interface. + +### Using the HyP3 Python SDK + +The HyP3 Python SDK provides the capability to search for products submitted by other users. Refer to +[this notebook](https://github.com/ASFHyP3/hyp3-sdk/blob/main/docs/search_other_user_jobs.ipynb "Using the HyP3 SDK to search for jobs run by another user" ){target=_blank} +to learn how. diff --git a/mkdocs.yml b/mkdocs.yml index 262e6a5c9..196285e60 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -57,6 +57,7 @@ nav: - API Reference: using/sdk_api.md - API: using/api.md - Credits: using/credits.md + - Downloads: using/downloading.md - Subscriptions: using/subscriptions.md - Products: - products.md @@ -78,7 +79,7 @@ nav: - Product Guide: guides/gunw_product_guide.md - AutoRIFT: https://its-live.jpl.nasa.gov/" target="_blank - Usage Guidelines: usage_guidelines.md - - Sentinel-1 Mission: sentinel1.md + - Sentinel-1 Mission: sentinel1.md - SAR Basics: - guides/introduction_to_sar.md - Introduction to SAR: guides/introduction_to_sar.md