pyWaPOR

downloads version Open In Colab

This repository contains a Python implementation of the algorithm used to generate the WaPOR datasets. It can be used to calculate evaporation, transpiration and biomass production maps.

Installation

Its recommended to install in a clean conda environment and use conda to install all the important packages from the conda-forge channel.

conda create -n my_pywapor_env --yes -c conda-forge python pip gdal pydap numpy pandas requests matplotlib pyproj scipy pycurl pyshp joblib bs4 rasterio xarray bottleneck geojson tqdm dask rioxarray pyvis shapely lxml cachetools cdsapi sentinelsat geopy numba

conda activate my_pywapor_env

Then use the package manager pip to install pywapor.

pip install pywapor

Usage

To run the model for one dekad (from 2021-07-01 to 2021-07-11 in this case) for the Fayoum irrigation scheme in Egypt (but feel free to change the boundingbox defined by latlim and lonlim) using mainly MODIS data, run the following code (run python in your console to activate Python and exit() to deactivate).

import pywapor

# User inputs.
timelim = ["2021-07-01", "2021-07-11"]
latlim = [28.9, 29.7]
lonlim = [30.2, 31.2]
project_folder = r"/my_first_ETLook_run/"

# Download and prepare input data.
ds_in  = pywapor.pre_et_look.main(project_folder, latlim, lonlim, timelim)

# Run the model.
ds_out = pywapor.et_look.main(ds_in)

Check out the documentation and the notebooks below to learn more!

Documentation

Notebooks
Name Colab
1. Introduction colab
2. Levels colab
3. Composites colab
4. Sideloading colab
5. Enhancers colab
6. pyWaPOR vs. WaPOR colab

WaPOR v3

WaPOR v2

WaPOR v1

Acknowledgments

The methodology for WaPOR was developed by the FRAME1 consortium, consisting of eLEAF (lead), VITO, ITC, University of Twente and Waterwatch foundation, commissioned by and in partnership with the Land and Water Division of FAO. The method for calculating evapotranspiration is based on the ETLook model developed by eLEAF in 2010. The method for calculating total biomass production is based on the C-Fix model.

The code in the pywapor.et_look_v2_v3 module of this repository, containing all core physical functions used by ETLook, was written by Henk Pelgrum (eLEAF) and Rutger Kassies (eLEAF). The remaining modules have been developed by Bert Coerver (FAO), Tim Hessels (WaterSat), and, in the framework of the ESA-funded ET4FAO project, Radoslaw Guzinski (DHI-GRAS), Hector Nieto (Complutig) and Laust Faerch (DHI-GRAS).

Contact

For questions, requests or issues with this repository, please contact Bert Coerver at bert.coerver@fao.org or the WaPOR team at wapor@fao.org.

Release Notes

3.2.0 (2023-01-03)
  • - Full Landsat support, automatically download and process Landsat (5-9) data.
  • - Fixed a bug that caused some MODIS data to be missing inside the selected boundingbox.
  • - You can now re-enter a password in case you've provided an incorrect one.
  • - Bugfixes (including more Windows specific ones).
  • - Updated weights to calculate albedo for Landsat and Sentinel.

3.1.0 (2022-09-22)
  • - Added a thermal sharpening algorithm (pyDMS) to increase LST resolution.
  • - Now, when after several failed attempts to download a variable, the code will continue processing other variables.
  • - Improved cloud-masking for VIIRSL1.
  • - Bugfixes (including several Windows specific ones).
  • - More information in the log.

3.0.0 (2022-08-31)
  • - Bugfixes. Most noteably, server side errors when downloading data are now handeled better, i.e. collect tools will retry several times when a download fails, but many other smaller issues have been fixed too.
  • - Performance improvements, mostly due to fewer reprojections.
  • - Better logging. The logs from SENTINEL and ERA5 are now directed to seperate channels and logs now show peak-memory-usage for critical calculation steps.
  • - `et_look` and `se_root` now accept a `chunks` keyword to adjust the chunksizes at which the calculations are done. Increase them if you have a lot of RAM available, decrease them for slower calculations but with less memory usage.
  • - Support for WaPOR v3 methodology. Choose `et_look_version = "v3"` and `se_root_version = "v3"` when running the respective models (`et_look` and `se_root`).
  • - Default configurations for WaPOR v3 input datasets, i.e. choose `sources = "level_2_v3"` when running `pre_et_look` or `pre_se_root`.
  • - New collect functions for COPERNICUS DEM.
  • - The data structure for STATICS is now consistent with the other products.

2.6.0 (2022-08-04)
  • - New collect functions for VIIRS (Level-1), SENTINEL-2, SENTINEL-3 and (ag)ERA5.
  • - pyWaPOR now works with Python versions greater than 3.8.

2.5.0 (2022-06-23)
  • - Rewritten collect tools.
  • - The entire workflow now works with netCDF.
  • - All the netCDF files are formatted to support the rio-acccessor.

2.4.2 (2022-04-26)
  • - New biomass module and NPP calculation.

2.4.1 (2022-03-11)
  • - NetCDF files are now compressed when saved to disk.
  • - Calculation of Total Biomass Production is now turned on by default.
  • - It is no longer required to provide all input variables to et_look, the model will calculate as many variables as possible with the given data. For example, if you are only interested in acquiring interception rates, it would now suffice to only prepare ndvi and p (precipitation) data with pre_et_look.
  • - et_look now automatically generates an interactive network graph visualising the executed computation steps.

2.4.0 (2022-02-03)
  • - Easily apply your own functions to data, i.e. use your own custom filters, gap-fillers etc.
  • - Side-load your own data, i.e. easily incorporate you own datasets.
  • - Added functions to process Landsat Level-2 to “ndvi”, “lst” and “r0”.
  • - Data is now stored and processed as netCDF (using xarray and dask).
  • - Calculations in et_look() and se_root() are now done in chunks, instead of using a for-loop.
  • - Some previously constant parameters now have spatial variability.
  • - Improved logging.
  • - Download functions now show progress and download-speed.
  • - MODIS datasets switched from v6.0 (decommissioned soon) to v6.1.
  • - The lapse-rate correction to temperature data is now more accurate and faster.
  • - VITO and WAPOR passwords are now checked when entered.
  • - Other bug-fixes and performance improvements.

2.3.0 (2021-11-19)
  • - Automatically create input composites before running ETLook.
  • - Choose composite lengths in number of days or dekads.
  • - Option to choose which products to use per variable.
  • - Calculate soil saturation separate from ETLook.
  • - PROBA-V support for NDVI and Albedo inputs.
  • - Define diagnostics pixels, for which extra outputs are created (e.g. charts, maps etc.).
  • - Bug-fixes and performance improvements.

License

APACHE