Levels
In this notebook we’ll take a look at how to run et_look at different levels than the standard level_1.
[1]:
!pip install pywapor --quiet
First we redefine some of the basic inputs variables that we’ve also seen in the previous notebooks.
[1]:
import pywapor
project_folder = r"/Users/hmcoerver/pywapor_notebooks_2"
latlim = [28.9, 29.7]
lonlim = [30.2, 31.2]
timelim = ["2021-07-01", "2021-07-11"]
composite_length = "DEKAD"
OMP: Info #276: omp_set_nested routine deprecated, please use omp_set_max_active_levels instead.
Level-2 inputs
Now that we have run ETLook for a simple use case, we’ll have a look at how to run it at different levels and with different input datasets.
When downloading input data with pywapor.pre_et_look.main as we did before, it is possible to select a level for which to download data. By default, this level is set to level_1.
We can have a closer look at what this actually means by running the following functions.
[2]:
level = "level_1"
sources = pywapor.general.levels.pre_et_look_levels(level)
sources
[2]:
{'ndvi': {'products': [{'source': 'MODIS',
'product_name': 'MOD13Q1.061',
'enhancers': 'default',
'is_example': True},
{'source': 'MODIS', 'product_name': 'MYD13Q1.061', 'enhancers': 'default'}],
'composite_type': 'mean',
'temporal_interp': 'linear',
'spatial_interp': 'nearest'},
'r0': {'products': [{'source': 'MODIS',
'product_name': 'MCD43A3.061',
'enhancers': 'default'}],
'composite_type': 'mean',
'temporal_interp': 'linear',
'spatial_interp': 'bilinear'},
'se_root': {'products': [{'source': functools.partial(<function se_root at 0x14041dea0>, bin_length='DEKAD', sources='level_1'),
'product_name': 'v2',
'enhancers': 'default'}],
'composite_type': 'max',
'temporal_interp': None,
'spatial_interp': 'bilinear'},
'p': {'products': [{'source': 'CHIRPS',
'product_name': 'P05',
'enhancers': 'default'}],
'composite_type': 'mean',
'temporal_interp': 'linear',
'spatial_interp': 'bilinear'},
'z': {'products': [{'source': 'SRTM',
'product_name': '30M',
'enhancers': 'default'}],
'composite_type': None,
'temporal_interp': None,
'spatial_interp': 'bilinear'},
'ra': {'products': [{'source': 'MERRA2',
'product_name': 'M2T1NXRAD.5.12.4',
'enhancers': 'default'}],
'composite_type': 'mean',
'temporal_interp': 'linear',
'spatial_interp': 'bilinear'},
't_air': {'products': [{'source': 'GEOS5',
'product_name': 'inst3_2d_asm_Nx',
'enhancers': 'default'}],
'composite_type': 'mean',
'temporal_interp': 'linear',
'spatial_interp': 'bilinear'},
't_air_max': {'products': [{'source': 'GEOS5',
'product_name': 'inst3_2d_asm_Nx',
'enhancers': 'default'}],
'composite_type': 'max',
'temporal_interp': 'linear',
'spatial_interp': 'bilinear'},
't_air_min': {'products': [{'source': 'GEOS5',
'product_name': 'inst3_2d_asm_Nx',
'enhancers': 'default'}],
'composite_type': 'min',
'temporal_interp': 'linear',
'spatial_interp': 'bilinear'},
'u2m': {'products': [{'source': 'GEOS5',
'product_name': 'inst3_2d_asm_Nx',
'enhancers': 'default'}],
'composite_type': 'mean',
'temporal_interp': 'linear',
'spatial_interp': 'bilinear'},
'v2m': {'products': [{'source': 'GEOS5',
'product_name': 'inst3_2d_asm_Nx',
'enhancers': 'default'}],
'composite_type': 'mean',
'temporal_interp': 'linear',
'spatial_interp': 'bilinear'},
'qv': {'products': [{'source': 'GEOS5',
'product_name': 'inst3_2d_asm_Nx',
'enhancers': 'default'}],
'composite_type': 'mean',
'temporal_interp': 'linear',
'spatial_interp': 'bilinear'},
'p_air': {'products': [{'source': 'GEOS5',
'product_name': 'inst3_2d_asm_Nx',
'enhancers': 'default'}],
'composite_type': 'mean',
'temporal_interp': 'linear',
'spatial_interp': 'bilinear'},
'p_air_0': {'products': [{'source': 'GEOS5',
'product_name': 'inst3_2d_asm_Nx',
'enhancers': 'default'}],
'composite_type': 'mean',
'temporal_interp': 'linear',
'spatial_interp': 'bilinear'},
'wv': {'products': [{'source': 'GEOS5',
'product_name': 'inst3_2d_asm_Nx',
'enhancers': 'default'}],
'composite_type': 'mean',
'temporal_interp': 'linear',
'spatial_interp': 'bilinear'},
'land_mask': {'products': [{'source': 'GLOBCOVER',
'product_name': '2009_V2.3_Global',
'enhancers': 'default'}],
'composite_type': None,
'temporal_interp': None,
'spatial_interp': 'nearest'},
'rs_min': {'products': [{'source': 'GLOBCOVER',
'product_name': '2009_V2.3_Global',
'enhancers': 'default'}],
'composite_type': None,
'temporal_interp': None,
'spatial_interp': 'nearest'},
'z_obst_max': {'products': [{'source': 'GLOBCOVER',
'product_name': '2009_V2.3_Global',
'enhancers': 'default'}],
'composite_type': None,
'temporal_interp': None,
'spatial_interp': 'nearest'},
'lw_offset': {'products': [{'source': 'STATICS',
'product_name': 'WaPOR2',
'enhancers': 'default'}],
'composite_type': None,
'temporal_interp': None,
'spatial_interp': 'bilinear'},
'lw_slope': {'products': [{'source': 'STATICS',
'product_name': 'WaPOR2',
'enhancers': 'default'}],
'composite_type': None,
'temporal_interp': None,
'spatial_interp': 'bilinear'},
'z_oro': {'products': [{'source': 'STATICS',
'product_name': 'WaPOR2',
'enhancers': 'default'}],
'composite_type': None,
'temporal_interp': None,
'spatial_interp': 'bilinear'},
'rn_offset': {'products': [{'source': 'STATICS',
'product_name': 'WaPOR2',
'enhancers': 'default'}],
'composite_type': None,
'temporal_interp': None,
'spatial_interp': 'bilinear'},
'rn_slope': {'products': [{'source': 'STATICS',
'product_name': 'WaPOR2',
'enhancers': 'default'}],
'composite_type': None,
'temporal_interp': None,
'spatial_interp': 'bilinear'},
't_amp_year': {'products': [{'source': 'STATICS',
'product_name': 'WaPOR2',
'enhancers': 'default'}],
'composite_type': None,
'temporal_interp': None,
'spatial_interp': 'bilinear'},
't_opt': {'products': [{'source': 'STATICS',
'product_name': 'WaPOR2',
'enhancers': 'default'}],
'composite_type': None,
'temporal_interp': None,
'spatial_interp': 'bilinear'},
'vpd_slope': {'products': [{'source': 'STATICS',
'product_name': 'WaPOR2',
'enhancers': 'default'}],
'composite_type': None,
'temporal_interp': None,
'spatial_interp': 'bilinear'}}
This dictionary specifices for each variable which products are used (and configures a couple of other settings that we’ll deal with in the next notebook). Let’s have a closer look at the sources for ndvi.
[3]:
sources["ndvi"]
[3]:
{'products': [{'source': 'MODIS',
'product_name': 'MOD13Q1.061',
'enhancers': 'default',
'is_example': True},
{'source': 'MODIS', 'product_name': 'MYD13Q1.061', 'enhancers': 'default'}],
'composite_type': 'mean',
'temporal_interp': 'linear',
'spatial_interp': 'nearest'}
You can see that at level_1, ndvi data comes from two products, i.e. MODIS.MOD13Q1.061 and MODIS.MYD13Q1.061. In a similar way you can check the sources for other variables, e.g. try running sources["r0"]. Also note that MODIS.MOD13Q1.061 has a property called is_example. This property defines which product is used to align all the other datasets spatially.
Let’s have a look at the datasets used for ndvi and r0 at level_2.
[4]:
sources = pywapor.general.levels.pre_et_look_levels("level_2")
sources["ndvi"]
[4]:
{'products': [{'source': 'PROBAV',
'product_name': 'S5_TOC_100_m_C1',
'enhancers': 'default',
'is_example': True}],
'composite_type': 'mean',
'temporal_interp': 'linear',
'spatial_interp': 'nearest'}
[5]:
sources["r0"]
[5]:
{'products': [{'source': 'PROBAV',
'product_name': 'S5_TOC_100_m_C1',
'enhancers': 'default'}],
'composite_type': 'mean',
'temporal_interp': 'linear',
'spatial_interp': 'nearest'}
As you can see, the difference between level_1 and level_2 is the dataset used to create the NDVI and ALBEDO inputs. For level_2, they are derived from PROBA-V images, which have a higher resolution (~100m) than the MODIS products (~250m for NDVI).
Let’s give level_2 a try!
⚠️
You’ll notice that you’ll be prompted for another username and password. This time for a VITO account. We could set this account up by running
pywapor.collect.setup_dl_accounts.setup_account("VITO")as well (like we did before for the NASA and WAPOR accounts). Butpywapor.pre_et_look.mainalso checks if the required accounts can be found and gives a prompt if thats’s not the case.You can create a VITO account here.
⚠️
We run pywapor.pre_et_look.main the same way as we’ve done previously, but now we add the parameter specifying the level (which has a default value of "level_1") and the composite_length of 1 day (instead of the default "DEKAD").
[6]:
level = "level_2"
ds = pywapor.pre_et_look.main(project_folder, latlim, lonlim, timelim,
sources = level,
bin_length = composite_length)
> PRE_ET_LOOK
--> Collecting `ndvi`, `r0` from `PROBAV.S5_TOC_100_m_C1`.
--> Searching PROBAV tiles.
--> Downloading 3 PROBAV tiles.
--> Converting PROBAV_S5_TOC_X21Y04_20210701_100M_V101.HDF5 to netcdf.
> peak-memory-usage: 672.3MB, execution-time: 0:00:10.653927.
> chunksize|dimsize: [y: 5792|10080, x: 5792|10080]
--> Converting PROBAV_S5_TOC_X21Y04_20210706_100M_V101.HDF5 to netcdf.
> peak-memory-usage: 672.4MB, execution-time: 0:00:10.333620.
> chunksize|dimsize: [y: 5792|10080, x: 5792|10080]
--> Converting PROBAV_S5_TOC_X21Y04_20210711_100M_V101.HDF5 to netcdf.
> peak-memory-usage: 672.4MB, execution-time: 0:00:10.395909.
> chunksize|dimsize: [y: 5792|10080, x: 5792|10080]
--> Merging files.
> peak-memory-usage: 7.3KB, execution-time: 0:00:02.049498.
> chunksize|dimsize: [time: 3|3, y: 808|808, x: 1009|1009]
> timesize: 3 [2021-07-03T12:00, ..., 2021-07-13T12:00]
--> Collecting `p` from `CHIRPS.P05`.
--> Saving merged data.
> peak-memory-usage: 570.1KB, execution-time: 0:00:02.059141.
> chunksize|dimsize: [time: 17|17, y: 18|18, x: 22|22]
> timesize: 17 [2021-06-28T00:30, ..., 2021-07-14T00:30]
--> Collecting `z` from `SRTM.30M`.
--> Saving merged data.
> peak-memory-usage: 118.8MB, execution-time: 0:00:02.739834.
> chunksize|dimsize: [time: 1|1, y: 2883|2883, x: 3603|3603]
> timesize: 1 [2000-02-11T00:00, ..., 2000-02-11T00:00]
--> Collecting `ra` from `MERRA2.M2T1NXRAD.5.12.4`.
--> Saving merged data.
> peak-memory-usage: 587.4KB, execution-time: 0:00:02.061495.
> chunksize|dimsize: [time: 408|408, y: 4|4, x: 3|3]
> timesize: 408 [2021-06-28T01:00, ..., 2021-07-15T00:00]
--> Collecting `t_air`, `t_air_max`, `t_air_min`, `u2m`, `v2m`, `qv`, `p_air`, `p_air_0`, `wv` from `GEOS5.inst3_2d_asm_Nx`.
--> Downloading data.
> peak-memory-usage: 1.3MB, execution-time: 0:00:30.158525.
> chunksize|dimsize: [time: 129|129, y: 5|5, x: 5|5]
--> Applying 'kelvin_to_celsius' to `t_air`.
--> Applying 'kelvin_to_celsius' to `t_air_max`.
--> Applying 'kelvin_to_celsius' to `t_air_min`.
--> Applying 'pa_to_kpa' to `p_air`.
--> Applying 'pa_to_kpa' to `p_air_0`.
--> Saving netCDF.
> peak-memory-usage: 1.9MB, execution-time: 0:00:02.131468.
> chunksize|dimsize: [time: 129|129, y: 5|5, x: 5|5]
> timesize: 129 [2021-06-28T01:30, ..., 2021-07-14T01:30]
--> Collecting `land_mask`, `rs_min`, `z_obst_max` from `GLOBCOVER.2009_V2.3_Global`.
--> Calculating `land_mask` from `lulc`.
--> Calculating `rs_min` from `lulc`.
--> Calculating `z_obst_max` from `lulc`.
--> Saving 2009_V2.3_Global.nc.
> peak-memory-usage: 4.4MB, execution-time: 0:00:02.099130.
> chunksize|dimsize: [y: 295|295, x: 367|367]
--> Collecting `lw_offset`, `lw_slope`, `z_oro`, `rn_offset`, `rn_slope`, `t_amp_year`, `t_opt`, `vpd_slope` from `STATICS.WaPOR2`.
--> Saving WaPOR2.nc.
> peak-memory-usage: 21.3MB, execution-time: 0:00:02.165801.
> chunksize|dimsize: [y: 403|403, x: 493|493]
--> Collecting `se_root` from `se_root.v2`.
> PRE_SE_ROOT
--> Collecting `lst` from `MODIS.MOD11A1.061`.
--> Applying 'mask_bitwise_qa' to `lst`.
--> Applying 'expand_time_dim' to `lst`.
--> Saving merged data.
> peak-memory-usage: 4.5MB, execution-time: 0:00:02.077309.
> chunksize|dimsize: [time: 22|22, y: 104|104, x: 130|130]
> timesize: 22 [2021-06-28T10:54, ..., 2021-07-14T10:54]
--> Collecting `lst` from `MODIS.MYD11A1.061`.
--> Applying 'mask_bitwise_qa' to `lst`.
--> Applying 'expand_time_dim' to `lst`.
--> Saving merged data.
> peak-memory-usage: 6.6MB, execution-time: 0:00:02.073125.
> chunksize|dimsize: [time: 32|32, y: 104|104, x: 130|130]
> timesize: 32 [2021-06-28T12:30, ..., 2021-07-14T12:36]
--> Collecting `ndvi` from `PROBAV.S5_TOC_100_m_C1`.
> timesize: 3 [2021-07-03T12:00, ..., 2021-07-13T12:00]
--> Collecting `t_air`, `t_air_max`, `t_air_min`, `u2m`, `v2m`, `qv`, `wv`, `p_air`, `p_air_0` from `GEOS5.inst3_2d_asm_Nx`.
> timesize: 129 [2021-06-28T01:30, ..., 2021-07-14T01:30]
--> Collecting `r0_bare`, `r0_full` from `STATICS.WaPOR2`.
--> Saving WaPOR2.nc.
--> Appending data to an existing file.
> peak-memory-usage: 7.5MB, execution-time: 0:00:02.120565.
> chunksize|dimsize: [y: 403|403, x: 493|493]
--> Spatially aligning 2 `lst` products together.
--> Using `MYD11A1.061.nc` as reprojecting example.
> shape: (104, 130), res: 0.0079° x 0.0079°.
--> Aligning times in `ndvi` (3) with `lst` (54, linear).
> peak-memory-usage: 587.0MB, execution-time: 0:00:30.684434.
> chunksize|dimsize: [time: 54|54, y: 500|808, x: 500|1009]
> shape: (54, 808, 1009), kwargs: [].
--> Aligning times in `p_air` (129) with `lst` (54, linear).
> peak-memory-usage: 712.0KB, execution-time: 0:00:02.045801.
> chunksize|dimsize: [time: 54|54, y: 5|5, x: 5|5]
> shape: (54, 5, 5), kwargs: [].
--> Aligning times in `p_air_0` (129) with `lst` (54, linear).
> peak-memory-usage: 723.2KB, execution-time: 0:00:02.062849.
> chunksize|dimsize: [time: 54|54, y: 5|5, x: 5|5]
> shape: (54, 5, 5), kwargs: [].
--> Aligning times in `qv` (129) with `lst` (54, linear).
> peak-memory-usage: 723.5KB, execution-time: 0:00:02.066948.
> chunksize|dimsize: [time: 54|54, y: 5|5, x: 5|5]
> shape: (54, 5, 5), kwargs: [].
--> Aligning times in `t_air` (129) with `lst` (54, linear).
> peak-memory-usage: 732.0KB, execution-time: 0:00:02.069571.
> chunksize|dimsize: [time: 54|54, y: 5|5, x: 5|5]
> shape: (54, 5, 5), kwargs: [].
--> Aligning times in `t_air_max` (129) with `lst` (54, linear).
> peak-memory-usage: 133.9KB, execution-time: 0:00:02.070025.
> chunksize|dimsize: [time: 54|54, y: 5|5, x: 5|5]
> shape: (54, 5, 5), kwargs: [].
--> Aligning times in `t_air_min` (129) with `lst` (54, linear).
> peak-memory-usage: 597.6KB, execution-time: 0:00:02.072299.
> chunksize|dimsize: [time: 54|54, y: 5|5, x: 5|5]
> shape: (54, 5, 5), kwargs: [].
--> Aligning times in `u2m` (129) with `lst` (54, linear).
> peak-memory-usage: 723.8KB, execution-time: 0:00:02.067245.
> chunksize|dimsize: [time: 54|54, y: 5|5, x: 5|5]
> shape: (54, 5, 5), kwargs: [].
--> Aligning times in `v2m` (129) with `lst` (54, linear).
> peak-memory-usage: 723.7KB, execution-time: 0:00:02.067488.
> chunksize|dimsize: [time: 54|54, y: 5|5, x: 5|5]
> shape: (54, 5, 5), kwargs: [].
--> Aligning times in `wv` (129) with `lst` (54, linear).
> peak-memory-usage: 723.9KB, execution-time: 0:00:02.046528.
> chunksize|dimsize: [time: 54|54, y: 5|5, x: 5|5]
> shape: (54, 5, 5), kwargs: [].
--> Using `S5_TOC_100_m_C1.nc` as reprojecting example.
> shape: (808, 1009), res: 0.0010° x 0.0010°.
--> Selected `reproject_chunk` for reprojection of MOD11A1.061.nc.
--> Correcting src_ds.
> peak-memory-usage: 27.9MB, execution-time: 0:00:02.104781.
> chunksize|dimsize: [time: 54|54, y: 104|104, x: 130|130]
--> Warping VRT to netCDF.
> peak-memory-usage: 6.6KB, execution-time: 0:00:01.215254.
--> Saving reprojected data from MOD11A1.061_fixed.nc:lst (nearest).
> peak-memory-usage: 1.3GB, execution-time: 0:00:02.462186.
> chunksize|dimsize: [time: 39|54, y: 586|808, x: 732|1009]
--> Selected `reproject_chunk` for reprojection of p_air_i.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 6.3KB, execution-time: 0:00:02.127818.
--> Saving reprojected data from p_air_i.nc:p_air (bilinear).
> peak-memory-usage: 995.3MB, execution-time: 0:00:02.496411.
> chunksize|dimsize: [time: 39|54, y: 586|808, x: 732|1009]
--> Selected `reproject_chunk` for reprojection of p_air_0_i.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 6.7KB, execution-time: 0:00:02.031158.
--> Saving reprojected data from p_air_0_i.nc:p_air_0 (bilinear).
> peak-memory-usage: 1.2GB, execution-time: 0:00:02.478908.
> chunksize|dimsize: [time: 39|54, y: 586|808, x: 732|1009]
--> Selected `reproject_chunk` for reprojection of qv_i.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 6.3KB, execution-time: 0:00:02.002310.
--> Saving reprojected data from qv_i.nc:qv (bilinear).
> peak-memory-usage: 1.2GB, execution-time: 0:00:02.422581.
> chunksize|dimsize: [time: 39|54, y: 586|808, x: 732|1009]
--> Selected `reproject_chunk` for reprojection of WaPOR2.nc.
--> Correcting src_ds.
> peak-memory-usage: 6.1MB, execution-time: 0:00:02.063886.
> chunksize|dimsize: [y: 403|403, x: 493|493]
--> Warping VRT to netCDF.
> peak-memory-usage: 4.6KB, execution-time: 0:00:00.052213.
--> Saving reprojected data from WaPOR2_fixed.nc:r0_bare (bilinear).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.100496.
> chunksize|dimsize: [y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of WaPOR2.nc.
--> Correcting src_ds.
> peak-memory-usage: 6.1MB, execution-time: 0:00:02.059941.
> chunksize|dimsize: [y: 403|403, x: 493|493]
--> Warping VRT to netCDF.
> peak-memory-usage: 4.5KB, execution-time: 0:00:00.052462.
--> Saving reprojected data from WaPOR2_fixed.nc:r0_full (bilinear).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.098670.
> chunksize|dimsize: [y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of t_air_i.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 6.3KB, execution-time: 0:00:01.965782.
--> Saving reprojected data from t_air_i.nc:t_air (bilinear).
> peak-memory-usage: 1.3GB, execution-time: 0:00:02.538999.
> chunksize|dimsize: [time: 39|54, y: 586|808, x: 732|1009]
--> Selected `reproject_chunk` for reprojection of t_air_max_i.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 6.3KB, execution-time: 0:00:01.974541.
--> Saving reprojected data from t_air_max_i.nc:t_air_max (bilinear).
> peak-memory-usage: 1.2GB, execution-time: 0:00:02.654904.
> chunksize|dimsize: [time: 39|54, y: 586|808, x: 732|1009]
--> Selected `reproject_chunk` for reprojection of t_air_min_i.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 6.3KB, execution-time: 0:00:02.049868.
--> Saving reprojected data from t_air_min_i.nc:t_air_min (bilinear).
> peak-memory-usage: 1.1GB, execution-time: 0:00:02.551579.
> chunksize|dimsize: [time: 39|54, y: 586|808, x: 732|1009]
--> Selected `reproject_chunk` for reprojection of u2m_i.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 6.3KB, execution-time: 0:00:01.989025.
--> Saving reprojected data from u2m_i.nc:u2m (bilinear).
> peak-memory-usage: 1.2GB, execution-time: 0:00:02.609529.
> chunksize|dimsize: [time: 39|54, y: 586|808, x: 732|1009]
--> Selected `reproject_chunk` for reprojection of v2m_i.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 6.5KB, execution-time: 0:00:01.974857.
--> Saving reprojected data from v2m_i.nc:v2m (bilinear).
> peak-memory-usage: 1.2GB, execution-time: 0:00:02.528068.
> chunksize|dimsize: [time: 39|54, y: 586|808, x: 732|1009]
--> Selected `reproject_chunk` for reprojection of wv_i.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 6.6KB, execution-time: 0:00:01.968493.
--> Saving reprojected data from wv_i.nc:wv (bilinear).
> peak-memory-usage: 1.2GB, execution-time: 0:00:02.622215.
> chunksize|dimsize: [time: 39|54, y: 586|808, x: 732|1009]
--> Applying 'rename_meteo'.
--> Applying 'remove_empty_statics'.
--> Applying 'add_constants_new'.
--> Applying 'bt_to_lst'.
--> Applying 'drop_empty_times'.
--> Creating merged file `se_root_in.nc`.
> peak-memory-usage: 463.0MB, execution-time: 0:00:58.304156.
> chunksize|dimsize: [time: 39|54, y: 586|808, x: 732|1009]
< PRE_SE_ROOT (0:03:14.523363)
> SE_ROOT
--> Running `se_root` (v2).
--> Calculating `seasonal_correction`.
--> Calculating `declination`.
--> Calculating `day_angle`.
--> Calculating `air_pressure_kpa2mbar`.
--> Calculating `air_pressure_kpa2mbar`.
--> Calculating `vegetation_cover`.
--> Calculating `air_temperature_kelvin_inst`.
--> Calculating `air_temperature_kelvin`.
--> Calculating `vapour_pressure_from_specific_humidity_inst`.
--> Calculating `vapour_pressure_from_specific_humidity`.
--> Calculating `specific_humidity_from_vapour_pressure`.
--> Calculating `moist_air_density_inst`.
--> Calculating `moist_air_density`.
--> Calculating `dry_air_density_inst`.
--> Calculating `dry_air_density`.
--> Calculating `air_density_inst`.
--> Calculating `air_density`.
--> Calculating `wind_speed_blending_height_bare`.
--> Calculating `longitude_rad`.
--> Calculating `latitude_rad`.
--> Calculating `hour_angle`.
--> Calculating `inverse_earth_sun_distance`.
--> Calculating `solar_elevation_angle`.
--> Calculating `solar_elevation_angle_refracted`.
--> Calculating `relative_optical_airmass`.
--> Calculating `rayleigh_optical_thickness`.
--> Calculating `linke_turbidity`.
--> Calculating `extraterrestrial_irradiance_normal`.
--> Calculating `beam_irradiance_normal_clear`.
--> Calculating `beam_irradiance_horizontal_clear`.
--> Calculating `diffuse_irradiance_horizontal_clear`.
--> Calculating `ra_clear_horizontal`.
--> Calculating `atmospheric_emissivity_inst`.
--> Calculating `net_radiation_bare`.
--> Calculating `net_radiation_full`.
--> Calculating `sensible_heat_flux_bare`.
--> Calculating `sensible_heat_flux_full`.
--> Calculating `wind_speed_blending_height_full_inst`.
--> Calculating `friction_velocity_bare_inst`.
--> Calculating `friction_velocity_full_inst`.
--> Calculating `monin_obukhov_length_bare`.
--> Calculating `monin_obukhov_length_full`.
--> Calculating `wind_speed_soil_inst`.
--> Calculating `psi_m`.
--> Calculating `aerodynamical_resistance_forced_convection_soil`.
--> Calculating `aerodynamical_resistance_forced_convection_bare`.
--> Calculating `psi_m`.
--> Calculating `psi_h`.
--> Calculating `aerodynamical_resistance_forced_convection_full`.
--> Calculating `psi_m`.
--> Calculating `psi_m`.
--> Calculating `psi_h`.
--> Calculating `psi_h`.
--> Calculating `maximum_temperature_bare`.
--> Calculating `maximum_temperature_full`.
--> Calculating `wet_bulb_temperature_inst_new`.
--> Calculating `maximum_temperature`.
--> Calculating `wet_bulb_temperature_kelvin_inst`.
--> Calculating `air_temperature_kelvin`.
--> Calculating `minimum_temperature`.
--> Calculating `soil_moisture_from_maximum_temperature`.
/Users/hmcoerver/miniconda3/envs/pywapor/lib/python3.10/site-packages/dask/core.py:119: RuntimeWarning: invalid value encountered in power
return func(*(_execute_task(a, cache) for a in args))
/Users/hmcoerver/miniconda3/envs/pywapor/lib/python3.10/site-packages/dask/core.py:119: RuntimeWarning: invalid value encountered in log
return func(*(_execute_task(a, cache) for a in args))
--> Saving output to `se_root_out.nc`.
> peak-memory-usage: 6.7GB, execution-time: 0:00:26.114548.
> chunksize|dimsize: [time: 1|32, y: 808|808, x: 1000|1009]
< SE_ROOT (0:00:44.203411)
> timesize: 32 [2021-07-03T12:48, ..., 2021-07-13T10:11]
--> Compositing 26 variables.
--> (1/26) Compositing `ndvi` (mean).
--> Saving `ndvi` composites.
> peak-memory-usage: 28.0MB, execution-time: 0:00:12.093993.
> chunksize|dimsize: [time_bins: 1|1, y: 808|808, x: 1009|1009]
--> (2/26) Compositing `r0` (mean).
--> Saving `r0` composites.
> peak-memory-usage: 28.0MB, execution-time: 0:00:12.090592.
> chunksize|dimsize: [time_bins: 1|1, y: 808|808, x: 1009|1009]
--> (3/26) Compositing `se_root` (max).
--> Saving `se_root` composites.
> peak-memory-usage: 438.4MB, execution-time: 0:00:02.044091.
> chunksize|dimsize: [time_bins: 1|1, y: 808|808, x: 1009|1009]
--> (4/26) Compositing `p` (mean).
--> Saving `p` composites.
> peak-memory-usage: 158.9KB, execution-time: 0:00:02.055707.
> chunksize|dimsize: [time_bins: 1|1, y: 18|18, x: 22|22]
--> (5/26) Compositing `z` (None).
--> Saving `z` composites.
> peak-memory-usage: 317.0MB, execution-time: 0:00:02.247752.
> chunksize|dimsize: [y: 2883|2883, x: 3603|3603]
--> (6/26) Compositing `ra` (mean).
--> Saving `ra` composites.
> peak-memory-usage: 127.2KB, execution-time: 0:00:02.051137.
> chunksize|dimsize: [time_bins: 1|1, y: 4|4, x: 3|3]
--> (7/26) Compositing `t_air` (mean).
--> Saving `t_air` composites.
> peak-memory-usage: 80.7KB, execution-time: 0:00:02.045642.
> chunksize|dimsize: [time_bins: 1|1, y: 5|5, x: 5|5]
--> (8/26) Compositing `t_air_max` (max).
--> Saving `t_air_max` composites.
> peak-memory-usage: 79.3KB, execution-time: 0:00:02.036475.
> chunksize|dimsize: [time_bins: 1|1, y: 5|5, x: 5|5]
--> (9/26) Compositing `t_air_min` (min).
--> Saving `t_air_min` composites.
> peak-memory-usage: 80.8KB, execution-time: 0:00:02.040938.
> chunksize|dimsize: [time_bins: 1|1, y: 5|5, x: 5|5]
--> (10/26) Compositing `u2m` (mean).
--> Saving `u2m` composites.
> peak-memory-usage: 83.0KB, execution-time: 0:00:02.053375.
> chunksize|dimsize: [time_bins: 1|1, y: 5|5, x: 5|5]
--> (11/26) Compositing `v2m` (mean).
--> Saving `v2m` composites.
> peak-memory-usage: 79.4KB, execution-time: 0:00:02.052054.
> chunksize|dimsize: [time_bins: 1|1, y: 5|5, x: 5|5]
--> (12/26) Compositing `qv` (mean).
--> Saving `qv` composites.
> peak-memory-usage: 80.6KB, execution-time: 0:00:02.055128.
> chunksize|dimsize: [time_bins: 1|1, y: 5|5, x: 5|5]
--> (13/26) Compositing `p_air` (mean).
--> Saving `p_air` composites.
> peak-memory-usage: 80.7KB, execution-time: 0:00:02.049269.
> chunksize|dimsize: [time_bins: 1|1, y: 5|5, x: 5|5]
--> (14/26) Compositing `p_air_0` (mean).
--> Saving `p_air_0` composites.
> peak-memory-usage: 80.5KB, execution-time: 0:00:02.057158.
> chunksize|dimsize: [time_bins: 1|1, y: 5|5, x: 5|5]
--> (15/26) Compositing `wv` (mean).
--> Saving `wv` composites.
> peak-memory-usage: 81.6KB, execution-time: 0:00:02.056278.
> chunksize|dimsize: [time_bins: 1|1, y: 5|5, x: 5|5]
--> (16/26) Compositing `land_mask` (None).
--> Saving `land_mask` composites.
> peak-memory-usage: 10.9MB, execution-time: 0:00:02.043257.
> chunksize|dimsize: [y: 295|295, x: 367|367]
--> (17/26) Compositing `rs_min` (None).
--> Saving `rs_min` composites.
> peak-memory-usage: 10.9MB, execution-time: 0:00:02.051270.
> chunksize|dimsize: [y: 295|295, x: 367|367]
--> (18/26) Compositing `z_obst_max` (None).
--> Saving `z_obst_max` composites.
> peak-memory-usage: 10.9MB, execution-time: 0:00:02.056363.
> chunksize|dimsize: [y: 295|295, x: 367|367]
--> (19/26) Compositing `lw_offset` (None).
--> Saving `lw_offset` composites.
> peak-memory-usage: 20.1MB, execution-time: 0:00:02.051517.
> chunksize|dimsize: [y: 403|403, x: 493|493]
--> (20/26) Compositing `lw_slope` (None).
--> Saving `lw_slope` composites.
> peak-memory-usage: 20.1MB, execution-time: 0:00:02.043761.
> chunksize|dimsize: [y: 403|403, x: 493|493]
--> (21/26) Compositing `z_oro` (None).
--> Saving `z_oro` composites.
> peak-memory-usage: 20.1MB, execution-time: 0:00:02.043064.
> chunksize|dimsize: [y: 403|403, x: 493|493]
--> (22/26) Compositing `rn_offset` (None).
--> Saving `rn_offset` composites.
> peak-memory-usage: 20.1MB, execution-time: 0:00:02.044100.
> chunksize|dimsize: [y: 403|403, x: 493|493]
--> (23/26) Compositing `rn_slope` (None).
--> Saving `rn_slope` composites.
> peak-memory-usage: 20.1MB, execution-time: 0:00:02.030701.
> chunksize|dimsize: [y: 403|403, x: 493|493]
--> (24/26) Compositing `t_amp_year` (None).
--> Saving `t_amp_year` composites.
> peak-memory-usage: 20.1MB, execution-time: 0:00:02.032750.
> chunksize|dimsize: [y: 403|403, x: 493|493]
--> (25/26) Compositing `t_opt` (None).
--> Saving `t_opt` composites.
> peak-memory-usage: 20.1MB, execution-time: 0:00:02.037533.
> chunksize|dimsize: [y: 403|403, x: 493|493]
--> (26/26) Compositing `vpd_slope` (None).
--> Saving `vpd_slope` composites.
> peak-memory-usage: 20.1MB, execution-time: 0:00:02.042518.
> chunksize|dimsize: [y: 403|403, x: 493|493]
--> Using `S5_TOC_100_m_C1.nc` as reprojecting example.
> shape: (808, 1009), res: 0.0010° x 0.0010°.
--> Selected `reproject_chunk` for reprojection of p_bin.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.4KB, execution-time: 0:00:00.051346.
--> Saving reprojected data from p_bin.nc:p (bilinear).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.081814.
> chunksize|dimsize: [time_bins: 1|1, y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of z_bin.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.4KB, execution-time: 0:00:00.279616.
--> Saving reprojected data from z_bin.nc:z (bilinear).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.120462.
> chunksize|dimsize: [y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of ra_bin.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.6KB, execution-time: 0:00:00.043899.
--> Saving reprojected data from ra_bin.nc:ra (bilinear).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.116231.
> chunksize|dimsize: [time_bins: 1|1, y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of t_air_bin.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.4KB, execution-time: 0:00:00.044204.
--> Saving reprojected data from t_air_bin.nc:t_air (bilinear).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.110589.
> chunksize|dimsize: [time_bins: 1|1, y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of t_air_max_bin.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.4KB, execution-time: 0:00:00.042731.
--> Saving reprojected data from t_air_max_bin.nc:t_air_max (bilinear).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.114914.
> chunksize|dimsize: [time_bins: 1|1, y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of t_air_min_bin.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 5.2KB, execution-time: 0:00:00.047023.
--> Saving reprojected data from t_air_min_bin.nc:t_air_min (bilinear).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.107576.
> chunksize|dimsize: [time_bins: 1|1, y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of u2m_bin.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.4KB, execution-time: 0:00:00.042843.
--> Saving reprojected data from u2m_bin.nc:u2m (bilinear).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.102620.
> chunksize|dimsize: [time_bins: 1|1, y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of v2m_bin.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.6KB, execution-time: 0:00:00.044675.
--> Saving reprojected data from v2m_bin.nc:v2m (bilinear).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.105844.
> chunksize|dimsize: [time_bins: 1|1, y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of qv_bin.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.4KB, execution-time: 0:00:00.045269.
--> Saving reprojected data from qv_bin.nc:qv (bilinear).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.081415.
> chunksize|dimsize: [time_bins: 1|1, y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of p_air_bin.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.6KB, execution-time: 0:00:00.047754.
--> Saving reprojected data from p_air_bin.nc:p_air (bilinear).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.080110.
> chunksize|dimsize: [time_bins: 1|1, y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of p_air_0_bin.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.6KB, execution-time: 0:00:00.045503.
--> Saving reprojected data from p_air_0_bin.nc:p_air_0 (bilinear).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.072551.
> chunksize|dimsize: [time_bins: 1|1, y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of wv_bin.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.4KB, execution-time: 0:00:00.048121.
--> Saving reprojected data from wv_bin.nc:wv (bilinear).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.079827.
> chunksize|dimsize: [time_bins: 1|1, y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of land_mask_bin.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.5KB, execution-time: 0:00:00.032520.
--> Saving reprojected data from land_mask_bin.nc:land_mask (nearest).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.053094.
> chunksize|dimsize: [y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of rs_min_bin.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.5KB, execution-time: 0:00:00.033517.
--> Saving reprojected data from rs_min_bin.nc:rs_min (nearest).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.081456.
> chunksize|dimsize: [y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of z_obst_max_bin.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.5KB, execution-time: 0:00:00.032787.
--> Saving reprojected data from z_obst_max_bin.nc:z_obst_max (nearest).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.079130.
> chunksize|dimsize: [y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of lw_offset_bin.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.5KB, execution-time: 0:00:00.049142.
--> Saving reprojected data from lw_offset_bin.nc:lw_offset (bilinear).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.057155.
> chunksize|dimsize: [y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of lw_slope_bin.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.5KB, execution-time: 0:00:00.049371.
--> Saving reprojected data from lw_slope_bin.nc:lw_slope (bilinear).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.095700.
> chunksize|dimsize: [y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of z_oro_bin.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.6KB, execution-time: 0:00:00.049022.
--> Saving reprojected data from z_oro_bin.nc:z_oro (bilinear).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.077587.
> chunksize|dimsize: [y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of rn_offset_bin.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.6KB, execution-time: 0:00:00.047129.
--> Saving reprojected data from rn_offset_bin.nc:rn_offset (bilinear).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.062078.
> chunksize|dimsize: [y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of rn_slope_bin.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.5KB, execution-time: 0:00:00.047261.
--> Saving reprojected data from rn_slope_bin.nc:rn_slope (bilinear).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.080825.
> chunksize|dimsize: [y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of t_amp_year_bin.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.6KB, execution-time: 0:00:00.050371.
--> Saving reprojected data from t_amp_year_bin.nc:t_amp_year (bilinear).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.064823.
> chunksize|dimsize: [y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of t_opt_bin.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.5KB, execution-time: 0:00:00.050235.
--> Saving reprojected data from t_opt_bin.nc:t_opt (bilinear).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.080937.
> chunksize|dimsize: [y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of vpd_slope_bin.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.5KB, execution-time: 0:00:00.050914.
--> Saving reprojected data from vpd_slope_bin.nc:vpd_slope (bilinear).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.049308.
> chunksize|dimsize: [y: 808|808, x: 1009|1009]
--> Calculating local means (r = 0.25°) of `z`.
--> Filling 3 missing pixels in 'z'.
--> Applying 'lapse_rate'.
--> Applying 'rename_vars'.
--> Applying 'fill_attrs'.
--> Applying 'calc_doys'.
--> Applying 'remove_empty_statics'.
--> Applying 'add_constants_new'.
--> Creating merged file `et_look_in.nc`.
> peak-memory-usage: 80.6MB, execution-time: 0:00:03.086375.
> chunksize|dimsize: [time_bins: 1|1, y: 808|808, x: 1009|1009]
< PRE_ET_LOOK (0:20:26.345738)
Again, we can inspect the xarray.Dataset like this.
[7]:
ds
[7]:
<xarray.Dataset>
Dimensions: (x: 1009, y: 808, time_bins: 1)
Coordinates:
* x (x) float64 30.2 30.2 30.2 30.2 ... 31.2 31.2 31.2 31.2
* y (y) float64 29.7 29.7 29.7 29.7 ... 28.9 28.9 28.9 28.9
spatial_ref int64 ...
* time_bins (time_bins) datetime64[ns] 2021-07-01
Data variables: (12/78)
ndvi (time_bins, y, x) float64 dask.array<chunksize=(1, 808, 1009), meta=np.ndarray>
r0 (time_bins, y, x) float64 dask.array<chunksize=(1, 808, 1009), meta=np.ndarray>
se_root (time_bins, y, x) float64 dask.array<chunksize=(1, 808, 1009), meta=np.ndarray>
p_24 (time_bins, y, x) float64 dask.array<chunksize=(1, 808, 1009), meta=np.ndarray>
z (y, x) float64 dask.array<chunksize=(808, 1009), meta=np.ndarray>
ra_24 (time_bins, y, x) float64 dask.array<chunksize=(1, 808, 1009), meta=np.ndarray>
... ...
o2 float64 ...
co2_ref int64 ...
gcgdm float64 ...
phot_eff float64 ...
r0_bare float64 ...
r0_full float64 ...And we can see the pixel size (in degrees) like this:
[8]:
res = ds.rio.resolution()
print(f"pixel-size: {res[0]:.4f}°")
pixel-size: 0.0010°
Next, we can run ETLook using the level_2 data just like we did for level_1.
[9]:
ds_out = pywapor.et_look.main(ds)
> ET_LOOK
--> Running `et_look` (v2).
--> Calculating `declination`.
--> Calculating `inverse_earth_sun_distance`.
--> Calculating `vegetation_cover`.
--> Calculating `leaf_area_index`.
--> Calculating `effective_leaf_area_index`.
--> Calculating `soil_fraction`.
--> Calculating `latitude_rad`.
--> Calculating `sunset_hour_angle`.
--> Calculating `daily_solar_radiation_toa_flat`.
--> Calculating `transmissivity`.
--> Calculating `stress_radiation`.
--> Calculating `air_pressure_kpa2mbar`.
--> Calculating `air_pressure_daily`.
--> Calculating `air_pressure`.
--> Calculating `vapour_pressure_from_specific_humidity_daily`.
--> Calculating `vapour_pressure_from_specific_humidity`.
--> Calculating `saturated_vapour_pressure`.
--> Calculating `vapour_pressure_deficit_daily`.
--> Calculating `vapour_pressure_deficit`.
--> Calculating `stress_vpd`.
--> Calculating `stress_temperature`.
--> Calculating `atmospheric_canopy_resistance`.
--> Calculating `air_temperature_kelvin_daily`.
--> Calculating `air_temperature_kelvin`.
--> Calculating `longwave_radiation_fao`.
--> Calculating `interception_mm`.
--> Calculating `latent_heat_daily`.
--> Calculating `latent_heat`.
--> Calculating `interception_wm2`.
--> Calculating `net_radiation`.
--> Calculating `net_radiation_canopy`.
--> Calculating `stress_moisture`.
--> Calculating `canopy_resistance`.
--> Calculating `obstacle_height`.
--> Calculating `roughness_length`.
--> Calculating `wind_speed`.
--> Calculating `initial_canopy_aerodynamic_resistance`.
--> Calculating `wind_speed_blending_height_daily`.
--> Calculating `wind_speed_blending_height`.
--> Calculating `dry_air_density_daily`.
--> Calculating `dry_air_density`.
--> Calculating `moist_air_density_daily`.
--> Calculating `moist_air_density`.
--> Calculating `air_density_daily`.
--> Calculating `air_density`.
--> Calculating `psychrometric_constant_daily`.
--> Calculating `psychrometric_constant`.
--> Calculating `slope_saturated_vapour_pressure_daily`.
--> Calculating `slope_saturated_vapour_pressure`.
--> Calculating `saturated_vapour_pressure`.
--> Calculating `initial_daily_transpiration`.
--> Calculating `initial_sensible_heat_flux_canopy_daily`.
--> Calculating `displacement_height`.
--> Calculating `initial_friction_velocity_daily`.
--> Calculating `transpiration`.
--> Calculating `transpiration_mm`.
--> Calculating `net_radiation_soil`.
--> Calculating `soil_resistance`.
--> Calculating `initial_soil_aerodynamic_resistance`.
--> Calculating `initial_friction_velocity_soil_daily`.
--> Calculating `soil_thermal_conductivity`.
--> Calculating `volumetric_heat_capacity`.
--> Calculating `damping_depth`.
--> Calculating `bare_soil_heat_flux`.
--> Calculating `soil_heat_flux`.
--> Calculating `initial_daily_evaporation`.
--> Calculating `initial_sensible_heat_flux_soil_daily`.
--> Calculating `evaporation`.
--> Calculating `evaporation_mm`.
--> Calculating `et_actual_mm`.
--> Calculating `net_radiation_grass`.
--> Calculating `et_reference`.
--> Calculating `et_reference_mm`.
--> Calculating `air_temperature_kelvin_daily`.
--> Calculating `air_temperature_kelvin`.
--> Calculating `air_temperature_kelvin_daily`.
--> Calculating `air_temperature_kelvin`.
--> Calculating `mean_temperature_kelvin_daytime`.
--> Calculating `temperature_dependency`.
--> Calculating `affinity_constant_co2`.
--> Calculating `inhibition_constant_o2`.
--> Calculating `co2_o2_specificity_ratio`.
--> Calculating `co2_level_annual`.
--> Calculating `autotrophic_respiration`.
--> Calculating `par`.
--> Calculating `fpar`.
--> Calculating `co2_fertilisation`.
--> Calculating `net_primary_production_max`.
--> Calculating `net_primary_production`.
--> Saving output to `et_look_out.nc`.
> peak-memory-usage: 1.8GB, execution-time: 0:00:03.020583.
> chunksize|dimsize: [time_bins: 1|1, y: 808|808, x: 1000|1009]
< ET_LOOK (0:00:10.921504)
We can calculate some statistic, like we did before.
[10]:
import numpy as np
et_data = ds_out.et_24_mm.isel(time_bins = 0).values
print("resolution:", et_data.shape)
print("total pixels:", et_data.size)
print("number of pixels with missing data:", np.sum(np.isnan(et_data)))
print("maximum value: {0:.2f}".format(np.nanmax(et_data)))
print("minimum value: {0:.2f}".format(np.nanmin(et_data)))
print("mean: {0:.2f}".format(np.nanmean(et_data)))
print("median: {0:.2f}".format(np.nanmedian(et_data)))
resolution: (808, 1009)
total pixels: 815272
number of pixels with missing data: 109479
maximum value: 10.46
minimum value: 0.00
mean: 2.06
median: 0.11
Notice that the amount of pixels with missing data is now higher. Which we can also see on a map of the higher resolution daily ET.
[11]:
ds_out.et_24_mm.isel(time_bins = 0).plot()
[11]:
<matplotlib.collections.QuadMesh at 0x162203f70>
Custom inputs
Besides using the predefined levels, it is also possible to create your own level or customize an existing one. Instead of passing a string to the sources input parameter of pywapor.pre_et_look.main, we can also pass a dictionary that specifies which products we would like to use. Let’s start with the basic level_1 configuration and make some modifications.
[12]:
level = "level_1"
et_look_sources = pywapor.general.levels.pre_et_look_levels(level)
et_look_sources["ndvi"]["products"] = [
{'source': 'MODIS',
'product_name': 'MOD13Q1.061',
'enhancers': 'default'},
{'source': 'MODIS',
'product_name': 'MYD13Q1.061',
'enhancers': 'default'},
{'source': 'PROBAV',
'product_name': 'S5_TOC_100_m_C1',
'enhancers': 'default',
'is_example': True}
]
et_look_sources["r0"]["products"] = [
{'source': 'MODIS',
'product_name': 'MCD43A3.061',
'enhancers': 'default'},
{'source': 'PROBAV',
'product_name': 'S5_TOC_100_m_C1',
'enhancers': 'default'}
]
With these changes, ndvi and r0 will be calculated based on three products, a combination of PROBAV and MODIS. At the resolution of PROBAV. These changes will only apply to pre_et_look however. To also adjust the inputs for pre_se_root we need to adjust those as well.
[13]:
se_root_sources = pywapor.general.levels.pre_se_root_levels(level)
se_root_sources["ndvi"]["products"] = et_look_sources["ndvi"]["products"]
Then finally, we pass these adjusted se_root_sources to et_look_sources like this.
[14]:
from functools import partial
et_look_sources["se_root"]["products"] = [
{'source': partial(pywapor.se_root.se_root, sources = se_root_sources),
'product_name': 'v2',
'enhancers': 'default'},
]
And we run pre_et_look with our custom sources.
[15]:
ds = pywapor.pre_et_look.main(project_folder, latlim, lonlim, timelim,
sources = et_look_sources,
bin_length = composite_length)
> PRE_ET_LOOK
--> Collecting `ndvi` from `MODIS.MOD13Q1.061`.
--> Applying 'mask_qa' to `ndvi`.
--> Saving merged data.
> peak-memory-usage: 4.7MB, execution-time: 0:00:02.084287.
> chunksize|dimsize: [time: 2|2, y: 403|403, x: 506|506]
> timesize: 2 [2021-07-04T00:00, ..., 2021-07-20T00:00]
--> Collecting `ndvi` from `MODIS.MYD13Q1.061`.
--> Applying 'mask_qa' to `ndvi`.
--> Saving merged data.
> peak-memory-usage: 2.3MB, execution-time: 0:00:02.069657.
> chunksize|dimsize: [time: 1|1, y: 403|403, x: 506|506]
> timesize: 1 [2021-07-12T00:00, ..., 2021-07-12T00:00]
--> Collecting `ndvi`, `r0` from `PROBAV.S5_TOC_100_m_C1`.
> timesize: 3 [2021-07-03T12:00, ..., 2021-07-13T12:00]
--> Collecting `r0` from `MODIS.MCD43A3.061`.
--> Applying 'shortwave_r0' to `r0`.
--> Applying 'mask_qa' to `r0`.
--> Saving merged data.
> peak-memory-usage: 10.4MB, execution-time: 0:00:02.090815.
> chunksize|dimsize: [time: 17|17, y: 207|207, x: 258|258]
> timesize: 17 [2021-06-28T12:00, ..., 2021-07-14T12:00]
--> Collecting `p` from `CHIRPS.P05`.
> timesize: 17 [2021-06-28T00:30, ..., 2021-07-14T00:30]
--> Collecting `z` from `SRTM.30M`.
> timesize: 1 [2000-02-11T00:00, ..., 2000-02-11T00:00]
--> Collecting `ra` from `MERRA2.M2T1NXRAD.5.12.4`.
> timesize: 408 [2021-06-28T01:00, ..., 2021-07-15T00:00]
--> Collecting `t_air`, `t_air_max`, `t_air_min`, `u2m`, `v2m`, `qv`, `p_air`, `p_air_0`, `wv` from `GEOS5.inst3_2d_asm_Nx`.
> timesize: 129 [2021-06-28T01:30, ..., 2021-07-14T01:30]
--> Collecting `land_mask`, `rs_min`, `z_obst_max` from `GLOBCOVER.2009_V2.3_Global`.
--> Collecting `lw_offset`, `lw_slope`, `z_oro`, `rn_offset`, `rn_slope`, `t_amp_year`, `t_opt`, `vpd_slope` from `STATICS.WaPOR2`.
--> Collecting `se_root` from `se_root.v2`.
--> Using se_root v2.
> PRE_SE_ROOT
--> Collecting `lst` from `MODIS.MOD11A1.061`.
> timesize: 22 [2021-06-28T10:54, ..., 2021-07-14T10:54]
--> Collecting `lst` from `MODIS.MYD11A1.061`.
> timesize: 32 [2021-06-28T12:30, ..., 2021-07-14T12:36]
--> Collecting `ndvi` from `MODIS.MOD13Q1.061`.
> timesize: 2 [2021-07-04T00:00, ..., 2021-07-20T00:00]
--> Collecting `ndvi` from `MODIS.MYD13Q1.061`.
> timesize: 1 [2021-07-12T00:00, ..., 2021-07-12T00:00]
--> Collecting `ndvi` from `PROBAV.S5_TOC_100_m_C1`.
> timesize: 3 [2021-07-03T12:00, ..., 2021-07-13T12:00]
--> Collecting `t_air`, `t_air_max`, `t_air_min`, `u2m`, `v2m`, `qv`, `wv`, `p_air`, `p_air_0` from `GEOS5.inst3_2d_asm_Nx`.
> timesize: 129 [2021-06-28T01:30, ..., 2021-07-14T01:30]
--> Collecting `r0_bare`, `r0_full` from `STATICS.WaPOR2`.
--> Spatially aligning 2 `lst` products together.
--> Using `MYD11A1.061.nc` as reprojecting example.
> shape: (104, 130), res: 0.0079° x 0.0079°.
--> Spatially aligning 3 `ndvi` products together.
--> Using `S5_TOC_100_m_C1.nc` as reprojecting example.
> shape: (808, 1009), res: 0.0010° x 0.0010°.
--> Selected `reproject_chunk` for reprojection of MOD13Q1.061.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.5KB, execution-time: 0:00:00.049487.
--> Saving reprojected data from MOD13Q1.061.nc:ndvi (nearest).
> peak-memory-usage: 62.2MB, execution-time: 0:00:02.119448.
> chunksize|dimsize: [time: 2|2, y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of MYD13Q1.061.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.6KB, execution-time: 0:00:00.031432.
--> Saving reprojected data from MYD13Q1.061.nc:ndvi (nearest).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.096849.
> chunksize|dimsize: [time: 1|1, y: 808|808, x: 1009|1009]
--> Aligning times in `ndvi` (6) with `lst` (54, linear).
--> Appending data to an existing file.
> peak-memory-usage: 1.2GB, execution-time: 0:00:28.666251.
> chunksize|dimsize: [time: 54|54, y: 500|808, x: 500|1009]
> shape: (54, 808, 1009), kwargs: [].
--> Aligning times in `p_air` (129) with `lst` (54, linear).
--> Appending data to an existing file.
> peak-memory-usage: 731.4KB, execution-time: 0:00:02.044138.
> chunksize|dimsize: [time: 54|54, y: 5|5, x: 5|5]
> shape: (54, 5, 5), kwargs: [].
--> Aligning times in `p_air_0` (129) with `lst` (54, linear).
--> Appending data to an existing file.
> peak-memory-usage: 723.9KB, execution-time: 0:00:02.065197.
> chunksize|dimsize: [time: 54|54, y: 5|5, x: 5|5]
> shape: (54, 5, 5), kwargs: [].
--> Aligning times in `qv` (129) with `lst` (54, linear).
--> Appending data to an existing file.
> peak-memory-usage: 724.1KB, execution-time: 0:00:02.072961.
> chunksize|dimsize: [time: 54|54, y: 5|5, x: 5|5]
> shape: (54, 5, 5), kwargs: [].
--> Aligning times in `t_air` (129) with `lst` (54, linear).
--> Appending data to an existing file.
> peak-memory-usage: 724.8KB, execution-time: 0:00:02.060276.
> chunksize|dimsize: [time: 54|54, y: 5|5, x: 5|5]
> shape: (54, 5, 5), kwargs: [].
--> Aligning times in `t_air_max` (129) with `lst` (54, linear).
--> Appending data to an existing file.
> peak-memory-usage: 723.5KB, execution-time: 0:00:02.056172.
> chunksize|dimsize: [time: 54|54, y: 5|5, x: 5|5]
> shape: (54, 5, 5), kwargs: [].
--> Aligning times in `t_air_min` (129) with `lst` (54, linear).
--> Appending data to an existing file.
> peak-memory-usage: 134.4KB, execution-time: 0:00:02.050902.
> chunksize|dimsize: [time: 54|54, y: 5|5, x: 5|5]
> shape: (54, 5, 5), kwargs: [].
--> Aligning times in `u2m` (129) with `lst` (54, linear).
--> Appending data to an existing file.
> peak-memory-usage: 600.9KB, execution-time: 0:00:02.048833.
> chunksize|dimsize: [time: 54|54, y: 5|5, x: 5|5]
> shape: (54, 5, 5), kwargs: [].
--> Aligning times in `v2m` (129) with `lst` (54, linear).
--> Appending data to an existing file.
> peak-memory-usage: 724.7KB, execution-time: 0:00:02.061449.
> chunksize|dimsize: [time: 54|54, y: 5|5, x: 5|5]
> shape: (54, 5, 5), kwargs: [].
--> Aligning times in `wv` (129) with `lst` (54, linear).
--> Appending data to an existing file.
> peak-memory-usage: 724.6KB, execution-time: 0:00:02.066654.
> chunksize|dimsize: [time: 54|54, y: 5|5, x: 5|5]
> shape: (54, 5, 5), kwargs: [].
--> Using `S5_TOC_100_m_C1.nc` as reprojecting example.
> shape: (808, 1009), res: 0.0010° x 0.0010°.
--> Selected `reproject_chunk` for reprojection of MOD11A1.061.nc.
--> Correcting src_ds.
> peak-memory-usage: 27.9MB, execution-time: 0:00:02.103804.
> chunksize|dimsize: [time: 54|54, y: 104|104, x: 130|130]
--> Warping VRT to netCDF.
> peak-memory-usage: 6.5KB, execution-time: 0:00:01.194365.
--> Saving reprojected data from MOD11A1.061_fixed.nc:lst (nearest).
> peak-memory-usage: 1.2GB, execution-time: 0:00:02.451172.
> chunksize|dimsize: [time: 39|54, y: 586|808, x: 732|1009]
--> Selected `reproject_chunk` for reprojection of p_air_i.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 6.3KB, execution-time: 0:00:01.981864.
--> Saving reprojected data from p_air_i.nc:p_air (bilinear).
> peak-memory-usage: 1.3GB, execution-time: 0:00:02.568315.
> chunksize|dimsize: [time: 39|54, y: 586|808, x: 732|1009]
--> Selected `reproject_chunk` for reprojection of p_air_0_i.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 6.7KB, execution-time: 0:00:01.979428.
--> Saving reprojected data from p_air_0_i.nc:p_air_0 (bilinear).
> peak-memory-usage: 1.1GB, execution-time: 0:00:02.452513.
> chunksize|dimsize: [time: 39|54, y: 586|808, x: 732|1009]
--> Selected `reproject_chunk` for reprojection of qv_i.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 6.3KB, execution-time: 0:00:01.986774.
--> Saving reprojected data from qv_i.nc:qv (bilinear).
> peak-memory-usage: 1.2GB, execution-time: 0:00:02.432647.
> chunksize|dimsize: [time: 39|54, y: 586|808, x: 732|1009]
--> Selected `reproject_chunk` for reprojection of WaPOR2.nc.
--> Correcting src_ds.
> peak-memory-usage: 6.1MB, execution-time: 0:00:02.073174.
> chunksize|dimsize: [y: 403|403, x: 493|493]
--> Warping VRT to netCDF.
> peak-memory-usage: 4.5KB, execution-time: 0:00:00.054495.
--> Saving reprojected data from WaPOR2_fixed.nc:r0_bare (bilinear).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.101392.
> chunksize|dimsize: [y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of WaPOR2.nc.
--> Correcting src_ds.
> peak-memory-usage: 6.1MB, execution-time: 0:00:02.073487.
> chunksize|dimsize: [y: 403|403, x: 493|493]
--> Warping VRT to netCDF.
> peak-memory-usage: 4.5KB, execution-time: 0:00:00.053565.
--> Saving reprojected data from WaPOR2_fixed.nc:r0_full (bilinear).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.107321.
> chunksize|dimsize: [y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of t_air_i.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 6.3KB, execution-time: 0:00:01.993387.
--> Saving reprojected data from t_air_i.nc:t_air (bilinear).
> peak-memory-usage: 1.3GB, execution-time: 0:00:02.646262.
> chunksize|dimsize: [time: 39|54, y: 586|808, x: 732|1009]
--> Selected `reproject_chunk` for reprojection of t_air_max_i.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 6.3KB, execution-time: 0:00:02.004193.
--> Saving reprojected data from t_air_max_i.nc:t_air_max (bilinear).
> peak-memory-usage: 1.2GB, execution-time: 0:00:02.645453.
> chunksize|dimsize: [time: 39|54, y: 586|808, x: 732|1009]
--> Selected `reproject_chunk` for reprojection of t_air_min_i.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 6.3KB, execution-time: 0:00:01.997677.
--> Saving reprojected data from t_air_min_i.nc:t_air_min (bilinear).
> peak-memory-usage: 1.2GB, execution-time: 0:00:02.651693.
> chunksize|dimsize: [time: 39|54, y: 586|808, x: 732|1009]
--> Selected `reproject_chunk` for reprojection of u2m_i.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 6.3KB, execution-time: 0:00:01.974481.
--> Saving reprojected data from u2m_i.nc:u2m (bilinear).
> peak-memory-usage: 1.3GB, execution-time: 0:00:02.531306.
> chunksize|dimsize: [time: 39|54, y: 586|808, x: 732|1009]
--> Selected `reproject_chunk` for reprojection of v2m_i.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 6.3KB, execution-time: 0:00:01.984028.
--> Saving reprojected data from v2m_i.nc:v2m (bilinear).
> peak-memory-usage: 1.2GB, execution-time: 0:00:02.582560.
> chunksize|dimsize: [time: 39|54, y: 586|808, x: 732|1009]
--> Selected `reproject_chunk` for reprojection of wv_i.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 6.3KB, execution-time: 0:00:02.022635.
--> Saving reprojected data from wv_i.nc:wv (bilinear).
> peak-memory-usage: 1.2GB, execution-time: 0:00:02.556210.
> chunksize|dimsize: [time: 39|54, y: 586|808, x: 732|1009]
--> Applying 'rename_meteo'.
--> Applying 'remove_empty_statics'.
--> Applying 'add_constants_new'.
--> Applying 'bt_to_lst'.
--> Applying 'drop_empty_times'.
--> Creating merged file `se_root_in_.nc`.
> peak-memory-usage: 463.0MB, execution-time: 0:00:58.274699.
> chunksize|dimsize: [time: 39|54, y: 586|808, x: 732|1009]
< PRE_SE_ROOT (0:02:48.085266)
> SE_ROOT
--> Running `se_root` (v2).
--> Calculating `seasonal_correction`.
--> Calculating `declination`.
--> Calculating `day_angle`.
--> Calculating `air_pressure_kpa2mbar`.
--> Calculating `air_pressure_kpa2mbar`.
--> Calculating `vegetation_cover`.
--> Calculating `air_temperature_kelvin_inst`.
--> Calculating `air_temperature_kelvin`.
--> Calculating `vapour_pressure_from_specific_humidity_inst`.
--> Calculating `vapour_pressure_from_specific_humidity`.
--> Calculating `specific_humidity_from_vapour_pressure`.
--> Calculating `moist_air_density_inst`.
--> Calculating `moist_air_density`.
--> Calculating `dry_air_density_inst`.
--> Calculating `dry_air_density`.
--> Calculating `air_density_inst`.
--> Calculating `air_density`.
--> Calculating `wind_speed_blending_height_bare`.
--> Calculating `longitude_rad`.
--> Calculating `latitude_rad`.
--> Calculating `hour_angle`.
--> Calculating `inverse_earth_sun_distance`.
--> Calculating `solar_elevation_angle`.
--> Calculating `solar_elevation_angle_refracted`.
--> Calculating `relative_optical_airmass`.
--> Calculating `rayleigh_optical_thickness`.
--> Calculating `linke_turbidity`.
--> Calculating `extraterrestrial_irradiance_normal`.
--> Calculating `beam_irradiance_normal_clear`.
--> Calculating `beam_irradiance_horizontal_clear`.
--> Calculating `diffuse_irradiance_horizontal_clear`.
--> Calculating `ra_clear_horizontal`.
--> Calculating `atmospheric_emissivity_inst`.
--> Calculating `net_radiation_bare`.
--> Calculating `net_radiation_full`.
--> Calculating `sensible_heat_flux_bare`.
--> Calculating `sensible_heat_flux_full`.
--> Calculating `wind_speed_blending_height_full_inst`.
--> Calculating `friction_velocity_bare_inst`.
--> Calculating `friction_velocity_full_inst`.
--> Calculating `monin_obukhov_length_bare`.
--> Calculating `monin_obukhov_length_full`.
--> Calculating `wind_speed_soil_inst`.
--> Calculating `psi_m`.
--> Calculating `aerodynamical_resistance_forced_convection_soil`.
--> Calculating `aerodynamical_resistance_forced_convection_bare`.
--> Calculating `psi_m`.
--> Calculating `psi_h`.
--> Calculating `aerodynamical_resistance_forced_convection_full`.
--> Calculating `psi_m`.
--> Calculating `psi_m`.
--> Calculating `psi_h`.
--> Calculating `psi_h`.
--> Calculating `maximum_temperature_bare`.
--> Calculating `maximum_temperature_full`.
--> Calculating `wet_bulb_temperature_inst_new`.
--> Calculating `maximum_temperature`.
--> Calculating `wet_bulb_temperature_kelvin_inst`.
--> Calculating `air_temperature_kelvin`.
--> Calculating `minimum_temperature`.
--> Calculating `soil_moisture_from_maximum_temperature`.
/Users/hmcoerver/miniconda3/envs/pywapor/lib/python3.10/site-packages/dask/core.py:119: RuntimeWarning: invalid value encountered in power
return func(*(_execute_task(a, cache) for a in args))
/Users/hmcoerver/miniconda3/envs/pywapor/lib/python3.10/site-packages/dask/core.py:119: RuntimeWarning: invalid value encountered in log
return func(*(_execute_task(a, cache) for a in args))
--> Saving output to `se_root_out_.nc`.
> peak-memory-usage: 6.7GB, execution-time: 0:00:26.079548.
> chunksize|dimsize: [time: 1|34, y: 808|808, x: 1000|1009]
< SE_ROOT (0:00:44.168502)
> timesize: 34 [2021-07-03T12:48, ..., 2021-07-13T13:30]
--> Compositing 26 variables.
--> (1/26) Compositing `ndvi` (mean).
--> Using `S5_TOC_100_m_C1.nc` as reprojecting example.
> shape: (808, 1009), res: 0.0010° x 0.0010°.
--> Selected `reproject_chunk` for reprojection of MOD13Q1.061.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.5KB, execution-time: 0:00:00.048948.
--> Saving reprojected data from MOD13Q1.061.nc:ndvi (nearest).
> peak-memory-usage: 62.2MB, execution-time: 0:00:02.137241.
> chunksize|dimsize: [time: 2|2, y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of MYD13Q1.061.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.5KB, execution-time: 0:00:00.032194.
--> Saving reprojected data from MYD13Q1.061.nc:ndvi (nearest).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.114722.
> chunksize|dimsize: [time: 1|1, y: 808|808, x: 1009|1009]
--> Saving `ndvi` composites.
> peak-memory-usage: 112.0MB, execution-time: 0:00:12.082383.
> chunksize|dimsize: [time_bins: 1|1, y: 808|808, x: 1009|1009]
--> (2/26) Compositing `r0` (mean).
--> Using `S5_TOC_100_m_C1.nc` as reprojecting example.
> shape: (808, 1009), res: 0.0010° x 0.0010°.
--> Selected `reproject_chunk` for reprojection of MCD43A3.061.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 5.1KB, execution-time: 0:00:00.625357.
--> Saving reprojected data from MCD43A3.061.nc:r0 (bilinear).
> peak-memory-usage: 528.7MB, execution-time: 0:00:02.677457.
> chunksize|dimsize: [time: 17|17, y: 808|808, x: 1009|1009]
--> Multiple `r0` images for an identical datetime found, reducing those with 'median'.
--> Saving `r0` composites.
> peak-memory-usage: 734.8MB, execution-time: 0:00:12.095157.
> chunksize|dimsize: [time_bins: 1|1, y: 808|808, x: 1009|1009]
--> (3/26) Compositing `se_root` (max).
--> Saving `se_root` composites.
> peak-memory-usage: 448.0MB, execution-time: 0:00:02.052120.
> chunksize|dimsize: [time_bins: 1|1, y: 808|808, x: 1009|1009]
--> (4/26) Compositing `p` (mean).
--> Saving `p` composites.
> peak-memory-usage: 159.6KB, execution-time: 0:00:02.055046.
> chunksize|dimsize: [time_bins: 1|1, y: 18|18, x: 22|22]
--> (5/26) Compositing `z` (None).
--> Saving `z` composites.
> peak-memory-usage: 317.0MB, execution-time: 0:00:02.238127.
> chunksize|dimsize: [y: 2883|2883, x: 3603|3603]
--> (6/26) Compositing `ra` (mean).
--> Saving `ra` composites.
> peak-memory-usage: 128.0KB, execution-time: 0:00:02.044662.
> chunksize|dimsize: [time_bins: 1|1, y: 4|4, x: 3|3]
--> (7/26) Compositing `t_air` (mean).
--> Saving `t_air` composites.
> peak-memory-usage: 79.0KB, execution-time: 0:00:02.048308.
> chunksize|dimsize: [time_bins: 1|1, y: 5|5, x: 5|5]
--> (8/26) Compositing `t_air_max` (max).
--> Saving `t_air_max` composites.
> peak-memory-usage: 81.7KB, execution-time: 0:00:02.048138.
> chunksize|dimsize: [time_bins: 1|1, y: 5|5, x: 5|5]
--> (9/26) Compositing `t_air_min` (min).
--> Saving `t_air_min` composites.
> peak-memory-usage: 80.7KB, execution-time: 0:00:02.048385.
> chunksize|dimsize: [time_bins: 1|1, y: 5|5, x: 5|5]
--> (10/26) Compositing `u2m` (mean).
--> Saving `u2m` composites.
> peak-memory-usage: 81.4KB, execution-time: 0:00:02.047380.
> chunksize|dimsize: [time_bins: 1|1, y: 5|5, x: 5|5]
--> (11/26) Compositing `v2m` (mean).
--> Saving `v2m` composites.
> peak-memory-usage: 80.5KB, execution-time: 0:00:02.050259.
> chunksize|dimsize: [time_bins: 1|1, y: 5|5, x: 5|5]
--> (12/26) Compositing `qv` (mean).
--> Saving `qv` composites.
> peak-memory-usage: 81.3KB, execution-time: 0:00:02.047323.
> chunksize|dimsize: [time_bins: 1|1, y: 5|5, x: 5|5]
--> (13/26) Compositing `p_air` (mean).
--> Saving `p_air` composites.
> peak-memory-usage: 79.6KB, execution-time: 0:00:02.049614.
> chunksize|dimsize: [time_bins: 1|1, y: 5|5, x: 5|5]
--> (14/26) Compositing `p_air_0` (mean).
--> Saving `p_air_0` composites.
> peak-memory-usage: 80.9KB, execution-time: 0:00:02.042857.
> chunksize|dimsize: [time_bins: 1|1, y: 5|5, x: 5|5]
--> (15/26) Compositing `wv` (mean).
--> Saving `wv` composites.
> peak-memory-usage: 81.4KB, execution-time: 0:00:02.051316.
> chunksize|dimsize: [time_bins: 1|1, y: 5|5, x: 5|5]
--> (16/26) Compositing `land_mask` (None).
--> Saving `land_mask` composites.
> peak-memory-usage: 10.9MB, execution-time: 0:00:02.056352.
> chunksize|dimsize: [y: 295|295, x: 367|367]
--> (17/26) Compositing `rs_min` (None).
--> Saving `rs_min` composites.
> peak-memory-usage: 10.9MB, execution-time: 0:00:02.040096.
> chunksize|dimsize: [y: 295|295, x: 367|367]
--> (18/26) Compositing `z_obst_max` (None).
--> Saving `z_obst_max` composites.
> peak-memory-usage: 10.9MB, execution-time: 0:00:02.048044.
> chunksize|dimsize: [y: 295|295, x: 367|367]
--> (19/26) Compositing `lw_offset` (None).
--> Saving `lw_offset` composites.
> peak-memory-usage: 20.1MB, execution-time: 0:00:02.062283.
> chunksize|dimsize: [y: 403|403, x: 493|493]
--> (20/26) Compositing `lw_slope` (None).
--> Saving `lw_slope` composites.
> peak-memory-usage: 20.1MB, execution-time: 0:00:02.052542.
> chunksize|dimsize: [y: 403|403, x: 493|493]
--> (21/26) Compositing `z_oro` (None).
--> Saving `z_oro` composites.
> peak-memory-usage: 20.1MB, execution-time: 0:00:02.048147.
> chunksize|dimsize: [y: 403|403, x: 493|493]
--> (22/26) Compositing `rn_offset` (None).
--> Saving `rn_offset` composites.
> peak-memory-usage: 20.1MB, execution-time: 0:00:02.045485.
> chunksize|dimsize: [y: 403|403, x: 493|493]
--> (23/26) Compositing `rn_slope` (None).
--> Saving `rn_slope` composites.
> peak-memory-usage: 20.1MB, execution-time: 0:00:02.043254.
> chunksize|dimsize: [y: 403|403, x: 493|493]
--> (24/26) Compositing `t_amp_year` (None).
--> Saving `t_amp_year` composites.
> peak-memory-usage: 20.1MB, execution-time: 0:00:02.049808.
> chunksize|dimsize: [y: 403|403, x: 493|493]
--> (25/26) Compositing `t_opt` (None).
--> Saving `t_opt` composites.
> peak-memory-usage: 20.1MB, execution-time: 0:00:02.043548.
> chunksize|dimsize: [y: 403|403, x: 493|493]
--> (26/26) Compositing `vpd_slope` (None).
--> Saving `vpd_slope` composites.
> peak-memory-usage: 20.1MB, execution-time: 0:00:02.047649.
> chunksize|dimsize: [y: 403|403, x: 493|493]
--> Using `S5_TOC_100_m_C1.nc` as reprojecting example.
> shape: (808, 1009), res: 0.0010° x 0.0010°.
--> Selected `reproject_chunk` for reprojection of p_bin.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.4KB, execution-time: 0:00:00.052701.
--> Saving reprojected data from p_bin.nc:p (bilinear).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.077692.
> chunksize|dimsize: [time_bins: 1|1, y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of z_bin.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.6KB, execution-time: 0:00:00.273980.
--> Saving reprojected data from z_bin.nc:z (bilinear).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.116027.
> chunksize|dimsize: [y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of ra_bin.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.6KB, execution-time: 0:00:00.047258.
--> Saving reprojected data from ra_bin.nc:ra (bilinear).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.091451.
> chunksize|dimsize: [time_bins: 1|1, y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of t_air_bin.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.4KB, execution-time: 0:00:00.047576.
--> Saving reprojected data from t_air_bin.nc:t_air (bilinear).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.092476.
> chunksize|dimsize: [time_bins: 1|1, y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of t_air_max_bin.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.4KB, execution-time: 0:00:00.046592.
--> Saving reprojected data from t_air_max_bin.nc:t_air_max (bilinear).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.089461.
> chunksize|dimsize: [time_bins: 1|1, y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of t_air_min_bin.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.4KB, execution-time: 0:00:00.048965.
--> Saving reprojected data from t_air_min_bin.nc:t_air_min (bilinear).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.094541.
> chunksize|dimsize: [time_bins: 1|1, y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of u2m_bin.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.4KB, execution-time: 0:00:00.046106.
--> Saving reprojected data from u2m_bin.nc:u2m (bilinear).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.087018.
> chunksize|dimsize: [time_bins: 1|1, y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of v2m_bin.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.4KB, execution-time: 0:00:00.049388.
--> Saving reprojected data from v2m_bin.nc:v2m (bilinear).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.094078.
> chunksize|dimsize: [time_bins: 1|1, y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of qv_bin.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.4KB, execution-time: 0:00:00.045999.
--> Saving reprojected data from qv_bin.nc:qv (bilinear).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.084085.
> chunksize|dimsize: [time_bins: 1|1, y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of p_air_bin.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.4KB, execution-time: 0:00:00.047358.
--> Saving reprojected data from p_air_bin.nc:p_air (bilinear).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.085413.
> chunksize|dimsize: [time_bins: 1|1, y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of p_air_0_bin.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.4KB, execution-time: 0:00:00.047125.
--> Saving reprojected data from p_air_0_bin.nc:p_air_0 (bilinear).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.077907.
> chunksize|dimsize: [time_bins: 1|1, y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of wv_bin.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.4KB, execution-time: 0:00:00.047779.
--> Saving reprojected data from wv_bin.nc:wv (bilinear).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.093219.
> chunksize|dimsize: [time_bins: 1|1, y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of land_mask_bin.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.5KB, execution-time: 0:00:00.033764.
--> Saving reprojected data from land_mask_bin.nc:land_mask (nearest).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.084901.
> chunksize|dimsize: [y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of rs_min_bin.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.5KB, execution-time: 0:00:00.034263.
--> Saving reprojected data from rs_min_bin.nc:rs_min (nearest).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.075139.
> chunksize|dimsize: [y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of z_obst_max_bin.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.5KB, execution-time: 0:00:00.034891.
--> Saving reprojected data from z_obst_max_bin.nc:z_obst_max (nearest).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.072843.
> chunksize|dimsize: [y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of lw_offset_bin.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.8KB, execution-time: 0:00:00.051831.
--> Saving reprojected data from lw_offset_bin.nc:lw_offset (bilinear).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.082949.
> chunksize|dimsize: [y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of lw_slope_bin.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.6KB, execution-time: 0:00:00.053474.
--> Saving reprojected data from lw_slope_bin.nc:lw_slope (bilinear).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.081424.
> chunksize|dimsize: [y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of z_oro_bin.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.6KB, execution-time: 0:00:00.052415.
--> Saving reprojected data from z_oro_bin.nc:z_oro (bilinear).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.077875.
> chunksize|dimsize: [y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of rn_offset_bin.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.5KB, execution-time: 0:00:00.053359.
--> Saving reprojected data from rn_offset_bin.nc:rn_offset (bilinear).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.075399.
> chunksize|dimsize: [y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of rn_slope_bin.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.6KB, execution-time: 0:00:00.052003.
--> Saving reprojected data from rn_slope_bin.nc:rn_slope (bilinear).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.081899.
> chunksize|dimsize: [y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of t_amp_year_bin.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.5KB, execution-time: 0:00:00.051138.
--> Saving reprojected data from t_amp_year_bin.nc:t_amp_year (bilinear).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.082502.
> chunksize|dimsize: [y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of t_opt_bin.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.5KB, execution-time: 0:00:00.051122.
--> Saving reprojected data from t_opt_bin.nc:t_opt (bilinear).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.082784.
> chunksize|dimsize: [y: 808|808, x: 1009|1009]
--> Selected `reproject_chunk` for reprojection of vpd_slope_bin.nc.
--> Warping VRT to netCDF.
> peak-memory-usage: 4.5KB, execution-time: 0:00:00.052030.
--> Saving reprojected data from vpd_slope_bin.nc:vpd_slope (bilinear).
> peak-memory-usage: 24.9MB, execution-time: 0:00:02.076794.
> chunksize|dimsize: [y: 808|808, x: 1009|1009]
--> Calculating local means (r = 0.25°) of `z`.
--> Filling 3 missing pixels in 'z'.
--> Applying 'lapse_rate'.
--> Applying 'rename_vars'.
--> Applying 'fill_attrs'.
--> Applying 'calc_doys'.
--> Applying 'remove_empty_statics'.
--> Applying 'add_constants_new'.
--> Creating merged file `et_look_in_.nc`.
> peak-memory-usage: 68.7MB, execution-time: 0:00:03.073477.
> chunksize|dimsize: [time_bins: 1|1, y: 808|808, x: 1009|1009]
< PRE_ET_LOOK (0:06:34.061266)
[16]:
ds
[16]:
<xarray.Dataset>
Dimensions: (y: 808, x: 1009, time_bins: 1)
Coordinates:
* y (y) float64 29.7 29.7 29.7 29.7 ... 28.9 28.9 28.9 28.9
* x (x) float64 30.2 30.2 30.2 30.2 ... 31.2 31.2 31.2 31.2
spatial_ref int64 ...
* time_bins (time_bins) datetime64[ns] 2021-07-01
Data variables: (12/78)
ndvi (time_bins, y, x) float64 dask.array<chunksize=(1, 808, 1009), meta=np.ndarray>
r0 (time_bins, y, x) float64 dask.array<chunksize=(1, 808, 1009), meta=np.ndarray>
se_root (time_bins, y, x) float64 dask.array<chunksize=(1, 808, 1009), meta=np.ndarray>
p_24 (time_bins, y, x) float64 dask.array<chunksize=(1, 808, 1009), meta=np.ndarray>
z (y, x) float64 dask.array<chunksize=(808, 1009), meta=np.ndarray>
ra_24 (time_bins, y, x) float64 dask.array<chunksize=(1, 808, 1009), meta=np.ndarray>
... ...
o2 float64 ...
co2_ref int64 ...
gcgdm float64 ...
phot_eff float64 ...
r0_bare float64 ...
r0_full float64 ...And then again, run pwapor.et_look.main like this.
[17]:
ds = pywapor.et_look.main(ds)
> ET_LOOK
--> Running `et_look` (v2).
--> Calculating `declination`.
--> Calculating `inverse_earth_sun_distance`.
--> Calculating `vegetation_cover`.
--> Calculating `leaf_area_index`.
--> Calculating `effective_leaf_area_index`.
--> Calculating `soil_fraction`.
--> Calculating `latitude_rad`.
--> Calculating `sunset_hour_angle`.
--> Calculating `daily_solar_radiation_toa_flat`.
--> Calculating `transmissivity`.
--> Calculating `stress_radiation`.
--> Calculating `air_pressure_kpa2mbar`.
--> Calculating `air_pressure_daily`.
--> Calculating `air_pressure`.
--> Calculating `vapour_pressure_from_specific_humidity_daily`.
--> Calculating `vapour_pressure_from_specific_humidity`.
--> Calculating `saturated_vapour_pressure`.
--> Calculating `vapour_pressure_deficit_daily`.
--> Calculating `vapour_pressure_deficit`.
--> Calculating `stress_vpd`.
--> Calculating `stress_temperature`.
--> Calculating `atmospheric_canopy_resistance`.
--> Calculating `air_temperature_kelvin_daily`.
--> Calculating `air_temperature_kelvin`.
--> Calculating `longwave_radiation_fao`.
--> Calculating `interception_mm`.
--> Calculating `latent_heat_daily`.
--> Calculating `latent_heat`.
--> Calculating `interception_wm2`.
--> Calculating `net_radiation`.
--> Calculating `net_radiation_canopy`.
--> Calculating `stress_moisture`.
--> Calculating `canopy_resistance`.
--> Calculating `obstacle_height`.
--> Calculating `roughness_length`.
--> Calculating `wind_speed`.
--> Calculating `initial_canopy_aerodynamic_resistance`.
--> Calculating `wind_speed_blending_height_daily`.
--> Calculating `wind_speed_blending_height`.
--> Calculating `dry_air_density_daily`.
--> Calculating `dry_air_density`.
--> Calculating `moist_air_density_daily`.
--> Calculating `moist_air_density`.
--> Calculating `air_density_daily`.
--> Calculating `air_density`.
--> Calculating `psychrometric_constant_daily`.
--> Calculating `psychrometric_constant`.
--> Calculating `slope_saturated_vapour_pressure_daily`.
--> Calculating `slope_saturated_vapour_pressure`.
--> Calculating `saturated_vapour_pressure`.
--> Calculating `initial_daily_transpiration`.
--> Calculating `initial_sensible_heat_flux_canopy_daily`.
--> Calculating `displacement_height`.
--> Calculating `initial_friction_velocity_daily`.
--> Calculating `transpiration`.
--> Calculating `transpiration_mm`.
--> Calculating `net_radiation_soil`.
--> Calculating `soil_resistance`.
--> Calculating `initial_soil_aerodynamic_resistance`.
--> Calculating `initial_friction_velocity_soil_daily`.
--> Calculating `soil_thermal_conductivity`.
--> Calculating `volumetric_heat_capacity`.
--> Calculating `damping_depth`.
--> Calculating `bare_soil_heat_flux`.
--> Calculating `soil_heat_flux`.
--> Calculating `initial_daily_evaporation`.
--> Calculating `initial_sensible_heat_flux_soil_daily`.
--> Calculating `evaporation`.
--> Calculating `evaporation_mm`.
--> Calculating `et_actual_mm`.
--> Calculating `net_radiation_grass`.
--> Calculating `et_reference`.
--> Calculating `et_reference_mm`.
--> Calculating `air_temperature_kelvin_daily`.
--> Calculating `air_temperature_kelvin`.
--> Calculating `air_temperature_kelvin_daily`.
--> Calculating `air_temperature_kelvin`.
--> Calculating `mean_temperature_kelvin_daytime`.
--> Calculating `temperature_dependency`.
--> Calculating `affinity_constant_co2`.
--> Calculating `inhibition_constant_o2`.
--> Calculating `co2_o2_specificity_ratio`.
--> Calculating `co2_level_annual`.
--> Calculating `autotrophic_respiration`.
--> Calculating `par`.
--> Calculating `fpar`.
--> Calculating `co2_fertilisation`.
--> Calculating `net_primary_production_max`.
--> Calculating `net_primary_production`.
--> Saving output to `et_look_out_.nc`.
> peak-memory-usage: 1.8GB, execution-time: 0:00:03.027793.
> chunksize|dimsize: [time_bins: 1|1, y: 808|808, x: 1000|1009]
< ET_LOOK (0:00:10.915344)
Notice how the some nodata areas from before have been filled in thanks to temporal interpolation!
[18]:
ds.et_24_mm.isel(time_bins = 0).plot()
[18]:
<matplotlib.collections.QuadMesh at 0x163170490>