Bias Correction Module

[andersy005]

Bias Correction Process

• Moving Window

  • to handle seasonality

• Normalize data

  • recenter (mean 0)
  • rescale ( sd 1)
  • detrend
  • transform to deskew (e.g., log precip)

• Distribution Mapping (DM)

  • 
  • reshape distribution

• Denormalize

  • 
  • 
  • reapply trend
  • reverse transform

[andersy005]

@kmpaul @davidedelvento

There is this new xarray-based package called Downscale. It is still at an extremely early stage of development. However, by looking at some of the features it will have, it looks quite promising.

Below are some those features:

• Interface directly with xarray, utilizing coordinates defined on DataArray objects.
• Toolbox of common bias correction techniques
• Toolbox of common spatial disaggregation/smoothing techniques
• Toolbox of common constructed analogue techniques
• Ability to mix & match bias correction and downscaling methods to form existing/novel techniques from downscaling literature.

For more details, you can check this paper doc Downscale.

The Downscale project is hosted here.

My plan for the bias correction module was to use xarray' s register_dataset_accessor() and register_dataarray_accessor() decorators for adding custom “accessors” on xarray objects to write a custom “biascorrect” accessor implementing a bias correction specific extension to xarray. The Downscale package is using the exact same concept.

Currently, I am keeping an eye on the development progress of Downscale to decide whether I should go ahead and (reinvent the wheel) implement my own module or take advantage of that package.

[andersy005]

Reference materials

esd - The Empirical-Statistical Downscaling tool
esd - github repo
Pacific Climate ClimDown

Reference Papers

Intercomparison of Machine Learning Methods for Statistical Downscaling: The Case of Daily and Extreme Precipitation

[andersy005]

The Bias correction workflow will follow the following steps:

• Step 1: Load Local NetCDF Files for both reference and model (present and future) datasets into Spark RDDs.

• Step 2: Temporal subsetting on the selected months or days

• Step 3: Spatial aggregation of observational data into the model grid

• Step 4: Create a statistical downscaling object and downscaling model output

• Step 5: Create plots and save results to netcdf files.