Benny Istanto bennyistanto

Enhanced Night Time Light (NTL) Classification for GEEST

Approach and Rationale

We propose refining the NTL classification in GEEST to better reflect local conditions, particularly for small island countries. Instead of using fixed thresholds or coverage percentages, we'll employ local statistics to create a more contextually relevant classification. This approach is supported by Elvidge et al. (2013), who emphasize the importance of considering local context in night-time light analysis.

Standard Classification Procedure:

Clip the global NTL raster to the country boundary.
Calculate statistics for the clipped raster: min, max, mean, median, and 75th percentile.

Rainfall Distribution Analysis

This rainfall distribution analysis script provides a comprehensive examination of observed and satellite-based daily precipitation data, offering valuable insights for hydrologists, climatologists, and water resource managers. The analysis begins by generating synthetic rainfall data to simulate both ground observations and satellite estimates, including the introduction of extreme events. This approach allows for a controlled comparison between different data sources and methods of analysis.

The script employs a range of advanced statistical techniques to characterize precipitation patterns and extreme events. It utilizes the method of L-moments to fit Gamma distributions to the overall rainfall patterns, providing a robust representation of the general precipitation behavior. For extreme rainfall events, the script applies Generalized Pareto Distributions (GPD) and incorporates cross-validation techniques to ensure reliable modeling of these critical occurrences. This dua

Moving Window illustration

You can paste the below code into an online Python compiler like https://python-fiddle.com/ and grab the result instantly.

This script will generate a plot that illustrates:

This highlighting the spatial moving window on the top layer. The selected pixel at (4,5) will be red, and the 24 surrounding pixels within the 5x5 window will be pink.
This illustrate how the temporal moving window considers data from multiple layers (time steps) when smoothing the value for a single pixel.

The moving window approach is used to balance the advantages of capturing both spatial and temporal variations in precipitation patterns. By considering both spatial and temporal windows, the bias correction method can address local variations and seasonal changes more effectively.

LSEQM Bias Correction

Bias correction of satellite-based precipitation estimates is crucial in climate studies to enhance their accuracy and reliability. This exercise explores a method that combines Linear Scaling (LS) and Empirical Quantile Mapping (EQM) with tail adjustment using the Generalized Pareto Distribution (GPD). This hybrid approach aims to correct both the mean bias and the distributional discrepancies between satellite estimates and ground-based observations.

The following sections will describe the methodology, the role of different functions, their strengths, and weaknesses in achieving accurate bias correction.

1 Linear Scaling Approach

Visual representation of flood probability model

You can paste the below code into an online Python compiler like https://python-fiddle.com/ and grab the result instantly.

This script will generate a plot that illustrates:

The original model (blue line).
A scenario with a steeper slope (red line), showing a more rapid increase in flood risk with rainfall.
A scenario with a gentler slope (green line), showing a more gradual increase in risk.
A scenario with a less negative intercept (purple line), resulting in higher probabilities at lower rainfall amounts.

xkcd-style Pettitt's illustration on Homogeneity testing

The script below try to simulate how Pettitt test detects inhomogeneous or homogeneous on precipitation data. Here's the updated approach:

Daily Precipitation Data:
- We have daily precipitation.
Generate Monthly Maximum Precipitation Values:

Create synthetic monthly maxima with distinct differences to ensure inhomogeneity.

	# Copy and paste the code below into https://python-fiddle.com/
	# You will get instant result: GEV 25-year threshold: 96.12570045491432
	import numpy as np
	from scipy.stats import genextreme

	# This data is the maximum daily precipitation in a month (let say May) for the period
	# 2001-2023
	data = [
	35.809994, 55.120003, 97.575005, 17.01, 43.260006, 50.285004, 51.489998, 59.05999,
	58.339996, 82.835, 42.875, 55.574997, 105.149994, 61.155, 83.68001, 73.56,

	import xarray as xr
	import numpy as np
	import os
	import glob
	import pandas as pd
	from tqdm import tqdm
	from scipy.stats import genextreme

	# Define directories and filenames for threshold calculations
	threshold_calculations = [

	import xarray as xr
	import numpy as np
	import os
	import glob
	import pandas as pd
	from tqdm import tqdm

	# Define the input and output directories
	max_calculations = [
	{"days": 1, "input_dir": "/mnt/d/temp/imerg/data/fdd_1day", "output_dir": "/mnt/d/temp/imerg/data/fddmaxd_1day"},

	import xarray as xr
	import os
	import glob
	from tqdm import tqdm
	import pandas as pd

	# Define directories and filenames for max value calculations
	max_calculations = [
	{"days": 1, "input_dir": "/mnt/d/temp/imerg/data/fdd_1day", "output_dir": "/mnt/d/temp/imerg/data/fddmaxm_1day"},
	{"days": 2, "input_dir": "/mnt/d/temp/imerg/data/fdd_2day", "output_dir": "/mnt/d/temp/imerg/data/fddmaxm_2day"},