Created
June 5, 2020 18:52
-
-
Save tlancon/10995e186801c66b3c7912aeb90bb949 to your computer and use it in GitHub Desktop.
Reject outliers from a 1D signal based on a sliding window and a deviation threshold.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
def reject_outliers(arr, window_size, threshold): | |
""" | |
Given a 1D signal, replace outliers with the average of the surrounding points. | |
Does the following for every point: | |
1. Computes the median of the sliding window | |
2. Computes the percentage difference between the current point and the | |
sliding window median | |
3. If that deviation exceeds the given threshold, replaces that point with | |
the average of the surrounding two points. | |
4. Otherwise, returns the same point. | |
Parameters | |
---------- | |
arr : 1D array | |
Signal with outliers | |
window_size : int | |
Size of sliding window | |
threshold : float | |
Minimum acceptable percentage deviation (as decimal) | |
Returns | |
------- | |
result : 1D array | |
Original signal with outliers removed. | |
""" | |
if len(arr.shape) > 1: | |
print('Only 1D arrays supported.') | |
return | |
result = np.empty_like(arr) | |
for i in range(0, arr.shape[0]): | |
min_i = int(max(0, np.floor(i - window_size / 2))) | |
max_i = int(min(arr.shape[0], np.floor(i + window_size / 2 + 1))) | |
med = np.median(arr[min_i:max_i]) | |
dev = np.abs(arr[i] - med) / med | |
if dev > threshold: | |
result[i] = np.average([arr[i-1], arr[i+1]]) | |
else: | |
result[i] = arr[i] | |
return result | |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment