Skip to content

Instantly share code, notes, and snippets.

@kmizumar
Created July 25, 2018 18:26
Show Gist options
  • Save kmizumar/086b1232f73c271104a5c969ac4ef5aa to your computer and use it in GitHub Desktop.
Save kmizumar/086b1232f73c271104a5c969ac4ef5aa to your computer and use it in GitHub Desktop.
DECIMAL(p1,s1) / DECIMAL(p2,s2) = DECIMAL(?,?) in Impala (CDH5.12.2)
MAX_PRECISION = 38
MAX_SCALE = MAX_PRECISION
MIN_ADJUSTED_SCALE = 6
DECIMAL_DIVISION_SCALE_INCREMENT = 4
MAX_DECIMAL4_PRECISION = 9
MAX_DECIMAL8_PRECISION = 18
def ps_div(p1, s1, p2, s2):
result_scale = max(DECIMAL_DIVISION_SCALE_INCREMENT, s1 + p2 + 1)
result_precision = p1 - s1 + s2 + result_scale
if result_precision > MAX_PRECISION:
result_scale = max(s1, s2)
result_precision = MAX_PRECISION
return (result_precision, result_scale)
def ps_div2(p1, s1, p2, s2):
result_scale = max(MIN_ADJUSTED_SCALE, s1 + p2 + 1)
result_precision = p1 - s1 + s2 + result_scale
if result_precision > MAX_PRECISION:
min_scale = min(result_scale, MIN_ADJUSTED_SCALE)
delta = result_precision - MAX_PRECISION
result_precision = MAX_PRECISION
result_scale = max(result_scale - delta, min_scale)
return (result_precision, result_scale)
@kmizumar
Copy link
Author

what the hell

[quickstart.cloudera:21000] > SET DECIMAL_V2=0;
DECIMAL_V2 set to 0
[quickstart.cloudera:21000] > SELECT CAST(1000 AS DECIMAL(18,0)) / CAST(3 AS DECIMAL(18,0));
Query: select CAST(1000 AS DECIMAL(18,0)) / CAST(3 AS DECIMAL(18,0))
Query submitted at: 2018-07-25 10:44:58 (Coordinator: http://quickstart.cloudera:25000)
Query progress can be monitored at: http://quickstart.cloudera:25000/query_plan?query_id=e64ced7f852e4cd7:bae0a00800000000
+--------------------------------------------------------+
| cast(1000 as decimal(18,0)) / cast(3 as decimal(18,0)) |
+--------------------------------------------------------+
| 333.3333333333333333333                                |
+--------------------------------------------------------+
Fetched 1 row(s) in 0.11s
[quickstart.cloudera:21000] > SELECT CAST(1000 AS DECIMAL(19,0)) / CAST(3 AS DECIMAL(19,0));
Query: select CAST(1000 AS DECIMAL(19,0)) / CAST(3 AS DECIMAL(19,0))
Query submitted at: 2018-07-25 10:45:03 (Coordinator: http://quickstart.cloudera:25000)
Query progress can be monitored at: http://quickstart.cloudera:25000/query_plan?query_id=984b971d834535a0:de89165200000000
+--------------------------------------------------------+
| cast(1000 as decimal(19,0)) / cast(3 as decimal(19,0)) |
+--------------------------------------------------------+
| 333                                                    |
+--------------------------------------------------------+
Fetched 1 row(s) in 0.11s
[quickstart.cloudera:21000] > 

@kmizumar
Copy link
Author

this means

  • DECIMAL(18,0) divide by DECIMAL(18,0) generates DECIMAL(37,19)
  • DECIMAL(19,0) divide by DECIMAL(19,0) generates DECIMAL(38,0).
>>> ps_div(18,0,18,0)
(37, 19)
>>> ps_div(19,0,19,0)
(38, 0)
>>> 

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment