Skip to content

Instantly share code, notes, and snippets.

@baranovxyz
Created May 11, 2020 20:57
Show Gist options
  • Save baranovxyz/3903ba87fb4f378802f9e972d6070e36 to your computer and use it in GitHub Desktop.
Save baranovxyz/3903ba87fb4f378802f9e972d6070e36 to your computer and use it in GitHub Desktop.
function compress(column) {
column.compression = 'DICTIONARY';
const dictionary = [];
const indexes = {};
column.values = column.values.map(value => {
if (value === null) return null;
// add string to dictionary if there is no such string yet
if (indexes[value] === undefined) {
indexes[value] = dictionary.length;
dictionary.push(value);
}
// return value index
return indexes[value];
});
return Object.assign(column, { dictionary });
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment