Compress string using gzip and native browser APIs

Compress string.js

function compress(string, encoding) {
  const byteArray = new TextEncoder().encode(string);
  const cs = new CompressionStream(encoding);
  const writer = cs.writable.getWriter();
  writer.write(byteArray);
  writer.close();
  return new Response(cs.readable).arrayBuffer();
}

function decompress(byteArray, encoding) {
  const cs = new DecompressionStream(encoding);
  const writer = cs.writable.getWriter();
  writer.write(byteArray);
  writer.close();
  return new Response(cs.readable).arrayBuffer().then(function (arrayBuffer) {
    return new TextDecoder().decode(arrayBuffer);
  });
}

Thanks for this!

Thanks for this!

No problem! This works well with a lot of the Web Crypto APIs

Thx - Typescript version:

export const compress = async (
  str: string,
  encoding = 'gzip' as CompressionFormat
): Promise<ArrayBuffer> => {
  const byteArray = new TextEncoder().encode(str)
  const cs = new CompressionStream(encoding)
  const writer = cs.writable.getWriter()
  writer.write(byteArray)
  writer.close()
  return new Response(cs.readable).arrayBuffer()
}

export const decompress = async (
  byteArray: string[],
  encoding = 'gzip' as CompressionFormat
): Promise<string> => {
  const cs = new DecompressionStream(encoding)
  const writer = cs.writable.getWriter()
  writer.write(byteArray)
  writer.close()
  const arrayBuffer = await new Response(cs.readable).arrayBuffer()
  return new TextDecoder().decode(arrayBuffer)
}

@USMortality nice

My TS version

const compress = (string: string, encoding: string): Promise<ArrayBuffer> => {
  const byteArray: Uint8Array = new TextEncoder().encode(string);
  const cs: CompressionStream = new CompressionStream(encoding);
  cs.writable.getWriter().write(byteArray).close();
  return new Response(cs.readable).arrayBuffer();
}

const decompress = (byteArray: Uint8Array, encoding: string): Promise<string> => {
  const cs: DecompressionStream = new DecompressionStream(encoding);
  cs.writable.getWriter().write(byteArray).close();
  return new Response(cs.readable).arrayBuffer().then((arrayBuffer: ArrayBuffer) => new TextDecoder().decode(arrayBuffer));
}

@Explosion-Scratch did you find a TS version that has CompressionStream defined? I can't find it in the latest release at all.

Resolves for me in VS: https://developer.mozilla.org/en-US/docs/Web/API/CompressionStream
tsc -version: Version 5.1.6

Ah confirmed that works. Looks like I need to try to bump our Typescript version which, sadly, might be a bit complicated. Thanks for your help!

thanks

How this can be used from another function? I would like to gzip compress a string and get the result in a string. But all i get is a Promise and i cannot get the result out of it. No, await does not work. Even if i make the function async, all i get is a Promise.

function compress(string, encoding) {
  const byteArray = new TextEncoder().encode(string);
  const cs = new CompressionStream(encoding);
  const writer = cs.writable.getWriter();
  writer.write(byteArray);
  writer.close();
  return new Response(cs.readable).arrayBuffer();
}

async function wrapper(s)
{
	var x = compress(s, 'gzip').then
	(
		function (arrayBuffer)
		{
			return new TextDecoder().decode(arrayBuffer);
		}
	);
	var y = await x;
	return y;
}

var a = wrapper('Example string...');
console.log(a);

And it will get a Promise. I would like to use it synchronously.

You should be able to use it like this:

const compressed = await(“hello world”, “utf-8”)
const decompressed = await decompress(compressed)

(Not tested; sent from my phone)

Yep, works:

function compress(string, encoding) {
  const byteArray = new TextEncoder().encode(string);
  const cs = new CompressionStream(encoding);
  const writer = cs.writable.getWriter();
  writer.write(byteArray);
  writer.close();
  return new Response(cs.readable).arrayBuffer();
}

function promptDownload(filename, content, mimetype)
{
	if (mimetype === undefined)
	{
		mimetype = 'application/octet-stream';
	}

	var a = window.document.createElement('A');
	a.href = window.URL.createObjectURL(new Blob([content], {type: mimetype}));
	a.download = filename;
	document.body.appendChild(a);
	a.click();
	document.body.removeChild(a);
}

async function do_stuff()
{
	const s = "Árvíztűrő tükörfúrógép\n";
	var c = await compress(s + s + s + s + s + s + s + s, 'gzip');
	promptDownload('stuff.txt.gz', c);
}
do_stuff();

Thank you for the help and Explosion-Scratch for writing it.

No problem @TCH68k - Happy to help, mention me if you want help with anything lol

How do you catch the error?

I tried to decompress random Buffer bytes and got an error thrown Uncaught (in promise) TypeError : The compress data was not valid. Invalid block data.

I tried to catch it with .catch(err=>{console.log(err)}) but failed. Only got TypeError : Failed to fetch.

I've been reading about async function and Promise, but still a dead end. Sorry if the question is too stupid or has an obvious answer.

@A99US You're using promises correctly, but you have to decompress something that is compressed already if that makes sense.

For example:
const compressed = await compress("Hello world") -> Compressed arrayBuffer, then:
await decompress(compressed) === "Hello World"

@Explosion-Scratch Thanks for your reply, Sir. I'm sorry if my question was ambiguous.

I know that it was going to fail. It's just an example to make the function throw an error at me. Another example would be if the compressed data is altered, it will throw error Uncaught (in promise) TypeError : Junk found after the end of compressed data. But if I use .catch(err=>{console.log(err)}), it catch a totally different message, TypeError : Failed to fetch.

My question is how can I catch the real error message?

My codes is like this:
return new Response(cs.readable).arrayBuffer() .catch(err=>{console.log(err)})

@A99US There are two errors going on here, and Failed to fetch is the first one (so the one you catch). await (new Response(cs.readable)).arrayBuffer() causes the error (for me TypeError: The compressed data was not valid) but because we're returning the promise, not the result of awaiting said promise that error message gets turned into "Failed to fetch".

Basically:

Compressed array buffer -> decompression stream writer -> try to create a Response object (error here so Response object gets returned, error in response object = Failed to fetch) -> (if no error) Convert response to string and return string

So if there's an error, you get a Response promise that throws an error, if there's not you get a string. The error you're seeing is from the zlib internal library and I'm not sure how to catch that error message.

@Explosion-Scratch I appreciate your times and your replies, Sir.
Cheers

@Explosion-Scratch So I've found a way to catch the error message. Apparently it's in the writer's Promise. I still dont know if it is the correct way, but it's one of those moments of "Dont know how or why it works, but it works". If you or anyone else understand more about it, please correct my codes.

async function decomp(str,mode="gzip"){
  // Using Node Buffer for encoder
  str = Buffer.from(str, "base64")
  const cs = new DecompressionStream(mode)
  const writer = cs.writable.getWriter()
  writer.write(str)
  writer.close()
  return await new Response(cs.readable).arrayBuffer()
    .then(
      // Resolve Function
      arr=>(Buffer.from(arr)).toString("utf8"),
      // Reject Function
      async _=>{throw new Error(await Promise.reject(await writer.closed))}
    )
}

let string = [
  // "These strings were compressed!" compressed with gzip and base64 encoder
  `H4sIAAAAAAAACgvJSC1OVSguKcrMSy9WKE8tSlVIzs8tKEotLk5NUQQAfREzrR4AAAA`,
  // Change capital R near the end
  `H4sIAAAAAAAACgvJSC1OVSguKcrMSy9WKE8tSlVIzs8tKEotLk5NUQQAfREzrr4AAAA`,
  // Change 1 char at the end
  `H4sIAAAAAAAACgvJSC1OVSguKcrMSy9WKE8tSlVIzs8tKEotLk5NUQQAfREzrR4AAAx`,
  // Add 1 char at the end
  `H4sIAAAAAAAACgvJSC1OVSguKcrMSy9WKE8tSlVIzs8tKEotLk5NUQQAfREzrR4AAAAx`,
  ],
  result = []

console.clear()

for(let item of string){
  try{
    result.push(
      "Succesfully decompressing!\n\n" + "String Result : "+ (await decomp(item))
    )
  }
  catch(err){
    result.push("Failed to decompress!\n\n"+ err.name +" : "+ err.message)
  }
}

console.log(result.join("\n\n=====================================================================\n\n"))