Skip to content

Instantly share code, notes, and snippets.

@Explosion-Scratch
Created November 1, 2021 18:51
Show Gist options
  • Save Explosion-Scratch/357c2eebd8254f8ea5548b0e6ac7a61b to your computer and use it in GitHub Desktop.
Save Explosion-Scratch/357c2eebd8254f8ea5548b0e6ac7a61b to your computer and use it in GitHub Desktop.
Compress string using gzip and native browser APIs
function compress(string, encoding) {
const byteArray = new TextEncoder().encode(string);
const cs = new CompressionStream(encoding);
const writer = cs.writable.getWriter();
writer.write(byteArray);
writer.close();
return new Response(cs.readable).arrayBuffer();
}
function decompress(byteArray, encoding) {
const cs = new DecompressionStream(encoding);
const writer = cs.writable.getWriter();
writer.write(byteArray);
writer.close();
return new Response(cs.readable).arrayBuffer().then(function (arrayBuffer) {
return new TextDecoder().decode(arrayBuffer);
});
}
@USMortality
Copy link

USMortality commented Sep 10, 2023

You should be able to use it like this:

const compressed = await(“hello world”, “utf-8”)
const decompressed = await decompress(compressed)

(Not tested; sent from my phone)

@TCH68k
Copy link

TCH68k commented Sep 11, 2023

Yep, works:

function compress(string, encoding) {
  const byteArray = new TextEncoder().encode(string);
  const cs = new CompressionStream(encoding);
  const writer = cs.writable.getWriter();
  writer.write(byteArray);
  writer.close();
  return new Response(cs.readable).arrayBuffer();
}

function promptDownload(filename, content, mimetype)
{
	if (mimetype === undefined)
	{
		mimetype = 'application/octet-stream';
	}

	var a = window.document.createElement('A');
	a.href = window.URL.createObjectURL(new Blob([content], {type: mimetype}));
	a.download = filename;
	document.body.appendChild(a);
	a.click();
	document.body.removeChild(a);
}

async function do_stuff()
{
	const s = "Árvíztűrő tükörfúrógép\n";
	var c = await compress(s + s + s + s + s + s + s + s, 'gzip');
	promptDownload('stuff.txt.gz', c);
}
do_stuff();

Thank you for the help and Explosion-Scratch for writing it.

@Explosion-Scratch
Copy link
Author

No problem @TCH68k - Happy to help, mention me if you want help with anything lol

@A99US
Copy link

A99US commented Aug 9, 2024

How do you catch the error?

I tried to decompress random Buffer bytes and got an error thrown Uncaught (in promise) TypeError : The compress data was not valid. Invalid block data.

I tried to catch it with .catch(err=>{console.log(err)}) but failed. Only got TypeError : Failed to fetch.

I've been reading about async function and Promise, but still a dead end. Sorry if the question is too stupid or has an obvious answer.

@Explosion-Scratch
Copy link
Author

@A99US You're using promises correctly, but you have to decompress something that is compressed already if that makes sense.

For example:
const compressed = await compress("Hello world") -> Compressed arrayBuffer, then:
await decompress(compressed) === "Hello World"

@A99US
Copy link

A99US commented Aug 9, 2024

@Explosion-Scratch Thanks for your reply, Sir. I'm sorry if my question was ambiguous.

I know that it was going to fail. It's just an example to make the function throw an error at me. Another example would be if the compressed data is altered, it will throw error Uncaught (in promise) TypeError : Junk found after the end of compressed data. But if I use .catch(err=>{console.log(err)}), it catch a totally different message, TypeError : Failed to fetch.

My question is how can I catch the real error message?

My codes is like this:
return new Response(cs.readable).arrayBuffer() .catch(err=>{console.log(err)})

@Explosion-Scratch
Copy link
Author

@A99US There are two errors going on here, and Failed to fetch is the first one (so the one you catch). await (new Response(cs.readable)).arrayBuffer() causes the error (for me TypeError: The compressed data was not valid) but because we're returning the promise, not the result of awaiting said promise that error message gets turned into "Failed to fetch".

Basically:

Compressed array buffer -> decompression stream writer -> try to create a Response object (error here so Response object gets returned, error in response object = Failed to fetch) -> (if no error) Convert response to string and return string

So if there's an error, you get a Response promise that throws an error, if there's not you get a string. The error you're seeing is from the zlib internal library and I'm not sure how to catch that error message.

@A99US
Copy link

A99US commented Aug 9, 2024

@Explosion-Scratch I appreciate your times and your replies, Sir.
Cheers

@A99US
Copy link

A99US commented Aug 10, 2024

@Explosion-Scratch So I've found a way to catch the error message. Apparently it's in the writer's Promise. I still dont know if it is the correct way, but it's one of those moments of "Dont know how or why it works, but it works". If you or anyone else understand more about it, please correct my codes.

async function decomp(str,mode="gzip"){
  // Using Node Buffer for encoder
  str = Buffer.from(str, "base64")
  const cs = new DecompressionStream(mode)
  const writer = cs.writable.getWriter()
  writer.write(str)
  writer.close()
  return await new Response(cs.readable).arrayBuffer()
    .then(
      // Resolve Function
      arr=>(Buffer.from(arr)).toString("utf8"),
      // Reject Function
      async _=>{throw new Error(await Promise.reject(await writer.closed))}
    )
}

let string = [
  // "These strings were compressed!" compressed with gzip and base64 encoder
  `H4sIAAAAAAAACgvJSC1OVSguKcrMSy9WKE8tSlVIzs8tKEotLk5NUQQAfREzrR4AAAA`,
  // Change capital R near the end
  `H4sIAAAAAAAACgvJSC1OVSguKcrMSy9WKE8tSlVIzs8tKEotLk5NUQQAfREzrr4AAAA`,
  // Change 1 char at the end
  `H4sIAAAAAAAACgvJSC1OVSguKcrMSy9WKE8tSlVIzs8tKEotLk5NUQQAfREzrR4AAAx`,
  // Add 1 char at the end
  `H4sIAAAAAAAACgvJSC1OVSguKcrMSy9WKE8tSlVIzs8tKEotLk5NUQQAfREzrR4AAAAx`,
  ],
  result = []

console.clear()

for(let item of string){
  try{
    result.push(
      "Succesfully decompressing!\n\n" + "String Result : "+ (await decomp(item))
    )
  }
  catch(err){
    result.push("Failed to decompress!\n\n"+ err.name +" : "+ err.message)
  }
}

console.log(result.join("\n\n=====================================================================\n\n"))

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment