Skip to content

Instantly share code, notes, and snippets.

@semlinker
Created July 25, 2022 10:43
Show Gist options
  • Save semlinker/cd92cf3191690d5753de1842a1648c18 to your computer and use it in GitHub Desktop.
Save semlinker/cd92cf3191690d5753de1842a1648c18 to your computer and use it in GitHub Desktop.
Download Chunk
async function download({ url, chunkSize, poolLimit = 1 }) {
const contentLength = await getContentLength(url);
const chunks = typeof chunkSize === "number" ? Math.ceil(contentLength / chunkSize) : 1;
const results = await asyncPool(
poolLimit,
[...new Array(chunks).keys()],
(i) => {
let start = i * chunkSize;
let end = i + 1 == chunks ? contentLength - 1 : (i + 1) * chunkSize - 1;
return getBinaryContent(url, start, end, i);
}
);
const sortedBuffers = results
.map((item) => new Uint8Array(item.buffer));
return concatenate(sortedBuffers);
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment