Skip to content

Instantly share code, notes, and snippets.

@kevcodez
Created October 7, 2022 15:02
Show Gist options
  • Save kevcodez/6a0b431e5cac9d717f10456b7e967f1f to your computer and use it in GitHub Desktop.
Save kevcodez/6a0b431e5cac9d717f10456b7e967f1f to your computer and use it in GitHub Desktop.
medium_mongodb_performance_parallel_fetching
async function getQuotesForPeriod(
assetIdentifier: string,
from: Date,
to: Date
): Promise<Quote[]> {
const quotesCollection = await getQuotesCollection();
/**
* When fetching multiple years of data, queries are fast, but retrieval not so much.
* To improve query performance, we split the interval into a few chunks and query/fetch the data in parallel,
* improving read throughput.
*/
const intervals = chunkInterval(from, to).reverse(); // descending
const all = await Promise.all(
intervals.map(async (interval) => {
return quotesCollection
.find(
{
isin: assetIdentifier,
date: { $gte: interval.start!, $lte: interval.end! },
},
)
.sort({ date: -1 })
.toArray()
})
);
return all.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment