Skip to content

Instantly share code, notes, and snippets.

@samueleresca
Last active August 30, 2020 18:21
Show Gist options
  • Save samueleresca/799104fb8acef5b65143093f1b662118 to your computer and use it in GitHub Desktop.
Save samueleresca/799104fb8acef5b65143093f1b662118 to your computer and use it in GitHub Desktop.
// Lets now assume that a single partition changes. We only need to recompute the state of this
// partition in order to update the metrics for the whole table.
DataFrame updatedUsManufacturers = LoadData(new[]
{
new object[] {3, "ManufacturerDNew", "US"}, new object[] {4, null, "US"},
new object[] {5, "ManufacturerFNew http://clickme.com", "US"},
});
// Recompute state of partition
InMemoryStateProvider updatedUsStates = new InMemoryStateProvider();
AnalysisRunner.Run(updatedUsManufacturers, analysis, updatedUsStates);
// Recompute metrics for whole tables from states. We do not need to touch old data!
AnalyzerContext updatedTableMetrics = AnalysisRunner.RunOnAggregatedStates(dataSetDE.Schema(), analysis,
new[] { deStates, usStates, cnStates });
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment