Playwright 1.47.0 Improved Sharding

_playwright_1.47.0_improved_sharding_patch.md

Playwright 1.47.0 Improved Sharding Patch

This patch includes the changes from microsoft/playwright#30962 (feat(test runner): improve sharding algorithm to better spread similar tests among shards)

Use patch-package to apply it on top of playwright v1.47.0.

Changes

This patch adds a new shardingMode configuration which allows to specify the sharding algorithm to be used…

shardingMode: 'partition'

That's the current behaviour, which is the default. Let me know if you have a better name to describe the current algorithm...

shardingMode: 'round-robin'

Distribute the test groups more evenly. It…

1. sorts test groups by number of tests in descending order
2. then loops through the test groups and assigns them to the shard with the lowest number of tests.

Here is a simple example where every test group represents a single test (e.g. --fully-parallel) ...

         [  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12]
Shard 1:    ^               ^               ^              : [  1, 5, 9 ]
Shard 2:        ^               ^               ^          : [  2, 6,10 ]
Shard 3:            ^               ^               ^      : [  3, 7,11 ]
Shard 4:                ^               ^               ^  : [  4, 8,12 ]

…or a more complex scenario where test groups have different number of tests…

Original Order: [ [1], [2, 3], [4, 5, 6], [7], [8], [9, 10], [11], [12] ]
Sorted Order:   [ [4, 5, 6], [2, 3], [9, 10], [1], [7], [8], [11], [12] ]
Shard 1:           ^-----^                                                : [ [ 4,   5,   6] ]
Shard 2:                      ^--^                       ^                : [ [ 2,  3],  [8] ]
Shard 3:                              ^---^                    ^          : [ [ 9, 10], [11] ]
Shard 4:                                       ^    ^                ^    : [ [1], [7], [12] ]

shardingMode: 'duration-round-robin'

It's very similar to round-robin, but it uses the duration of a tests previous run as cost factor. The duration will be read from .last-run.json when available. When a test can not be found in .last-run.json it will use the average duration of available tests. When no last run info is available, the behaviour would be identical to round-robin.

Other changes

• Add testDurations?: { [testId: string]: number } to .last-run.json
• Add builtin lastrun reporter, which allows merge-reports to generate a .last-run.json to be generated

Appendix

Below are some runtime stats from a project I've been working on, which shows the potential benefit of this change.

The tests runs had to complete 161 tests. Single test duration ranges from a few seconds to over 2 minutes.

The partition run gives the baseline performance and illustrates the problem quite good. We have a single shard that takes almost 16 min while another one completes in under 5 min.

The round-robin algorithm gives a bit better performance, but it still has a shard that requires twice the time of another shard.

The duration-round-robin run was using the duration info from a previous run and achieves the best result by far. All shards complete in 10-11 minutes. 🏆 🎉

playwright+1.47.0.patch

diff --git a/node_modules/playwright/lib/common/config.js b/node_modules/playwright/lib/common/config.js
index b26c4a8..998c313 100644
--- a/node_modules/playwright/lib/common/config.js
+++ b/node_modules/playwright/lib/common/config.js
@@ -47,6 +47,9 @@ class FullConfigInternal {
     this.cliFailOnFlakyTests = void 0;
     this.testIdMatcher = void 0;
     this.defineConfigWasUsed = false;
+    this.shardingMode = void 0;
+    this.lastRunFile = void 0;
+    this.lastRunInfo = void 0;
     if (configCLIOverrides.projects && userConfig.projects) throw new Error(`Cannot use --browser option when configuration file defines projects. Specify browserName in the projects instead.`);
     const {
       resolvedConfigFile,
@@ -86,6 +89,8 @@ class FullConfigInternal {
       workers: 0,
       webServer: null
     };
+    this.shardingMode = takeFirst(configCLIOverrides.shardingMode, userConfig.shardingMode, 'partition');
+    this.lastRunFile = configCLIOverrides.lastRunFile;
     for (const key in userConfig) {
       if (key.startsWith('@')) this.config[key] = userConfig[key];
     }
diff --git a/node_modules/playwright/lib/common/configLoader.js b/node_modules/playwright/lib/common/configLoader.js
index 75fc987..6ed46bd 100644
--- a/node_modules/playwright/lib/common/configLoader.js
+++ b/node_modules/playwright/lib/common/configLoader.js
@@ -272,11 +272,11 @@ async function loadConfigFromFileRestartIfNeeded(configFile, overrides, ignoreDe
   if (restartWithExperimentalTsEsm(location.resolvedConfigFile)) return null;
   return await loadConfig(location, overrides, ignoreDeps);
 }
-async function loadEmptyConfigForMergeReports() {
+async function loadEmptyConfigForMergeReports(overrides) {
   // Merge reports is "different" for no good reason. It should not pick up local config from the cwd.
   return await loadConfig({
     configDir: process.cwd()
-  });
+  }, overrides);
 }
 function restartWithExperimentalTsEsm(configFile, force = false) {
   // Opt-out switch.
diff --git a/node_modules/playwright/lib/program.js b/node_modules/playwright/lib/program.js
index cc41b52..8c99311 100644
--- a/node_modules/playwright/lib/program.js
+++ b/node_modules/playwright/lib/program.js
@@ -155,6 +155,7 @@ function addMergeReportsCommand(program) {
   });
   command.option('-c, --config <file>', `Configuration file. Can be used to specify additional configuration for the output report.`);
   command.option('--reporter <reporter>', `Reporter to use, comma-separated, can be ${_config.builtInReporters.map(name => `"${name}"`).join(', ')} (default: "${_config.defaultReporter}")`);
+  command.option('--last-run-file <file>', `Path to a json file where the last run information is written to (default: test-results/.last-run.json)`);
   command.addHelpText('afterAll', `
 Arguments [dir]:
   Directory containing blob reports.
@@ -202,9 +203,10 @@ async function runTests(args, opts) {
   }
   const config = await (0, _configLoader.loadConfigFromFileRestartIfNeeded)(opts.config, cliOverrides, opts.deps === false);
   if (!config) return;
-  if (opts.lastFailed) {
+  if (opts.lastFailed || config.shardingMode === 'duration-round-robin') {
     const lastRunInfo = await (0, _runner.readLastRunInfo)(config);
-    config.testIdMatcher = id => lastRunInfo.failedTests.includes(id);
+    if (opts.lastFailed) config.testIdMatcher = id => lastRunInfo.failedTests.includes(id);
+    if (config.shardingMode === 'duration-round-robin') config.lastRunInfo = lastRunInfo;
   }
   config.cliArgs = args;
   config.cliGrep = opts.grep;
@@ -260,7 +262,8 @@ async function listTestFiles(opts) {
 }
 async function mergeReports(reportDir, opts) {
   const configFile = opts.config;
-  const config = configFile ? await (0, _configLoader.loadConfigFromFileRestartIfNeeded)(configFile) : await (0, _configLoader.loadEmptyConfigForMergeReports)();
+  const cliOverrides = overridesFromOptions(opts);
+  const config = configFile ? await (0, _configLoader.loadConfigFromFileRestartIfNeeded)(configFile, cliOverrides) : await (0, _configLoader.loadEmptyConfigForMergeReports)(cliOverrides);
   if (!config) return;
   const dir = _path.default.resolve(process.cwd(), reportDir || '');
   const dirStat = await _fs.default.promises.stat(dir).catch(e => null);
@@ -289,6 +292,8 @@ function overridesFromOptions(options) {
       current: shardPair[0],
       total: shardPair[1]
     } : undefined,
+    shardingMode: options.shardingMode ? resolveShardingModeOption(options.shardingMode) : undefined,
+    lastRunFile: options.lastRunFile ? _path.default.resolve(process.cwd(), options.lastRunFile) : undefined,
     timeout: options.timeout ? parseInt(options.timeout, 10) : undefined,
     tsconfig: options.tsconfig ? _path.default.resolve(process.cwd(), options.tsconfig) : undefined,
     ignoreSnapshots: options.ignoreSnapshots ? !!options.ignoreSnapshots : undefined,
@@ -322,6 +327,12 @@ function overridesFromOptions(options) {
   }
   return overrides;
 }
+const shardingModes = ['partition', 'round-robin', 'duration-round-robin'];
+function resolveShardingModeOption(shardingMode) {
+  if (!shardingMode) return undefined;
+  if (!shardingModes.includes(shardingMode)) throw new Error(`Unsupported sharding mode "${shardingMode}", must be one of: ${shardingModes.map(mode => `"${mode}"`).join(', ')}`);
+  return shardingMode;
+}
 function resolveReporterOption(reporter) {
   if (!reporter || !reporter.length) return undefined;
   return reporter.split(',').map(r => [resolveReporter(r)]);
@@ -335,7 +346,7 @@ function resolveReporter(id) {
   });
 }
 const kTraceModes = ['on', 'off', 'on-first-retry', 'on-all-retries', 'retain-on-failure', 'retain-on-first-failure'];
-const testOptions = [['--browser <browser>', `Browser to use for tests, one of "all", "chromium", "firefox" or "webkit" (default: "chromium")`], ['-c, --config <file>', `Configuration file, or a test directory with optional "playwright.config.{m,c}?{js,ts}"`], ['--debug', `Run tests with Playwright Inspector. Shortcut for "PWDEBUG=1" environment variable and "--timeout=0 --max-failures=1 --headed --workers=1" options`], ['--fail-on-flaky-tests', `Fail if any test is flagged as flaky (default: false)`], ['--forbid-only', `Fail if test.only is called (default: false)`], ['--fully-parallel', `Run all tests in parallel (default: false)`], ['--global-timeout <timeout>', `Maximum time this test suite can run in milliseconds (default: unlimited)`], ['-g, --grep <grep>', `Only run tests matching this regular expression (default: ".*")`], ['-gv, --grep-invert <grep>', `Only run tests that do not match this regular expression`], ['--headed', `Run tests in headed browsers (default: headless)`], ['--ignore-snapshots', `Ignore screenshot and snapshot expectations`], ['--last-failed', `Only re-run the failures`], ['--list', `Collect all the tests and report them, but do not run`], ['--max-failures <N>', `Stop after the first N failures`], ['--no-deps', 'Do not run project dependencies'], ['--output <dir>', `Folder for output artifacts (default: "test-results")`], ['--only-changed [ref]', `Only run test files that have been changed between 'HEAD' and 'ref'. Defaults to running all uncommitted changes. Only supports Git.`], ['--pass-with-no-tests', `Makes test run succeed even if no tests were found`], ['--project <project-name...>', `Only run tests from the specified list of projects, supports '*' wildcard (default: run all projects)`], ['--quiet', `Suppress stdio`], ['--repeat-each <N>', `Run each test N times (default: 1)`], ['--reporter <reporter>', `Reporter to use, comma-separated, can be ${_config.builtInReporters.map(name => `"${name}"`).join(', ')} (default: "${_config.defaultReporter}")`], ['--retries <retries>', `Maximum retry count for flaky tests, zero for no retries (default: no retries)`], ['--shard <shard>', `Shard tests and execute only the selected shard, specify in the form "current/all", 1-based, for example "3/5"`], ['--timeout <timeout>', `Specify test timeout threshold in milliseconds, zero for unlimited (default: ${_config.defaultTimeout})`], ['--trace <mode>', `Force tracing mode, can be ${kTraceModes.map(mode => `"${mode}"`).join(', ')}`], ['--tsconfig <path>', `Path to a single tsconfig applicable to all imported files (default: look up tsconfig for each imported file separately)`], ['--ui', `Run tests in interactive UI mode`], ['--ui-host <host>', 'Host to serve UI on; specifying this option opens UI in a browser tab'], ['--ui-port <port>', 'Port to serve UI on, 0 for any free port; specifying this option opens UI in a browser tab'], ['-u, --update-snapshots', `Update snapshots with actual results (default: only create missing snapshots)`], ['-j, --workers <workers>', `Number of concurrent workers or percentage of logical CPU cores, use 1 to run in a single worker (default: 50%)`], ['-x', `Stop after the first failure`]];
+const testOptions = [['--browser <browser>', `Browser to use for tests, one of "all", "chromium", "firefox" or "webkit" (default: "chromium")`], ['-c, --config <file>', `Configuration file, or a test directory with optional "playwright.config.{m,c}?{js,ts}"`], ['--debug', `Run tests with Playwright Inspector. Shortcut for "PWDEBUG=1" environment variable and "--timeout=0 --max-failures=1 --headed --workers=1" options`], ['--fail-on-flaky-tests', `Fail if any test is flagged as flaky (default: false)`], ['--forbid-only', `Fail if test.only is called (default: false)`], ['--fully-parallel', `Run all tests in parallel (default: false)`], ['--global-timeout <timeout>', `Maximum time this test suite can run in milliseconds (default: unlimited)`], ['-g, --grep <grep>', `Only run tests matching this regular expression (default: ".*")`], ['-gv, --grep-invert <grep>', `Only run tests that do not match this regular expression`], ['--headed', `Run tests in headed browsers (default: headless)`], ['--ignore-snapshots', `Ignore screenshot and snapshot expectations`], ['--last-failed', `Only re-run the failures`], ['--last-run-file <file>', `Path to a json file where the last run information is read from and written to (default: test-results/.last-run.json)`], ['--list', `Collect all the tests and report them, but do not run`], ['--max-failures <N>', `Stop after the first N failures`], ['--no-deps', 'Do not run project dependencies'], ['--output <dir>', `Folder for output artifacts (default: "test-results")`], ['--only-changed [ref]', `Only run test files that have been changed between 'HEAD' and 'ref'. Defaults to running all uncommitted changes. Only supports Git.`], ['--pass-with-no-tests', `Makes test run succeed even if no tests were found`], ['--project <project-name...>', `Only run tests from the specified list of projects, supports '*' wildcard (default: run all projects)`], ['--quiet', `Suppress stdio`], ['--repeat-each <N>', `Run each test N times (default: 1)`], ['--reporter <reporter>', `Reporter to use, comma-separated, can be ${_config.builtInReporters.map(name => `"${name}"`).join(', ')} (default: "${_config.defaultReporter}")`], ['--retries <retries>', `Maximum retry count for flaky tests, zero for no retries (default: no retries)`], ['--shard <shard>', `Shard tests and execute only the selected shard, specify in the form "current/all", 1-based, for example "3/5"`], ['--sharding-mode <mode>', `Sharding algorithm to use; "partition", "round-robin" or "duration-round-robin". Defaults to "partition".`], ['--timeout <timeout>', `Specify test timeout threshold in milliseconds, zero for unlimited (default: ${_config.defaultTimeout})`], ['--trace <mode>', `Force tracing mode, can be ${kTraceModes.map(mode => `"${mode}"`).join(', ')}`], ['--tsconfig <path>', `Path to a single tsconfig applicable to all imported files (default: look up tsconfig for each imported file separately)`], ['--ui', `Run tests in interactive UI mode`], ['--ui-host <host>', 'Host to serve UI on; specifying this option opens UI in a browser tab'], ['--ui-port <port>', 'Port to serve UI on, 0 for any free port; specifying this option opens UI in a browser tab'], ['-u, --update-snapshots', `Update snapshots with actual results (default: only create missing snapshots)`], ['-j, --workers <workers>', `Number of concurrent workers or percentage of logical CPU cores, use 1 to run in a single worker (default: 50%)`], ['-x', `Stop after the first failure`]];
 addTestCommand(_program.program);
 addShowReportCommand(_program.program);
 addListFilesCommand(_program.program);
diff --git a/node_modules/playwright/lib/reporters/lastrun.js b/node_modules/playwright/lib/reporters/lastrun.js
new file mode 100644
index 0000000..f5685ea
--- /dev/null
+++ b/node_modules/playwright/lib/reporters/lastrun.js
@@ -0,0 +1,67 @@
+"use strict";
+
+Object.defineProperty(exports, "__esModule", {
+  value: true
+});
+exports.default = void 0;
+var _fs = _interopRequireDefault(require("fs"));
+var _path = _interopRequireDefault(require("path"));
+var _base = require("./base");
+function _interopRequireDefault(obj) { return obj && obj.__esModule ? obj : { default: obj }; }
+/**
+ * Copyright (c) Microsoft Corporation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+class LastRunReporter extends _base.BaseReporter {
+  constructor(options) {
+    var _resolveOutputFile;
+    super();
+    this.lastRun = {
+      failedTests: [],
+      status: 'passed',
+      testDurations: {}
+    };
+    this.resolvedOutputFile = void 0;
+    this.resolvedOutputFile = (_resolveOutputFile = (0, _base.resolveOutputFile)('LASTRUN', {
+      fileName: '.last-run.json',
+      ...options
+    })) === null || _resolveOutputFile === void 0 ? void 0 : _resolveOutputFile.outputFile;
+  }
+  printsToStdio() {
+    return !this.resolvedOutputFile;
+  }
+  onTestEnd(test, result) {
+    super.onTestEnd(test, result);
+    this.lastRun.testDurations[test.id] = result.duration;
+    if (result.status === 'failed') this.lastRun.failedTests.push(test.id);
+  }
+  async onEnd(result) {
+    await super.onEnd(result);
+    this.lastRun.status = result.status;
+    await this.outputReport(this.lastRun, this.resolvedOutputFile);
+  }
+  async outputReport(lastRun, resolvedOutputFile) {
+    const reportString = JSON.stringify(lastRun, undefined, 2);
+    if (resolvedOutputFile) {
+      await _fs.default.promises.mkdir(_path.default.dirname(resolvedOutputFile), {
+        recursive: true
+      });
+      await _fs.default.promises.writeFile(resolvedOutputFile, reportString);
+    } else {
+      console.log(reportString);
+    }
+  }
+}
+var _default = exports.default = LastRunReporter;
\ No newline at end of file
diff --git a/node_modules/playwright/lib/runner/loadUtils.js b/node_modules/playwright/lib/runner/loadUtils.js
index 8404253..ba1fa6e 100644
--- a/node_modules/playwright/lib/runner/loadUtils.js
+++ b/node_modules/playwright/lib/runner/loadUtils.js
@@ -182,7 +182,7 @@ async function createRootSuite(testRun, errors, shouldFilterOnly, additionalFile
     for (const projectSuite of rootSuite.suites) testGroups.push(...(0, _testGroups.createTestGroups)(projectSuite, config.config.workers));
 
     // Shard test groups.
-    const testGroupsInThisShard = (0, _testGroups.filterForShard)(config.config.shard, testGroups);
+    const testGroupsInThisShard = (0, _testGroups.filterForShard)(config.shardingMode, config.config.shard, testGroups, config.lastRunInfo);
     const testsInThisShard = new Set();
     for (const group of testGroupsInThisShard) {
       for (const test of group.tests) testsInThisShard.add(test);
diff --git a/node_modules/playwright/lib/runner/reporters.js b/node_modules/playwright/lib/runner/reporters.js
index a0d69f3..109bffa 100644
--- a/node_modules/playwright/lib/runner/reporters.js
+++ b/node_modules/playwright/lib/runner/reporters.js
@@ -14,6 +14,7 @@ var _github = _interopRequireDefault(require("../reporters/github"));
 var _html = _interopRequireDefault(require("../reporters/html"));
 var _json = _interopRequireDefault(require("../reporters/json"));
 var _junit = _interopRequireDefault(require("../reporters/junit"));
+var _lastrun = _interopRequireDefault(require("../reporters/lastrun"));
 var _line = _interopRequireDefault(require("../reporters/line"));
 var _list = _interopRequireDefault(require("../reporters/list"));
 var _markdown = _interopRequireDefault(require("../reporters/markdown"));
@@ -81,6 +82,15 @@ async function createReporters(config, mode, isTestServer, descriptions) {
       omitFailures: true
     }) : new _dot.default());
   }
+  if (!isTestServer && (mode === 'test' || mode === 'merge')) {
+    var _config$lastRunFile;
+    // If we are not in the test server, always add a last-run reporter.
+    const lastRunOutputFile = ((_config$lastRunFile = config.lastRunFile) !== null && _config$lastRunFile !== void 0 ? _config$lastRunFile : config.projects.length >= 0) ? _path.default.join(config.projects[0].project.outputDir, '.last-run.json') : undefined;
+    reporters.push(new _lastrun.default({
+      ...runOptions,
+      outputFile: lastRunOutputFile
+    }));
+  }
   return reporters;
 }
 async function createReporterForTestServer(file, messageSink) {
diff --git a/node_modules/playwright/lib/runner/runner.js b/node_modules/playwright/lib/runner/runner.js
index bcad21e..11e38ea 100644
--- a/node_modules/playwright/lib/runner/runner.js
+++ b/node_modules/playwright/lib/runner/runner.js
@@ -78,7 +78,6 @@ class Runner {
       status
     });
     if (modifiedResult && modifiedResult.status) status = modifiedResult.status;
-    if (!listOnly) await writeLastRunInfo(testRun, status);
     await reporter.onExit();
 
     // Calling process.exit() might truncate large stdout/stderr output.
@@ -145,22 +144,6 @@ class Runner {
   }
 }
 exports.Runner = Runner;
-async function writeLastRunInfo(testRun, status) {
-  var _testRun$rootSuite;
-  const [project] = (0, _projectUtils.filterProjects)(testRun.config.projects, testRun.config.cliProjectFilter);
-  if (!project) return;
-  const outputDir = project.project.outputDir;
-  await _fs.default.promises.mkdir(outputDir, {
-    recursive: true
-  });
-  const lastRunReportFile = _path.default.join(outputDir, '.last-run.json');
-  const failedTests = (_testRun$rootSuite = testRun.rootSuite) === null || _testRun$rootSuite === void 0 ? void 0 : _testRun$rootSuite.allTests().filter(t => !t.ok()).map(t => t.id);
-  const lastRunReport = JSON.stringify({
-    status,
-    failedTests
-  }, undefined, 2);
-  await _fs.default.promises.writeFile(lastRunReportFile, lastRunReport);
-}
 async function readLastRunInfo(config) {
   const [project] = (0, _projectUtils.filterProjects)(config.projects, config.cliProjectFilter);
   if (!project) return {
@@ -169,7 +152,7 @@ async function readLastRunInfo(config) {
   };
   const outputDir = project.project.outputDir;
   try {
-    const lastRunReportFile = _path.default.join(outputDir, '.last-run.json');
+    const lastRunReportFile = config.lastRunFile || _path.default.join(outputDir, '.last-run.json');
     return JSON.parse(await _fs.default.promises.readFile(lastRunReportFile, 'utf8'));
   } catch {}
   return {
diff --git a/node_modules/playwright/lib/runner/testGroups.js b/node_modules/playwright/lib/runner/testGroups.js
index 952aafe..c5d8dcf 100644
--- a/node_modules/playwright/lib/runner/testGroups.js
+++ b/node_modules/playwright/lib/runner/testGroups.js
@@ -106,7 +106,7 @@ function createTestGroups(projectSuite, workers) {
   }
   return result;
 }
-function filterForShard(shard, testGroups) {
+function filterForShard(mode, shard, testGroups, lastRunInfo) {
   // Note that sharding works based on test groups.
   // This means parallel files will be sharded by single tests,
   // while non-parallel files will be sharded by the whole file.
@@ -114,6 +114,23 @@ function filterForShard(shard, testGroups) {
   // Shards are still balanced by the number of tests, not files,
   // even in the case of non-paralleled files.
 
+  if (mode === 'round-robin') return filterForShardRoundRobin(shard, testGroups);
+  if (mode === 'duration-round-robin') return filterForShardRoundRobin(shard, testGroups, lastRunInfo);
+  return filterForShardPartition(shard, testGroups);
+}
+
+/**
+ * Shards tests by partitioning them into equal parts.
+ *
+ * ```
+ *        [  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12]
+ * Shard 1:  ^---------^                                      : [  1, 2, 3 ]
+ * Shard 2:              ^---------^                          : [  4, 5, 6 ]
+ * Shard 3:                          ^---------^              : [  7, 8, 9 ]
+ * Shard 4:                                      ^---------^  : [ 10,11,12 ]
+ * ```
+ */
+function filterForShardPartition(shard, testGroups) {
   let shardableTotal = 0;
   for (const group of testGroups) shardableTotal += group.tests.length;
 
@@ -134,3 +151,41 @@ function filterForShard(shard, testGroups) {
   }
   return result;
 }
+
+/**
+ * Shards tests by round-robin.
+ *
+ * ```
+ *          [  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12]
+ * Shard 1:    ^               ^               ^              : [  1, 5, 9 ]
+ * Shard 2:        ^               ^               ^          : [  2, 6,10 ]
+ * Shard 3:            ^               ^               ^      : [  3, 7,11 ]
+ * Shard 4:                ^               ^               ^  : [  4, 8,12 ]
+ * ```
+ */
+function filterForShardRoundRobin(shard, testGroups, lastRunInfo) {
+  const weights = new Array(shard.total).fill(0);
+  const shardSet = new Array(shard.total).fill(0).map(() => new Set());
+  const averageDuration = lastRunInfo ? Object.values((lastRunInfo === null || lastRunInfo === void 0 ? void 0 : lastRunInfo.testDurations) || {}).reduce((a, b) => a + b, 1) / Math.max(1, Object.values((lastRunInfo === null || lastRunInfo === void 0 ? void 0 : lastRunInfo.testDurations) || {}).length) : 0;
+  const weight = group => {
+    if (!lastRunInfo)
+      // If we don't have last run info, we just count the number of tests.
+      return group.tests.length;
+    // If we have last run info, we use the duration of the tests.
+    return group.tests.reduce((sum, test) => {
+      var _lastRunInfo$testDura;
+      return sum + Math.max(1, ((_lastRunInfo$testDura = lastRunInfo.testDurations) === null || _lastRunInfo$testDura === void 0 ? void 0 : _lastRunInfo$testDura[test.id]) || averageDuration);
+    }, 0);
+  };
+
+  // We sort the test groups by group duration in descending order.
+  const sortedTestGroups = testGroups.slice().sort((a, b) => weight(b) - weight(a));
+
+  // Then we add each group to the shard with the smallest number of tests.
+  for (const group of sortedTestGroups) {
+    const index = weights.reduce((minIndex, currentLength, currentIndex) => currentLength < weights[minIndex] ? currentIndex : minIndex, 0);
+    weights[index] += weight(group);
+    shardSet[index].add(group);
+  }
+  return shardSet[shard.current - 1];
+}
diff --git a/node_modules/playwright/types/test.d.ts b/node_modules/playwright/types/test.d.ts
index 21bd59a..50a73cd 100644
--- a/node_modules/playwright/types/test.d.ts
+++ b/node_modules/playwright/types/test.d.ts
@@ -1400,7 +1400,7 @@ interface TestConfig<TestArgs = {}, WorkerArgs = {}> {
   /**
    * Shard tests and execute only the selected shard. Specify in the one-based form like `{ total: 5, current: 2 }`.
    *
-   * Learn more about [parallelism and sharding](https://playwright.dev/docs/test-parallel) with Playwright Test.
+   * Learn more about [parallelism](https://playwright.dev/docs/test-parallel) and [sharding](https://playwright.dev/docs/test-sharding) with Playwright Test.
    *
    * **Usage**
    *
@@ -1426,6 +1426,21 @@ interface TestConfig<TestArgs = {}, WorkerArgs = {}> {
     total: number;
   };
 
+  /**
+   * Defines the algorithm to be used for sharding. Defaults to `'partition'`.
+   * - `'partition'` - divide the set of test groups by number of shards. e.g. first half goes to shard 1/2 and
+   *   seconds half to shard 2/2.
+   * - `'round-robin'` - spread test groups to shards in a round-robin way. e.g. loop over test groups and always
+   *   assign to the shard that has the lowest number of tests.
+   * - `'duration-round-robin'` - use duration info from `.last-run.json` to spread test groups to shards in a
+   *   round-robin way. e.g. loop over test groups and always assign to the shard that has the lowest duration of
+   *   tests. new tests which were not present in the last run will use an average duration time. When no
+   *   `.last-run.json` could be found the behavior is identical to `'round-robin'`.
+   *
+   * Learn more about [sharding](https://playwright.dev/docs/test-sharding) with Playwright Test.
+   */
+  shardingMode?: "partition"|"round-robin"|"duration-round-robin";
+
   /**
    * **NOTE** Use
    * [testConfig.snapshotPathTemplate](https://playwright.dev/docs/api/class-testconfig#test-config-snapshot-path-template)
@@ -5103,6 +5118,7 @@ export interface PlaywrightWorkerOptions {
   video: VideoMode | /** deprecated */ 'retry-with-video' | { mode: VideoMode, size?: ViewportSize };
 }
 
+export type ShardingMode = Exclude<PlaywrightTestConfig['shardingMode'], undefined>;
 export type ScreenshotMode = 'off' | 'on' | 'only-on-failure';
 export type TraceMode = 'off' | 'on' | 'retain-on-failure' | 'on-first-retry' | 'on-all-retries' | 'retain-on-first-failure';
 export type VideoMode = 'off' | 'on' | 'retain-on-failure' | 'on-first-retry';