Skip to content

Instantly share code, notes, and snippets.

@Pierstoval
Last active April 21, 2020 09:13
Show Gist options
  • Save Pierstoval/b219bec7ad20749102b143f559ef507e to your computer and use it in GitHub Desktop.
Save Pierstoval/b219bec7ad20749102b143f559ef507e to your computer and use it in GitHub Desktop.

Native mocks versus PHPUnit mocks

Mock type Tests duration
Native mocks 160.10 seconds
Mockery 177.01 seconds (+10%)
PHPUnit mocks 232.29 seconds (+45%)
Prophecy 3868.80 seconds (+2416%)

Benchmarked on the exact same Windows machine, with PHP 7.3.7, PHPUnit 8.2.5, inside a Docker container, testing consisting on executing the same test 400k times.

I did not show native classes here, as we only focus on PHPUnit's system.

Benchmarks:

Native anonymous class (base time)

$repo = new class() extends SubscriptionRepository {
    public $executions = [];
    public function __construct()
    {
        // Don't call parent constructor in case it needs dependencies
    }
    public function hasSimilarActiveSubscriptions(Subscription $subscription): bool
    {
        // Only mock methods that have to be used during the test
        $this->executions[] = $subscription;
        return true;
    }
};

// Tests ...

static::assertCount(1, $repo->executions);
static::assertSame($subscription, $repo->executions[0]);

Mockery (10% slower)

$repo = Mockery::mock(SubscriptionRepository::class);
$repo
    ->shouldReceive('hasSimilarActiveSubscriptions')
    ->with($subscription)
    ->once()
    ->andReturn(true)
;

// Tests ...

PHPUnit (45% slower)

$repo = $this->createMock(SubscriptionRepository::class);
$repo->expects(static::once())
    ->method('hasSimilarActiveSubscriptions')
    ->with($subscription)
    ->willReturn(true)
;

// Tests ...

Prophecy (2416% slower)

$repo = $this->prophesize(SubscriptionRepository::class);
$repo
    ->hasSimilarActiveSubscriptions($subscription)
    ->shouldBeCalledOnce()
    ->willReturn(true)
;

// Don't forget that "$repo" must be injected using "$repo->reveal()"

// Tests ...

I think it is worth mentioning that I could only run Prophecy tests with batches of 25k tests instead of 400k. More was leading to segfaults. Therefore, the time is an average (but significantly slower anyway).

@jakzal
Copy link

jakzal commented Aug 14, 2019

Interesting results!

You inspired me to make some blackfire profiles for similar tests (I'm on MacOS).

Repeat 1000

Test double Wall Time Memory Profile
anonymous 2.84s 2.46MB https://blackfire.io/profiles/3ad71640-32ab-4aa0-afda-e1722bcfa04a/graph
mockery 4.62s 6.53MB https://blackfire.io/profiles/082c5427-3ad0-4302-8568-bd4ca766644c/graph
phpunit 5.8s 2.96MB https://blackfire.io/profiles/22c7f37e-0879-4534-853c-d9ae9b673006/graph
prophecy 13s 11MB https://blackfire.io/profiles/9c730704-5773-40ab-bf46-d169d4a0239f/graph

Repeat 100000

Test double Wall Time Memory Profile
anonymous 4min 19s 13.9MB https://blackfire.io/profiles/187d2d70-fba7-4199-9539-d7c1f51f46bb/graph
mockery 6min 46s 14.5MB https://blackfire.io/profiles/026b4fc5-4570-484a-8e6a-2db79cfdd943/graph
phpunit 5min 24s 14.9MB https://blackfire.io/profiles/fc62fec2-b02c-4b7a-8d00-cb65bc77cfc4/graph
prophecy 10min 57s 635MB https://blackfire.io/profiles/c07d697d-4ff5-4c4a-8c93-2ab7da47aaaf/graph

@oleg-andreyev
Copy link

Wow! Did not expect that prophecy is SO slow!

@jakzal
Copy link

jakzal commented Aug 15, 2019

@oleg-andreyev on most projects it doesn't matter. Would be good to get to the bottom of it and improve performance/memory usage though.

@oleg-andreyev
Copy link

@jakzal you are right that in most cases it does not matter, but when you have a large set of tests it matters. As you may know recently Symfony moved from Phpunit mocks to anonymous classes were it was possible and build become a bit faster. It great that @Pierstoval gathered this information now community can help and improve it.

@Pierstoval
Copy link
Author

Well on some projects I have hundreds of tests, and if they run in 5 minutes, I'd be perfectly okay for them to be run in 3 minutes instead by giving them a perf boost just by changing the mocking system.

I'm going to open a PR on a personal project on which I have ~400 tests, change all createMock() calls to native implementations (if possible) and check the performance differences with the PHPUnit's --repeat=... option, but this will be in the future when all tests will be refactored 😛

@Pierstoval
Copy link
Author

And by the way, thanks @jakzal for providing more benchmarks that confirm the initial theory 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment