2. Writing Tests for PHPUnit

Asserting return values

A test usually follows the “Arrange, Act, Assert” structure:

• Arrange: Set up all necessary preconditions and inputs

• Act: Perform the action under test

• Assert: Verify that the expected results have occurred

Let us look at testCanBeCreatedFromValidEmail() from the example shown above through the lens of “Arrange, Act, Assert”:

public function testCanBeCreatedFromValidEmail(): void
{
    // Arrange
    $string = 'user@example.org';

    // Act
    $email = Email::fromString($string);

    // Assert
    $this->assertSame($string, $email->asString());
}

The Arrange phase sets up the input: the string 'user@example.org'.

The Act phase calls Email::fromString() to create an Email object from this string.

The Assert phase uses assertSame() to verify that the Email object’s string representation matches the original input.

Asserting return values like this is the most common operation in a test method. See Assertions for the full list of assertion methods that PHPUnit provides.

Ideally, each test method should only verify one aspect of the system under test. In testCanBeCreatedFromValidEmail(), however, we technically test two things: that an Email object can be constructed from a valid string (the named constructor does not throw an exception and we do get an Email object) and that asString() returns the string representation of the Email value object. This is a typical deviation from the norm when testing the happy path of a constructor.

The happy path is the scenario where everything works as expected: valid input is provided and the system behaves correctly. Its opposite is the unhappy path (also called the error path): the scenario where something goes wrong, for instance when invalid input is provided and an exception is expected. Testing the happy path of a constructor often involves verifying both that the object is created successfully and that it holds the expected state.

Expecting exceptions

When you expect an action to raise an exception, the test follows the “Arrange, Expect, Act” structure. The assertion (expectation) is set up before the action that is expected to throw the exception.

Let us look at testCannotBeCreatedFromInvalidEmail() from the example shown above through the lens of “Arrange, Expect, Act”:

public function testCannotBeCreatedFromInvalidEmail(): void
{
    // Expect
    $this->expectException(InvalidArgumentException::class);

    // Act
    Email::fromString('invalid');
}

The Expect phase tells PHPUnit that the code is expected to throw an InvalidArgumentException.

The Act phase calls Email::fromString() with an invalid email address. If the expected exception is not thrown, the test fails.

The expectException() method must be called before the code that is expected to throw the exception. This is why the structure is “Arrange, Expect, Act” and not “Arrange, Act, Assert”.

In addition to the expectException() method, the expectExceptionCode(), expectExceptionMessage(), and expectExceptionMessageMatches() methods exist to set up expectations for exceptions raised by the code under test.

Verifying side effects

Not everything that needs testing is a return value or an exception. A method may have side effects: it may call a method on a collaborating object or perform an I/O operation such as writing to a database or to the filesystem.

Sometimes the system under test depends on values returned by a collaborating object. Test Stubs allow you to control these indirect inputs.

Sometimes you need to verify that the system under test calls a method on a collaborating object with the expected arguments. Mock Objects allow you to verify these indirect outputs.

Test Stubs and Mock Objects are discussed in detail in the chapter on Test Doubles.

Data Providers

A test method can accept arbitrary arguments. These arguments are to be provided by one or more data provider methods (additionProvider() in the example shown below). The data provider method to be used is specified using the PHPUnit\Framework\Attributes\DataProvider attribute.

A data provider method must be public and static and its name must not start with test. It must return a value that is iterable, either an array or an object that implements the Traversable interface. In each iteration step, it must yield an array. For each of these arrays, the test method will be called with the contents of the array as its arguments.

<?php declare(strict_types=1);
use PHPUnit\Framework\Attributes\DataProvider;
use PHPUnit\Framework\TestCase;

final class NumericDataSetsTest extends TestCase
{
    public static function additionProvider(): array
    {
        return [
            [0, 0, 0],
            [0, 1, 1],
            [1, 0, 1],
            [1, 1, 3],
        ];
    }

    #[DataProvider('additionProvider')]
    public function testAdd(int $a, int $b, int $expected): void
    {
        $this->assertSame($expected, $a + $b);
    }
}

Running the test shown above yields the output shown below:

./tools/phpunit tests/NumericDataSetsTest.php
PHPUnit 12.5.2 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.5.4

...F                                                                4 / 4 (100%)

Time: 00:00.001, Memory: 25.89 MB

There was 1 failure:

1) NumericDataSetsTest::testAdd#3 with data (1, 1, 3)
Failed asserting that 2 is identical to 3.

/path/to/tests/NumericDataSetsTest.php:20

FAILURES!
Tests: 4, Assertions: 4, Failures: 1.

It is useful to name each data set with a string key. Output will be more verbose as it will contain the name of the data set that breaks a test.

<?php declare(strict_types=1);
use PHPUnit\Framework\Attributes\DataProvider;
use PHPUnit\Framework\TestCase;

final class NamedDataSetsTest extends TestCase
{
    public static function additionProvider(): array
    {
        return [
            'adding zeros'  => [0, 0, 0],
            'zero plus one' => [0, 1, 1],
            'one plus zero' => [1, 0, 1],
            'one plus one'  => [1, 1, 3],
        ];
    }

    #[DataProvider('additionProvider')]
    public function testAdd(int $a, int $b, int $expected): void
    {
        $this->assertSame($expected, $a + $b);
    }
}

Running the test shown above yields the output shown below:

./tools/phpunit tests/NamedDataSetsTest.php
PHPUnit 12.5.2 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.5.4

...F                                                                4 / 4 (100%)

Time: 00:00.001, Memory: 25.89 MB

There was 1 failure:

1) NamedDataSetsTest::testAdd@one plus one with data (1, 1, 3)
Failed asserting that 2 is identical to 3.

/path/to/tests/NamedDataSetsTest.php:20

FAILURES!
Tests: 4, Assertions: 4, Failures: 1.

Individual data sets can be selected from the command line via the --filter option; see Filtering by test name for the syntax.

When a test receives input from both a data provider method and from one or more tests it depends on, the arguments from the data provider will come before the ones from depended-upon tests. The arguments from depended-upon tests will be the same for each data set.

When a test depends on a test that uses data providers, the depending test will be executed when the test it depends upon is successful for at least one data set. The result of a test that uses data providers cannot be injected into a depending test.

All data providers are executed before both the call to a before-class method such as setUpBeforeClass() and the first call to a before-test method such as setUp(). Because of this, you cannot access any properties of the actual test case object within a data provider. This also means that no code coverage data is collected while data provider methods are executed.

The data sets provided by a data provider method should only contain (arrays of) scalar values, immutable value objects, or test stubs. Services or large object graphs should not be created in a data provider method. Mock objects cannot be created in a data provider method.

Test Dependencies

PHPUnit supports the declaration of explicit dependencies between test methods. Such dependencies do not define the order in which the test methods are to be executed but they allow the returning of an instance of the test fixture by a producer and passing it to the dependent consumers.

• A producer is a test method that yields its unit under test as return value.

• A consumer is a test method that depends on one or more producers and their return values.

This example shows how to use the PHPUnit\Framework\Attributes\Depends attribute to express dependencies between test methods:

<?php declare(strict_types=1);
use PHPUnit\Framework\Attributes\Depends;
use PHPUnit\Framework\TestCase;

final class StackTest extends TestCase
{
    public function testEmpty(): array
    {
        $stack = [];
        $this->assertEmpty($stack);

        return $stack;
    }

    #[Depends('testEmpty')]
    public function testPush(array $stack): array
    {
        $stack[] = 'foo';
        $this->assertSame('foo', $stack[count($stack) - 1]);
        $this->assertNotEmpty($stack);

        return $stack;
    }

    #[Depends('testPush')]
    public function testPop(array $stack): void
    {
        $this->assertSame('foo', array_pop($stack));
        $this->assertEmpty($stack);
    }
}

Running the test shown above yields the output shown below:

./tools/phpunit tests/StackTest.php
PHPUnit 12.5.2 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.5.4

...                                                                 3 / 3 (100%)

Time: 00:00.001, Memory: 25.89 MB

OK (3 tests, 5 assertions)

In the example above, the first test, testEmpty(), creates a new array and asserts that it is empty. The test then returns the fixture as its result. The second test, testPush(), depends on testEmpty() and is passed the result of that depended-upon test as its argument. Finally, testPop() depends upon testPush().

To localize defects, we want our attention to be focussed on relevant failing tests. This is why PHPUnit skips the execution of a test when a depended-upon test has failed. This improves defect localization by exploiting the dependencies between tests as shown in Example 2.6.

<?php declare(strict_types=1);
use PHPUnit\Framework\Attributes\Depends;
use PHPUnit\Framework\TestCase;

final class DependencyFailureTest extends TestCase
{
    public function testOne(): void
    {
        $this->assertTrue(false);
    }

    #[Depends('testOne')]
    public function testTwo(): void
    {
    }
}

Running the test shown above yields the output shown below:

./tools/phpunit --display-skipped tests/DependencyFailureTest.php
PHPUnit 12.5.2 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.5.4

FS                                                                  2 / 2 (100%)

Time: 00:00, Memory: 25.89 MB

There was 1 failure:

1) DependencyFailureTest::testOne
Failed asserting that false is true.

/path/to/tests/DependencyFailureTest.php:9

--

There was 1 skipped test:

1) DependencyFailureTest::testTwo
This test depends on "DependencyFailureTest::testOne" to pass

FAILURES!
Tests: 2, Assertions: 1, Failures: 1, Skipped: 1.

A test may have more than one test dependency attribute.

By default, PHPUnit does not change the order in which tests are executed, so you have to ensure that the dependencies of a test can actually be met before the test is run.

A test that has more than one test dependency attribute will get a fixture from the first producer as the first argument, a fixture from the second producer as the second argument, and so on.

Incomplete Tests

When you are working on a new test case class, you might want to begin by writing empty test methods such as:

public function testSomething(): void
{
}

to keep track of the tests that you have to write.

The problem with empty test methods is that they cannot fail and may be misinterpreted as a success. This misinterpretation leads to the test reports being useless: you cannot see whether a test is actually successful or just not implemented yet.

Calling $this->assertTrue(false), for instance, in the unfinished test method does not help either, since then the test will be interpreted as a failure. This would be just as wrong as interpreting an unimplemented test as a success.

If we think of a successful test as a green light and a test failure as a red light, then we need an additional yellow light to mark a test as being incomplete or not yet implemented.

By calling the method markTestIncomplete() in a test method, we can mark the test as incomplete:

<?php declare(strict_types=1);
use PHPUnit\Framework\TestCase;

final class WorkInProgressTest extends TestCase
{
    public function testSomething(): void
    {
        // Optional: Test anything here, if you want.
        $this->assertTrue(true, 'This should already work.');

        // Stop here and mark this test as incomplete.
        $this->markTestIncomplete(
            'This test has not been implemented yet.',
        );
    }
}

An incomplete test is denoted by an I in the output of the PHPUnit command-line test runner, as shown in the following example:

./tools/phpunit --display-incomplete tests/WorkInProgressTest.php
PHPUnit 12.5.2 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.5.4

I                                                                   1 / 1 (100%)

Time: 00:00, Memory: 25.89 MB

There was 1 incomplete test:

1) WorkInProgressTest::testSomething
This test has not been implemented yet.

/path/to/tests/WorkInProgressTest.php:12

OK, but there were issues!
Tests: 1, Assertions: 1, Incomplete: 1.

Skipping tests

Not all tests can be run in every environment. Consider, for instance, a database abstraction layer that has several drivers for the different database systems it supports. The tests for the PostgreSQL driver can only be run if a PostgreSQL server is available.

Example 2.8 shows a test case class, DatabaseTest, that contains one test method, testConnection(). In the test case class’ setUp() template method we check whether the pgsql extension is available and use the markTestSkipped() method to skip the test if it is not.

<?php declare(strict_types=1);
use PHPUnit\Framework\TestCase;

final class DatabaseTest extends TestCase
{
    protected function setUp(): void
    {
        if (!extension_loaded('pgsql')) {
            $this->markTestSkipped(
                'The PostgreSQL extension is not available',
            );
        }
    }

    public function testConnection(): void
    {
        // ...
    }
}

A test that has been skipped is denoted by an S in the output of the PHPUnit command-line test runner, as shown in the following example:

./tools/phpunit --display-skipped tests/DatabaseTest.php
PHPUnit 12.5.2 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.5.4

S                                                                   1 / 1 (100%)

Time: 00:00, Memory: 25.89 MB

There was 1 skipped test:

1) DatabaseTest::testConnection
The PostgreSQL extension is not available

OK, but some tests were skipped!
Tests: 1, Assertions: 0, Skipped: 1.

<?php declare(strict_types=1);
use PHPUnit\Framework\Attributes\RequiresPhpExtension;
use PHPUnit\Framework\TestCase;

#[RequiresPhpExtension('pgsql')]
final class DatabaseTest extends TestCase
{
    public function testConnection(): void
    {
        // ...
    }
}

Testing output

Sometimes you want to assert that the execution of a method, for instance, generates an expected output (via echo or print, for example). PHPUnit uses PHP’s Output Buffering feature to provide the functionality that is necessary for this.

Example 2.10 shows how to use the expectOutputString() method to set the expected output. If this expected output is not generated, the test will be counted as a failure.

<?php declare(strict_types=1);
use PHPUnit\Framework\TestCase;

final class OutputTest extends TestCase
{
    public function testExpectFooActualFoo(): void
    {
        $this->expectOutputString('foo');

        print 'foo';
    }

    public function testExpectBarActualBaz(): void
    {
        $this->expectOutputString('bar');

        print 'baz';
    }
}

Running the test shown above yields the output shown below:

./tools/phpunit tests/OutputTest.php
PHPUnit 12.5.2 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.5.4

.F                                                                  2 / 2 (100%)

Time: 00:00, Memory: 25.89 MB

There was 1 failure:

1) OutputTest::testExpectBarActualBaz
Failed asserting that two strings are identical.
--- Expected
+++ Actual
@@ @@
-'bar'
+'baz'

FAILURES!
Tests: 2, Assertions: 2, Failures: 1.

The expectOutputRegex() method can be used instead of expectOutputString() when you want to match the expected output against a regular expression:

$this->expectOutputRegex('/foo .+ bar/');

If the generated output does not match the regular expression, the test will be counted as a failure.

Testing error log output

Sometimes you want to assert that the code under test calls PHP’s error_log() function. PHPUnit captures error_log() output during test execution.

The expectErrorLog() method can be used to expect that error_log() is called at least once during the test. If error_log() is not called, the test will be counted as a failure.

<?php declare(strict_types=1);
use PHPUnit\Framework\TestCase;

final class ErrorLogTest extends TestCase
{
    public function testSomethingIsLogged(): void
    {
        // Code under test that calls error_log()
        error_log('something happened');

        $this->expectErrorLog();
    }
}

When expectErrorLog() is not used and the code under test calls error_log(), the logged output is printed as part of the test output (with date prefixes stripped).