This is not an article on the theoretical proper way to implement a testing policy and/or infrastructure. This is much more real world than that. This is about finding yourself in a situation were you need to refactor or add features to an existing substantial code base. Before undertaking such an adventure you would like to lay down some tests for regression purposes. The hitch is that the code is in a framework that hasn't put testing support first.
Many PHP frameworks qualify for the statement above but the one we will talk about in this article is Codeigniter. I wont use this article to debate the quality of the Codeigniter code base. It is what it is and finds itself used for a very many (in production) websites. What this article is about is addressing the situation that there are many developers out there that may find themselves working on a product utilizing a framework such as Codeigniter and have the desire to at least use testing for regression of the code they are adding/removing to the product.
- Know where you are Test the current state of the app. This helps you to learn the inner workings of the app and document it for you and others.
- Know where you are going Test Driven Development
- Know where you've been The tests you have written are a Regression Net to catch unforseen bugs as you continue to evolve the code base.
- Reduction in time between release cycles
- Less human QA
- Bugs are caught as soon as they are committed.
This means they are fresh in the mind of the dev thus easier to fix. Also, this immediacy instills a sense of responsibility and ownership of the code to the dev and aspirations to higher quality code. - Tests are documentation
When co workers ask "how a piece of code works", point them to the tests involved in that code. - Tests improve the quality of code.
Since tests are first class clients of the code they test, when writing the tests, deficiencies will be spotted and code can be refactored. TDD drives the creation of fluent, decoupled, cohesive code.
This article isn't about Unit testing in the pure sense. Basically, a unit test is a test of one method in one class. There is nothing stopping you from writing pure unit tests on code that is used with your product (based on Codeigniter) as long as that code under test is fully decoupled from Codeigniter. For example some standalone library. Where trouble ensues is in testing code that is coupled with Codeigniter in some way. This fact alone is not cause for alarm, what is the issue is that Codeigniter is heavily based and reliant on Globals and Singletons. Again, not here to debate the quality of the Codeigniter framework, but many in the programming community regard both Globals and Singletons as anti-patterns and of concern here is that they make testing very difficult, if not impossible.
- http://c2.com/cgi/wiki?GlobalVariablesAreBad
- http://misko.hevery.com/code-reviewers-guide/flaw-brittle-global-state-singletons/
- Use your singletons wisely
- Testing Code That Uses Singletons » Sebastian Bergmann
- It's hard to test code that uses singletons
We have enabled the use of PHPUnit with CodeIgniter but this is not actually unit testing. What we have enabled is what most would call Integration testing.
What we have accomplished is to build the CI world with all its Globals and Singletons. Our tests will use those, in doing so, making them integration tests with the CI environment.
You can certainly write pure unit tests for things such as library code you write to use with CI that is fully decoupled from CI. You can also decouple somewhat your tests from CI by using Mocks and Stubs.
- examples of test errors if you just try to use PHPUnit Lets say we want extend/override Codeigniter's Config library. Before doing so, you want to cover the existing class in a few Unit tests to catch any regression errors as you make your changes. So you start a PHPUnit test
class DateHelperTest extends PHPUnit_Framework_TestCase {
public function setUp()
{
require_once dirname(__FILE__)."/../../system/libraries/Config.php";
}
public function testConfig() {
$this->markTestIncomplete();
}
}
You will get this...
$ phpunit tests_integration/DateHelperTest.php PHPUnit 3.4.15 by Sebastian Bergmann. No direct script access allowed
That is due to these blocks
if ( ! defined('BASEPATH')) exit('No direct script access allowed');
being at the top of Codeigniter classes. So right off the bat, we are learning that practically nothing in Codeigniter is meant to be utilized unless the full build of the env has happened (include index.php, which among many other things will define BASEPATH).
So, we decide to fudge the BASEPATH be defining it in the test's setUp method
! defined('BASEPATH')? define('BASEPATH', 1):null;
and when we continue on we get:
$ phpunit tests_unit/ConfigRegressionTest.php PHPUnit 3.4.15 by Sebastian Bergmann. PHP Fatal error: Call to undefined function get_config() in /x/CodeIgniter_1.7.3/system/libraries/Config.php on line 47
This is due to the fact that get_config() is a global function that is created as part of the inclusion of index.php. This gives more evidence that Codeigniter will be very difficult (if not impossible) to test without bootstrapping the entire Codeigniter environment.
You can continue this path of
- failed, due to missing X
- OK, find and include X
- attempt to rerun test but since many Codeigniter dependencies are buried so deep, as you write new tests you will continue to discover them which will make your test fragile and difficult to write.
So lets take the approach of providing the fully build Codeigniter environments for the tests. By definition these will no longer be Unit tests but they are still of great value as they provide regression detection as we extend and override Codeigniter.
Codeigniter has a primitive built-in testing library. I don't like using these 'roll your own' xunit testing for a couple of reasons.
- I want a test framework that is well supported by Continual Integration servers, like PHPUnit.
- The last thing you want bugs in is your testing framework and there and many more eyeballs on PHPUnit vs. Codeigniter's library.
There are also a couple of existing "testing plugins" for codeigniter but I steered away from them as there own code bases are quite extensive. My goal is to simple 'allow' the use of PHPUnit with Codeigniter and in doing that with to introduce as little additional code as possible.
so, what i will demonstrate here is enabling that build of the codeigniter environment with the least amount of glue code as possible. Thus allowing us to use PHPUnit as needed to test...
main issue to overcome
- When Codeigniter runs, it creates a slue of 'Global' values in its bootstrap file called Codeigniter.php. When that same code runs withing a test method, those values are not true global scope, they are scoped to the test method.
The testing framework needs to know were you keep index.php and Codeigniter.php. You define these in the bootstrap.php file.
Codeigniter.php
We up-global the variable that have load_class assignments so they are in the Global scope of the tests.
index.php
- Point the
requireCodeigniter.php to point to our augmented Codeigniter.php file - Convert the definitions of
$system_pathand$application_folderto be fully resolved
And that is it for the changes to any of the Codeigniter files. They are really just there to provide the same global scope Codeigniter relies on in Production to the tests running Codeigniter.
$ phpunit --bootstrap testing/bootstrap.php testing/tests_unit/ConfigRegressionTest.php PHPUnit 3.4.15 by Sebastian Bergmann. <html> <head> <title>Welcome to CodeIgniter</title> ... <p>If you are exploring CodeIgniter for the very first time, you should start by reading the <a href="user_guide/">User Guide</a>.</p> <p><br />Page rendered in 0.0081 seconds</p> </body> </html>. Time: 0 seconds, Memory: 8.50Mb OK (1 test, 1 assertion)
Making progress, our test passed but Codeigniter, making the assumption it is in a web environment has output the HTML page representing a request to the home ("/") path for this app. For this testing situation the controller requested and it accompanying output is actually irrelevant to the test. We just needed to request 'a' controller to enable the CI environment to be built. All this HTML output doesn't affect our tests passing or failing but it is an enormous amount of noise we need to get rid of. You could go into the BaseTestCase and wrap the inclusion of index.php with ob functions
ob_start();
require dirname(__FILE__) ."/CI_artifacts/".TEST_INDEX_PHP_FILENAME;
ob_clean();
This will capture output from requesting a controller action and then just toss it. The problem is this strategy captures all the output, even when Codeigniter my be trying to tell us something we want to know like errors and warnings. Because of this, I don't recommend this technique. Instead, create a 'dev/null' controller that outputs nothing.
final class Devnull extends Controller {
final public function index()
{
// do nothing
}
}
Then set that path in the bootstrap file
define('TESTING_CONTROLLER_DEFAULT_PATH', '/devnull');
Now if there are any sort of errors with the request to the controller, it will be seen, but devoid of that, there will be no noise output for test runs.
$ phpunit --bootstrap testing/bootstrap.php testing/tests_unit/ConfigRegressionTest.php PHPUnit 3.4.15 by Sebastian Bergmann. . Time: 0 seconds, Memory: 8.50Mb OK (1 test, 1 assertion)
You refer to ConfigRegressionTest.php , but what does this file look like?