Introduce two wrapper scripts as wrappers for run-tests.php:
- run-tests.sh
- run-tests-with-failures.sh
The first is straightforward, sets the correct environment variables, and passes on any arguments to run-tests.php.
The second is intended for test groups where there are currently failures, and we want to fail only if there are new regressions. The diff-failures.php helper is a wrapper around diff to re-format the run-tests.php output into an easily diffable format. A unified diff is output, so it's easy to update *-failures.txt files.
Tests are broken down into groups mostly for ease of maintenance.
- basic: tests/
- Zend: Zend/tests/
- ext: ext/*/tests/ (that are included in -common)
- ext-bz2: ext/bz2/tests/ (example of extension in a separate package)
If this system works, then future patches can add the rest of extension tests, and sapi tests.