Mapping of database storage and its relations to in-memory objects to ease processing can be seen as challenging and noble task, but its the vietnam of CS, a holywar topic.

The good parts

Let me mention the good parts of ORMs that I like, so that you don't think that I dislike them blindly

1. Easy CRUD​ operations and thus code is less polluted with SQL
   
2. Validation of write operations, escaping
3. Generated objects as results. Useful because of strict schema definition, autocomplete, json generation.
4. Methods like save(), delete() are more specific than plain query() interface
5. Filtering can be composed with strict blocks using criteria API (that is more modular) than with SQL string concatenation

The bad parts

Abstraction should be easier than SQL, but its not. It leaks

ORM as a concept is flawed because of inadequate and leaky abstraction. This can be noticed in many details you need to learn and research, which makes up a very steep learning curve for almost any ORM engine.

1. Transaction as a concept is not native to single threaded in-memory process. Its an alien from relational world. You still need to either explicitly state in code that you're starting transaction and add try-catch-retry part, or you must always use transaction per request which usually locks too much. Another problem is that you can't control order of queries, which make it very easy to encounter a deadlock in concurrent requests. So ORM does not abstract the concept of transactions for you

2. Associations in DB are very general and only deal with count – 1:n, n:m. But in process/memory world, it must be  resolved to something more specific – inheritance, incapsulation, reference. It also affects when should linked rows be loaded. For abstraction layer, thats too much details that need to be configured. Usually it is solved by tying object declarations between each other  – Doctrine uses Discriminator Column Annotation, Bookshelf uses association bindings.

3. Equality. The semantics of equal function may depend on where data came from. What should happen if you compare an object that came from DB and a plain object constructed natively if all fields are the same? It depends. In DB you can have multiple rows with same data entries and you can specify what UNIQUE index means. In memory world you can compare by reference or by value, but in reality your model needs context where data came from, so that it can decide if its the same data or not

When writing protractor UI tests, I wanted to capture all console.log and console.errors to get an early glimpse of possible bugs that occur underneath visible functionality. 

Official protractor 3 spec HOWTO's clearly specify to use built-in methods. The problem is - they don't work. At least in Firefox which has most stable driver so far. They do spit out over 900 useless logs related to css, graphics and fonts, but not what I really need.

browser.manage().logs().get('browser').then(function(browserLog) {
  console.log(browserLog);
});

Lately I have been working on integration (API) tests and I like it. I run half of the application, but I'm not tied to UI that might change so often. Its a golden ratio between slow end-to-end tests and very quick but isolated unit tests. Lets see a special type of such tests, which use prepared data for each one of them. 

Tests like these are required when the project grows so big that running tests with one DB can result in unstable tests and conflicts. Somewhere you might get gradual increase in list length, somewhere you want a fixed ID but you have autoincrement in place, or you might want to delete something. 

Its especially noticeable in e2e tests where you are forced to handle entire data lifecycle for tests to remain in working condition. Controlling lifecycle of creation-changes-deletion, forces tests to be dependent on each other, which means you can't run test separately. I had an experience with testing when I wanted to login with custom user role, but there was no prepared test users in DB and there was no user creation in UI either (from adminpanel side). How do you test that, but with generated test data from different users not breaking tests?

Example

use kurapov\tests\database\IsolatedDataIntegrationTestBase;

class UserIsolatedDataTest extends IsolatedDataIntegrationTestBase {
    /**
     * @test
     */
    function postRemove_UserByManager() {
        //$this->db->execute(file_get_contents(dirname(realpath(__FILE__)) . '/' . __CLASS__ . '/' . __FUNCTION__ . '.sql'));
        
        $this->db->execute(
"INSERT INTO `user` (`id`, `email`, `password`) VALUES (1,'manager@kurapov.ee','553ae7da92f5505a92bbb8c9d47be76ab9f65bc2');
INSERT INTO `user` (`id`, `email`, `password`) VALUES (2,'user@kurapov.ee','f4542db9ba30f7958ae42c113dd87ad21fb2eddb');"
        );

        $this->loginAs('manager@kurapov.ee');
        $result = $this->curlPOST($this->baseURL . 'User/remove', ['id' => 2]);
        $this->assertNotContains('error', $result);
        $this->assertEquals("{'status':'ok'}", $result, $result);
    }
}

If you take care after your project and code, then you write unit tests. But with them, there are «special cases». Filesystem and resources are one of them. Solving it straight on is done by making separate folder tree just for tests, hoping that they are stable enough to run after fails and that they will not touch and delete actual paths.

A more correct approach is to use in-memory virtual file system, vfs for short. And since resources are, by their nature, streams, so is the name of mock library for it - vfsStream. So lets install it..

composer install mikey179/vfsStream

Now we need to initialize it vfs. Note that all functions operate within the context of its root folder. If you start making virtual folders without providing it, they will not appear in the result tree.

public function setUp() {
    //vfsStream::setup();
    //vfsStreamWrapper::register();
    //vfsStreamWrapper::setRoot(vfsStream::newDirectory('root', 0777));
    //$this->rootDir = vfsStreamWrapper::getRoot();
    $this->rootDir = vfsStream::setup('root', 0777);
}

Jira from Atlassian — is the most advanced task and bug tracker, which is agile enough to adapt to organization workflow. But if you are not using Bamboo yet, preferring PHPCI for continuous integration, then it might be beneficial to see test results grouped by feature.

This is quite a contraversial topic, because some test-enthusiasts don't understand why would you need to tie tests to features. These developers think simply that «all tests must pass», or just think its overcomplicates things without any benefit.

I think this method does help to increase transparency of feature coverage. If a developer is making a new feature, then it doesn't mean that he will cover it with tests immediately. And even code is 100% unit-test covered, it doesn't mean that there is no bug. You need to write integration tests and those don't usually generate coverage report. So seeing that at least some tests are created for feautre X, is useful for boosting confidence. This is useful if you are a developer who is not following TDD "test first" rule and if you are a manager and quality reporting is not transparent and granular enough.

My second point is that code, tests and features get old. You need to recycle old parts of application. Usually unit-test coverage helps to find code that never gets executed. With integration tests, again, there is no such flexibility — your code is covered, but it is never executed on the client/frontend side (deprecated API, service got migrated). In such cases, usually no one wants to risk removing enitre module and it lives on in your codebase as a zombie. Thats why high-level feature deprecation, requires deletion of tests and code. Annotation in this case is what helps you to tie everything together.

Finally, jira task annotation is an informative link, an extended way to comment your code, where you can post a screenshot, read full history of changes, see a big picture. Sometimes I practice linking complicated code areas and decisions with jira issue.

An interesting side effect which i noticed after using such technique - you are forced to organize your issues, which is a good thing. You don't feel comfortable making duplicate issues, distilling issues it too many subtasks or concentrate everything in one. You want to tie tasks among each other. Specification is thus based on tests, but without cucumbers.

Installing on PHPUnit

To tie tests, we'll need API client for Jira..

composer install chobie/jira-api-restclient

Now in tests/bootstrap.php, which is for bootstrapping phpunit, let's insert custom authentication for API and include installed library:

define('JIRA_LOGIN', '');
define('JIRA_PASS', '');
require __DIR__ . '/../vendor/composer/chobie/jira-api-restclient/src/Jira/Api.php';

In order for Jira to store data for every issue, we need to add a custom field. To do that, go to Settings and add new custom field, Tests..

myapp.atlassian.net/secure/admin/ViewCustomFields.jspa

Заметьте под каким номером поле сохранилось - он будет использоваться в коде. В моём случае это customfield_10402.

Code

Next, we need to add extra functionality to each test class. But since we can't break singular inheritance from PHPUnit, we'll use traits. And in order for tests to refrain from pinging API after each test execution, we'll cache results in json file and make a network request in the end of test suite run. Substitute all constants and URLs.

trait JiraConnect {

    private $ticket;
    private $currentTest;
//    private $failed = false;

    static $testResults = [
        //ticket=>text
    ];

    //Store test results in file for cross-class results
    function saveJira($status) {
        if (!$this->ticket) {
            return;
        }

        $storage = PATH_APP . '/tests/ticket_status.json';
        if (!file_exists($storage)) {
            touch($storage);
        }

        $s = json_decode(file_get_contents($storage), true);

        if (!isset($s['testResults'][$this->ticket])) {
            $s['testResults'][$this->ticket] = [];
        }
        $s['testResults'][$this->ticket][$this->currentTest] = ($status ? 'ok' : 'fail');

        file_put_contents($storage, json_encode($s));
    }

    static function updateJiraTestStatus() {
//        $name = $this->currentTest;

        if (!defined('JIRA_LOGIN') || JIRA_LOGIN == '') {
//            echo('Please define JIRA_LOGIN, JIRA_PASS in bootstrap.php to update JIRA issues');
            return;
        }

        $api = new \chobie\Jira\Api(
            "https://myapp.atlassian.net",
            new chobie\Jira\Api\Authentication\Basic(JIRA_LOGIN, JIRA_PASS)
        );

        $storage = PATH_APP . '/tests/ticket_status.json';
        $s       = json_decode(file_get_contents($storage), true);

        foreach ($s['testResults'] as $ticket => $testResults) {
            $sResult = "";
            foreach ($testResults as $class => $status) {
                $sResult .= $status . " — " . $class . "\n";
            }

            $updObj                    = new stdClass();
            $updObj->customfield_10402 = [
                ['set' => $sResult]
            ];

            $r = $api->editIssue($ticket, [
                "update" => $updObj
            ]);
        }
    }

    function usingMethod($class, $method) {
//        echo $class;
        $this->ticket      = $this->getTicket($class, $method);
        $this->currentTest = $class . '::' . $method;

        return $this;
    }

    function getTicket($class, $method) {
        $r   = new ReflectionMethod($class, $method);
        $doc = $r->getDocComment();
        preg_match_all('#@ticket (.*?)\n#s', $doc, $annotations);

        if (isset($annotations[1][0]))
            return $annotations[1][0];
    }


    function tearDown() {
        $className = explode('::', $this->toString())[0];
        $this->usingMethod($className, $this->getName());
        $this->saveJira(!$this->hasFailed());
    }

    static function tearDownAfterClass() {
        self::updateJiraTestStatus();
    }


    function onNotSuccessfulTest(Exception $e) {

        if (method_exists($e, 'getComparisonFailure') && $e->getComparisonFailure()) {
            $trace = $e->getComparisonFailure()->getTrace();
        } elseif (method_exists($e, 'getSerializableTrace')) {
            $trace = $e->getSerializableTrace();
        }

        if (isset($trace)) {
            $method = $trace[4]['function'];
            $class  = $trace[4]['class'];

            $this->usingMethod($class, $method)->saveJira(false);
        }
        throw $e;
    }
}

Now the integration test..

require_once 'JiraConnect.php';
class EndpointConnector extends \PHPUnit_Framework_TestCase {
    use JiraConnect;
    
    /**
     * @test
     * @depends login
     * @group security
     * @ticket MY-389
     */
    function postAdd_SQLInjections() {…}
}

After each success or failure, JIRA issue will be filled with the results. Its better to have such configuration on staging server, to see latest state

Disadvantages — although you can search through «Tests» field, filtration is not as nice. There is no highlighting (like status has above). There is no view for all features and tests, no execution logs. Its not a full-blown CI server integration. Another problem - running tests fill «Activity stream», which mixes actual user changes. Finally, it runs only with phpunit so far and I haven't done anything to protractor e2e tests, which would be even more beneficial in Jira