Nothing found, try being more general or more specific

Connecting tests using @ticket annotations with Jira

Jira from Atlassian — is the most advanced task and bug tracker, which is agile enough to adapt to organization workflow. But if you are not using Bamboo yet, preferring PHPCI for continuous integration, then it might be beneficial to see test results grouped by feature.

This is quite a contraversial topic, because some test-enthusiasts don't understand why would you need to tie tests to features. These developers think simply that «all tests must pass», or just think its overcomplicates things without any benefit.

I think this method does help to increase transparency of feature coverage. If a developer is making a new feature, then it doesn't mean that he will cover it with tests immediately. And even code is 100% unit-test covered, it doesn't mean that there is no bug. You need to write integration tests and those don't usually generate coverage report. So seeing that at least some tests are created for feautre X, is useful for boosting confidence. This is useful if you are a developer who is not following TDD "test first" rule and if you are a manager and quality reporting is not transparent and granular enough.

My second point is that code, tests and features get old. You need to recycle old parts of application. Usually unit-test coverage helps to find code that never gets executed. With integration tests, again, there is no such flexibility — your code is covered, but it is never executed on the client/frontend side (deprecated API, service got migrated). In such cases, usually no one wants to risk removing enitre module and it lives on in your codebase as a zombie. Thats why high-level feature deprecation, requires deletion of tests and code. Annotation in this case is what helps you to tie everything together.

Finally, jira task annotation is an informative link, an extended way to comment your code, where you can post a screenshot, read full history of changes, see a big picture. Sometimes I practice linking complicated code areas and decisions with jira issue.

An interesting side effect which i noticed after using such technique - you are forced to organize your issues, which is a good thing. You don't feel comfortable making duplicate issues, distilling issues it too many subtasks or concentrate everything in one. You want to tie tasks among each other. Specification is thus based on tests, but without cucumbers.

Installing on PHPUnit

To tie tests, we'll need API client for Jira..

composer install chobie/jira-api-restclient

Now in tests/bootstrap.php, which is for bootstrapping phpunit, let's insert custom authentication for API and include installed library:

define('JIRA_LOGIN', '');
define('JIRA_PASS', '');
require __DIR__ . '/../vendor/composer/chobie/jira-api-restclient/src/Jira/Api.php';

In order for Jira to store data for every issue, we need to add a custom field. To do that, go to Settings and add new custom field, Tests..

myapp.atlassian.net/secure/admin/ViewCustomFields.jspa

Jira custom.png

Заметьте под каким номером поле сохранилось - он будет использоваться в коде. В моём случае это customfield_10402.

URL.png

Code

Next, we need to add extra functionality to each test class. But since we can't break singular inheritance from PHPUnit, we'll use traits. And in order for tests to refrain from pinging API after each test execution, we'll cache results in json file and make a network request in the end of test suite run. Substitute all constants and URLs.

trait JiraConnect {

    private $ticket;
    private $currentTest;
//    private $failed = false;

    static $testResults = [
        //ticket=>text
    ];

    //Store test results in file for cross-class results
    function saveJira($status) {
        if (!$this->ticket) {
            return;
        }

        $storage = PATH_APP . '/tests/ticket_status.json';
        if (!file_exists($storage)) {
            touch($storage);
        }

        $s = json_decode(file_get_contents($storage), true);

        if (!isset($s['testResults'][$this->ticket])) {
            $s['testResults'][$this->ticket] = [];
        }
        $s['testResults'][$this->ticket][$this->currentTest] = ($status ? 'ok' : 'fail');

        file_put_contents($storage, json_encode($s));
    }

    static function updateJiraTestStatus() {
//        $name = $this->currentTest;

        if (!defined('JIRA_LOGIN') || JIRA_LOGIN == '') {
//            echo('Please define JIRA_LOGIN, JIRA_PASS in bootstrap.php to update JIRA issues');
            return;
        }

        $api = new \chobie\Jira\Api(
            "https://myapp.atlassian.net",
            new chobie\Jira\Api\Authentication\Basic(JIRA_LOGIN, JIRA_PASS)
        );

        $storage = PATH_APP . '/tests/ticket_status.json';
        $s       = json_decode(file_get_contents($storage), true);

        foreach ($s['testResults'] as $ticket => $testResults) {
            $sResult = "";
            foreach ($testResults as $class => $status) {
                $sResult .= $status . " — " . $class . "\n";
            }

            $updObj                    = new stdClass();
            $updObj->customfield_10402 = [
                ['set' => $sResult]
            ];

            $r = $api->editIssue($ticket, [
                "update" => $updObj
            ]);
        }
    }

    function usingMethod($class, $method) {
//        echo $class;
        $this->ticket      = $this->getTicket($class, $method);
        $this->currentTest = $class . '::' . $method;

        return $this;
    }

    function getTicket($class, $method) {
        $r   = new ReflectionMethod($class, $method);
        $doc = $r->getDocComment();
        preg_match_all('#@ticket (.*?)\n#s', $doc, $annotations);

        if (isset($annotations[1][0]))
            return $annotations[1][0];
    }


    function tearDown() {
        $className = explode('::', $this->toString())[0];
        $this->usingMethod($className, $this->getName());
        $this->saveJira(!$this->hasFailed());
    }

    static function tearDownAfterClass() {
        self::updateJiraTestStatus();
    }


    function onNotSuccessfulTest(Exception $e) {

        if (method_exists($e, 'getComparisonFailure') && $e->getComparisonFailure()) {
            $trace = $e->getComparisonFailure()->getTrace();
        } elseif (method_exists($e, 'getSerializableTrace')) {
            $trace = $e->getSerializableTrace();
        }

        if (isset($trace)) {
            $method = $trace[4]['function'];
            $class  = $trace[4]['class'];

            $this->usingMethod($class, $method)->saveJira(false);
        }
        throw $e;
    }
}

Now the integration test..

require_once 'JiraConnect.php';
class EndpointConnector extends \PHPUnit_Framework_TestCase {
    use JiraConnect;
    
    /**
     * @test
     * @depends login
     * @group security
     * @ticket MY-389
     */
    function postAdd_SQLInjections() {…}
}

After each success or failure, JIRA issue will be filled with the results. Its better to have such configuration on staging server, to see latest state

Task view.png

Disadvantages — although you can search through «Tests» field, filtration is not as nice. There is no highlighting (like status has above). There is no view for all features and tests, no execution logs. Its not a full-blown CI server integration. Another problem - running tests fill «Activity stream», which mixes actual user changes. Finally, it runs only with phpunit so far and I haven't done anything to protractor e2e tests, which would be even more beneficial in Jira