For one of our projects, we implemented functional autotests using Robot Framework(test automation framework). Then additionally to functional tests we decided to add CSS regression tests. As a tool we chose WebdriverCSS , this plugin is an automatic visual regression-testing tool for WebdriverIO. Also, we use WebdriverCSS Adminpanel to manage all image repositories of our CSS regression tests made with WebdriverCSS.
The main requirement for our project was to generate only one report, provided by Robot Framework. So, we just added results of our regression tests in our RobotFramwork report. Here is how we did that:
1) Robot Framework test: regression tests are done via node.js code. If "${result} run regression test ${test_path}/tests/access/regressions/404_403_page.js" is failed - > an image with diff will be added in logs
`*** Settings ***
Force Tags 403 404 regression
Resource ../../Resources/Project.robot
Resource ../../Resources/Common.robot
*** Test Cases ***
Regression Css Test to Check 404 Error Page
${result} run regression test ${test_path}/tests/access/regressions/404_403_page.js
${status} ${value}= Run keyword and ignore error Should Be True ${result}
Run Keyword If '${status}' == 'FAIL' run keywords
Log <img src="Regression/403_404/diffs/access denied.access_denied.1366px.diff.png"> html=yes
Should Be True ${result}
Run Keyword If '${status}' == 'PASS' Log <img src="Regression/403_404/my-shots/access denied.access_denied.1366px.baseline.png"> html=yes2) "run regression test" implementation: it's custom function which just runs regression test(node.js script).
from Naked.toolshed.shell import execute_js, muterun_js
def run_regression_test(js_file_path):
success = execute_js(js_file_path)
if success:
return True3) regression test - js_file_path (using node.js)
var assert = require('assert');
var proj = require('../../../Resources/project.conf.js');
var client = require('webdriverio').remote({
desiredCapabilities: {
browserName: proj.browserName
}
});
require('webdrivercss').init(client, {
screenshotRoot: 'Results/Regression/403_404/my-shots',
failedComparisonsRoot: 'Results/Regression/403_404/diffs',
misMatchTolerance: 0.05,
screenWidth: proj.screenWidth
});
client
.init()
.url(proj.domain + 'not-found')
.webdrivercss('not found', [
{
name: 'not_found',
elem: proj.content,
}, {
name: 'not_found_full_page',
exclude:[
proj.pane_page_logo,
proj.pane_site_name,
proj.pane_main_menu,
proj.pane_footer_menu,
]
}
], function(err, res) {
assert.ifError(err);
assert.ok(res.not_found[0].isWithinMisMatchTolerance);
assert.ok(res.not_found_full_page[0].isWithinMisMatchTolerance);
})
.end()