_edited.jpg)
This week, I have been helping my colleagues migrate our existing WebdriverIO tests to Cypress. For anyone who wants to know what Cypress is and how it's different from Selenium, check out this webinar from Gil Tayar. Anyway, while migrating some of the tests, I remembered that the existing test suite was using Cucumber tags to filter which tests are run on which test environments. Test filtering is currently a useful feature for us as we only run a subset of tests on PR environments. With the Cypress framework that we currently have, we have decided to follow the standard approach of writing the tests using Mocha as its test runner as opposed to Cucumber. While Mocha has a feature to filter its tests by using the —grep pattern, this is currently not supported within Cypress and have been requested as a new feature by their users. I didn’t really want to use the Cypress Cucumber plugin just to utilise the tagging feature so I decided to implement a simple workaround which I am planning to propose to my colleagues while I wait for Cypress to implement this feature 🙂.
The first thing that I did was I created a module called test-filter.ts which basically filters your Cypress tests based on what tag or tags are provided. This accepts two parameters, the tags that you defined on your spec file and the function that will run when the tags are found. If the environment variable 'TEST_TAGS' exist, we simply split its values and store this to an array variable called tags. The Array.some() function is then used which would return true if there is at least one tag from the definedTags array that is included in the tags array. If it's true, we run the test.
The next step is to import this module to my spec files. As an example, the below code snippet shows that I have added in two tags for this test (smoke and test). The function that I pass to TestFilter is the entire describe block containing my one test.
To run the tests with the tagging functionality, I had to export an environment variable to Cypress called 'TEST_TAGS'. For example, if we have 4 spec files and 3 of the spec files are tagged 'regression' and 1 is tagged 'smoke', running the command below will only execute the test tagged with 'smoke'.
CYPRESS_TEST_TAGS=smoke npm run cy:run:local:dev
Multiple tags are also supported and this can be run by running the example command below:
CYPRESS_TEST_TAGS=testA,testB npm run cy:run:local:dev
We can see from the screenshot below that there are two tests ran (one test tagged as testA and the other as testB).
There is a limitation though with this approach and it's not as powerful as Cucumber tags. For example if we look at the code snippet below and we only want to run 'Test B', running the command CYPRESS_TEST_TAGS=testB npm run cy:run:local:dev will still run 'Test A' since it's not wrapped with TestFilter. Since I don’t want to tag 'Test A' in this example, for it to work properly, I had to also wrapped this test with TestFilter and pass it an empty array.
Although it has a limitation, using this approach clearly has some advantages that I think our team can benefit at. If we need to only run, let's say, our smoke tests, we don't need to define a separate folder for it in Cypress and write duplicate tests. We can just pick a selection of our existing tests and tag it accordingly.
To see an example project in action, please look at my github repo here for reference.
Thanks a lot for this, you made my day ;)
However, I'm using cypress with cucumber then on test results, since it does always display each executed test for Given/And/Then/... in order to know the test has not run I at least want a log saying test has been skipped, I then by default added a "Test skipped" log when tag is not found and added as optional a noRunTest Function parameter to run when no tag has been found (or environment var is not defined):
const TestFilter = (definedTags: string[], runTest: Function, noRunTest?: Function) => { let isFound: boolean = false; if (Cypress.env('TEST_TAGS')) { const tags = Cypress.env('TEST_TAGS').split(','); isFound = definedTags.some(definedTag => tags.includes(definedTag)); } if (isFound) { …Hello, In your approach, it's possible to put different tags in the same spec file? I'm running your code on Windows, but execute 0 tests. I don´t understand... I execute in terminal:
set CYPRESS_TEST_TAGS=regression
npx cypress run --spec file.ts
Thanks for your answer :)
Hi Marie,
Can we have Cypress not to run the tests which doesn't match the tags criteria.
I am having 28 tests in my project and if I want to run only 1 test tagged as 'Production-Tests', Cypress still try to run all 28 tests.
And if that one test fails, we see this as the report.
1 of 28 failed (4%)
Can't we have 1 of 1 failed (100%)?
Cheers!
Good solution - have been thinking about a nice way to do this also. Agree that one benefit of using tags over file naming conventions or grepping is it allows you to specify exact or verbose tags and also it allows spec files to appear in multiple groups.
Maybe would be nice to avoid wrapping the tests and instead find a way to annotate the modules themselves using a comment. This way you could filter what tests are discovered by Cypress rather than patching the execution process.
Hi Marie (great name btw), this is working great for me minus one thing: my tests are not being identified in Test Runner (GUI). do you know how to resolve this? Thank you!