Selenium WebDriver – How To Automatically Skip Tests Based On Test Environment

Overview:

This article is almost same as the previous article – How To Automatically Skip Tests Based On Defects Status – But a slightly different use case!! If you have not read the previous article, I would suggest you to read that first.

I run my automated tests in all the environments including Production. There are some test steps which I can not run on Production due to some restrictions. Basically we need to skip those steps which are not applicable for Production. I do not want to throw any testNG’s Skipped exception. I would follow a slightly different approach in this article. Lets see how we can get that implemented.

Sample testNG Test:

I create a sample test as shown here just for the demo.

public class UserRegistrationTest {

      @BeforeTest
      public void doDataBaseSetup(){
          if(!testEnvironment.equals("PRODUCTION")){
              DBUtil.doCleanUp();
          }   
      }
    
      @Test(dependsOnMethods="doDataBaseSetup")
      public void launchRegistrationPage() {
          RegistrationPage.launch();
          Assert.assertTrue(RegistrationPage.isAt());
      }

    @Test(dependsOnMethods="launchRegistrationPage")
      public void enterDemographicInformation() {
          RegistrationPage.enterDemographicInformation();
          RegistrationPage.submit();
          Assert.assertTrue(ProductsListPage.isAt());
      }
         
      @Test(dependsOnMethods="enterDemographicInformation")
      public void showCustomerData() {
          if(!testEnvironment.equals("PRODUCTION")){
              AdminPage.login();
              AdminPage.searchForCustomerData();
              Assert.assertTrue(AdminPage.isCustomerFound());
          }
      } 
}

Since I do have only limited access on Production, I can not run all the steps – which will make the test fail on the report. On top of that – I hate the if-else conditions in the test everywhere – which makes the code less readable.

We need a clean solution to ignore those methods which are not applicable for Production.

I have an enum which contains all the possible environments where I would be running my tests.

public enum TestEnvironment {
    
    DEV,
    QA,
    STAGING,
    PROD

}

I also have another class TestParameters which contains all the input parameters for the test suite. One of the methods can return the current environment on which the test is being executed as shown here.

public class TestParameters {
    
    // Not showing all the methods.
    
    public static TestEnvironment getEnvironment(){
        return testEnvironment;
    }
}

First, I am going to create a separate annotation as shown here.

@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.METHOD)
public @interface NotOnProduction {

}

Then, I need to create some kind of interceptor to completely ignore the execution of the steps which are not applicable on Production.

Below interceptor basically gets executed before testNG starts executing any test class. The intercept method filters all the method which are not applicable for Production only If the current test execution is happening on Production. If the test execution is on other environment, all the test steps should be executed.

public class TestEnvironmentListener implements IMethodInterceptor {
    
    Predicate<IMethodInstance> isTestExecutingOnProduction = (tip) -> {
        return TestParameters.getEnvironment()
                                  .equals(TestEnvironment.PROD);
    };

    Predicate<IMethodInstance> isNotApplicableForProduction = (method) -> {
        return method.getMethod()
                     .getConstructorOrMethod()
                     .getMethod()
                     .isAnnotationPresent(NotOnProduction.class);
    };
    
    @Override
    public List<IMethodInstance> intercept(List<IMethodInstance> testMethods, ITestContext context) {    
        
        //removeIf method returns a boolean if any of the item is removed
        boolean isTestRemoved = testMethods.removeIf(isTestExecutingOnProduction.and(isNotApplicableForProduction));
        
        //if any item is removed, then there is a chance that we might get execption due to missing dependencies.
        testMethods.stream()
                   .map(IMethodInstance::getMethod)
                   .forEach(method -> method.setIgnoreMissingDependencies(isTestRemoved));
        
        return testMethods;
    }

}

The rest is simple. We need to connect everything together. Add the listener in the testNG suite xml or in the base test.

@Listener(TestEnvironmentListener.class) // should be included in the base class
public class UserRegistrationTest {

    Test
    @NotOnProduction
    public void doDataBaseSetup(){
        DBUtil.doCleanUp();
    }

    @Test(dependsOnMethods="doDataBaseSetup")
    public void launchRegistrationPage() {
        RegistrationPage.launch();
        Assert.assertTrue(RegistrationPage.isAt());
    }

    @Test(dependsOnMethods="launchRegistrationPage")
    public void enterDemographicInformation() {
        RegistrationPage.enterDemographicInformation();
        RegistrationPage.submit();
        Assert.assertTrue(ProductsListPage.isAt());
    }
     
    @Test(dependsOnMethods="enterDemographicInformation")
    @NotOnProduction
    public void showCustomerData() {
        AdminPage.login();
        AdminPage.searchForCustomerData();
        Assert.assertTrue(AdminPage.isCustomerFound());
    } 
}

When we execute the test, if the test is executing on Production and if the annotation is found on the method, testNG will silently ignore those and executes only the remaining tests.

Test Execution on QA:

qa-test-1

 

Test Execution on PROD:

prod-test-1

NotOnProduction methods are simply disappeared when it is PROD.

Summary:

By creating a custom annotation and using testNG’s listener, We are able to skip methods without any issues and maintains the order the test execution.

Advantages:

  • No ugly if-else conditions everywhere.
  • No Skipped Exception.
  • No failed tests.
  • Readable tests.

 

Happy Testing & Subscribe 🙂

 

Share This:

Categories: Articles, Framework, Selenium, TestNG

2 comments

  • Musaffir

    This is just nice and neat , thanks for sharing..
    I also ran in to the same need as this situation where I don’t want to run some of my test methods in production environment . Currently I have implemented it by overriding ‘transform’ mehtod of testNg listener-
    org.testng.internal.annotations.IAnnotationTransformer

    The transform method has an argument – ITestAnnotation annotation
    which allows to enable or disable a test method present in the test class by passing boolean value –
    annotation.setEnabled(false);

    I decide this boolean value for the test , based on a configuration file outside of the test class, where each test is marked as Y/N – to run and not to run … and by default Y..

    The problem I see with my approach is I need to maintain multple config file … at least two such config file , one for Test server and one for Live server… and during run time pass a key to identify which config file to choose…. and let the test annotation to get enabled or disabled for the execution….

    I may think of modifying my appproach similar to what you have shared here… Thanks again…useful post 🙂

    Warm Regards
    Musaffir

    • vIns

      Thanks. There should be multiple approaches to achieve this. I like your approach as well to maintain that via config file.

Leave a Reply

Your email address will not be published. Required fields are marked *