Gyan Code

Selenium with TestNG

Annotations, Assertions, DataProvider & XML

Introduction to TestNG

TestNG is a powerful testing framework inspired by JUnit and NUnit. It provides better test configuration, parallel execution, data-driven testing, and detailed reporting capabilities.

Setup & Installation

Maven Dependencies (pom.xml)

<!-- TestNG -->
<dependency>
    <groupId>org.testng</groupId>
    <artifactId>testng</artifactId>
    <version>7.10.2</version>
    <scope>test</scope>
</dependency>

<!-- Selenium WebDriver -->
<dependency>
    <groupId>org.seleniumhq.selenium</groupId>
    <artifactId>selenium-java</artifactId>
    <version>4.27.0</version>
</dependency>

Gradle Dependencies (build.gradle)

dependencies {
    testImplementation 'org.testng:testng:7.10.2'
    implementation 'org.seleniumhq.selenium:selenium-java:4.27.0'
}

TestNG Annotations

Annotations define test execution flow and setup/teardown operations.

Annotation Hierarchy

Annotation          | Execution Order | Description
--------------------+-----------------+--------------------------------
@BeforeSuite        | 1              | Runs once before all tests
@BeforeTest         | 2              | Runs before each test tag
@BeforeClass        | 3              | Runs once before test class
@BeforeMethod       | 4              | Runs before each @Test method
@Test               | 5              | The actual test method
@AfterMethod        | 6              | Runs after each @Test method
@AfterClass         | 7              | Runs once after test class
@AfterTest          | 8              | Runs after each test tag
@AfterSuite         | 9              | Runs once after all tests

Basic Test Class

import org.testng.annotations.*;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;

public class LoginTest {
    WebDriver driver;

    @BeforeSuite
    public void beforeSuite() {
        System.out.println("Before Suite - Setup global config");
    }

    @BeforeClass
    public void setupClass() {
        System.out.println("Before Class - Initialize driver");
        driver = new ChromeDriver();
    }

    @BeforeMethod
    public void beforeMethod() {
        System.out.println("Before Method - Navigate to URL");
        driver.get("https://example.com");
    }

    @Test
    public void testLogin() {
        System.out.println("Executing test");
        // Test logic here
    }

    @AfterMethod
    public void afterMethod() {
        System.out.println("After Method - Clear cookies");
        driver.manage().deleteAllCookies();
    }

    @AfterClass
    public void tearDownClass() {
        System.out.println("After Class - Close driver");
        driver.quit();
    }

    @AfterSuite
    public void afterSuite() {
        System.out.println("After Suite - Cleanup");
    }
}

Annotation Tips

  • Use @BeforeMethod for common setup before each test
  • Use @BeforeClass for one-time setup like driver initialization
  • Use @AfterMethod for cleanup after each test
  • Use @AfterClass to close browser and release resources

Writing Test Methods

Simple Test

@Test
public void testPageTitle() {
    driver.get("https://www.google.com");
    String title = driver.getTitle();
    Assert.assertEquals(title, "Google");
}

Test with Priority

// Tests execute in priority order (lower first)
@Test(priority = 1)
public void testLogin() {
    // Login test
}

@Test(priority = 2)
public void testDashboard() {
    // Dashboard test - runs after login
}

Test with Description

@Test(description = "Verify user can login with valid credentials")
public void testValidLogin() {
    // Test implementation
}

Dependent Tests

@Test
public void testLogin() {
    // Login functionality
}

@Test(dependsOnMethods = {"testLogin"})
public void testAddToCart() {
    // Only runs if testLogin passes
}

Enable/Disable Tests

@Test(enabled = false)
public void testUnderDevelopment() {
    // This test will be skipped
}

Timeout for Tests

@Test(timeOut = 5000) // 5 seconds
public void testPageLoad() {
    driver.get("https://example.com");
    // Fails if execution takes more than 5 seconds
}

Expected Exceptions

@Test(expectedExceptions = NoSuchElementException.class)
public void testElementNotFound() {
    driver.findElement(By.id("nonExistent"));
    // Test passes if NoSuchElementException is thrown
}

Assertions in TestNG

Assertions validate expected outcomes. TestNG provides both hard and soft assertions.

Hard Assertions

import org.testng.Assert;

@Test
public void testAssertions() {
    // Equality
    Assert.assertEquals(actual, expected);
    Assert.assertEquals(driver.getTitle(), "Expected Title");

    // Boolean
    Assert.assertTrue(condition);
    Assert.assertFalse(condition);

    // Null checks
    Assert.assertNull(object);
    Assert.assertNotNull(object);

    // With custom message
    Assert.assertEquals(actual, expected, "Values don't match!");
}

Common Assertion Methods

// Compare values
Assert.assertEquals(actual, expected);
Assert.assertNotEquals(actual, expected);

// Check conditions
Assert.assertTrue(element.isDisplayed());
Assert.assertFalse(element.isEnabled());

// Check null/not null
Assert.assertNull(driver.findElement(By.id("id")));
Assert.assertNotNull(element);

// Fail test explicitly
Assert.fail("Test failed due to timeout");

Soft Assertions

import org.testng.asserts.SoftAssert;

@Test
public void testSoftAssertions() {
    SoftAssert softAssert = new SoftAssert();

    // These continue even if they fail
    softAssert.assertEquals(driver.getTitle(), "Title");
    softAssert.assertTrue(false, "This will fail but test continues");
    softAssert.assertNotNull(null, "Another failure");

    // Must call assertAll() to mark test as failed
    softAssert.assertAll();
}

Data-Driven Testing

DataProvider allows running the same test with different data sets.

Basic DataProvider

@DataProvider(name = "loginData")
public Object[][] getLoginData() {
    return new Object[][] {
        {"user1@test.com", "password123"},
        {"user2@test.com", "password456"},
        {"user3@test.com", "password789"}
    };
}

@Test(dataProvider = "loginData")
public void testLogin(String email, String password) {
    driver.findElement(By.id("email")).sendKeys(email);
    driver.findElement(By.id("password")).sendKeys(password);
    driver.findElement(By.id("loginBtn")).click();
    
    // Add assertions
}

DataProvider from Excel

import org.apache.poi.ss.usermodel.*;
import java.io.FileInputStream;

@DataProvider(name = "excelData")
public Object[][] getExcelData() throws Exception {
    String filePath = "./testdata/TestData.xlsx";
    FileInputStream file = new FileInputStream(filePath);
    Workbook workbook = WorkbookFactory.create(file);
    Sheet sheet = workbook.getSheetAt(0);
    
    int rowCount = sheet.getLastRowNum();
    int colCount = sheet.getRow(0).getLastCellNum();
    
    Object[][] data = new Object[rowCount][colCount];
    
    for (int i = 0; i < rowCount; i++) {
        Row row = sheet.getRow(i + 1);
        for (int j = 0; j < colCount; j++) {
            data[i][j] = row.getCell(j).toString();
        }
    }
    
    workbook.close();
    file.close();
    return data;
}

DataProvider with Method

// DataProvider in different class
public class TestDataProvider {
    @DataProvider(name = "searchData")
    public static Object[][] getSearchData() {
        return new Object[][] {
            {"Selenium"},
            {"TestNG"},
            {"Java"}
        };
    }
}

// Using DataProvider from different class
public class SearchTest {
    @Test(dataProvider = "searchData", dataProviderClass = TestDataProvider.class)
    public void testSearch(String keyword) {
        driver.findElement(By.name("q")).sendKeys(keyword);
        // Add search logic
    }
}

Parallel DataProvider

@DataProvider(name = "parallelData", parallel = true)
public Object[][] getParallelData() {
    return new Object[][] {
        {"Test1"},
        {"Test2"},
        {"Test3"}
    };
}

DataProvider Best Practices

  • Keep test data separate from test logic
  • Use meaningful DataProvider names
  • Store large datasets in external files (Excel, CSV, JSON)
  • Use parallel execution for independent data sets

TestNG XML Configuration

TestNG XML files define test suites, organize tests, and configure execution parameters.

Basic testng.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Test Suite">
    <test name="Login Tests">
        <classes>
            <class name="tests.LoginTest"/>
        </classes>
    </test>
</suite>

Multiple Test Classes

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Regression Suite">
    <test name="All Tests">
        <classes>
            <class name="tests.LoginTest"/>
            <class name="tests.HomePageTest"/>
            <class name="tests.CheckoutTest"/>
        </classes>
    </test>
</suite>

Run Specific Methods

<suite name="Smoke Suite">
    <test name="Smoke Tests">
        <classes>
            <class name="tests.LoginTest">
                <methods>
                    <include name="testValidLogin"/>
                    <include name="testLogout"/>
                </methods>
            </class>
        </classes>
    </test>
</suite>

Package-Level Execution

<suite name="Package Suite">
    <test name="Package Tests">
        <packages>
            <package name="tests.regression"/>
            <package name="tests.smoke"/>
        </packages>
    </test>
</suite>

Parallel Execution

<suite name="Parallel Suite" parallel="tests" thread-count="3">
    <test name="Chrome Tests">
        <parameter name="browser" value="chrome"/>
        <classes>
            <class name="tests.LoginTest"/>
        </classes>
    </test>
    
    <test name="Firefox Tests">
        <parameter name="browser" value="firefox"/>
        <classes>
            <class name="tests.LoginTest"/>
        </classes>
    </test>
</suite>

<!-- Parallel options:
     tests    - Run test tags in parallel
     classes  - Run test classes in parallel
     methods  - Run test methods in parallel
-->

Parameters in XML

<suite name="Parameterized Suite">
    <parameter name="url" value="https://example.com"/>
    <parameter name="browser" value="chrome"/>
    
    <test name="Tests">
        <classes>
            <class name="tests.LoginTest"/>
        </classes>
    </test>
</suite>

Using Parameters in Test

import org.testng.annotations.Parameters;

public class LoginTest {
    WebDriver driver;
    
    @Parameters({"browser", "url"})
    @BeforeClass
    public void setup(String browser, String url) {
        if (browser.equalsIgnoreCase("chrome")) {
            driver = new ChromeDriver();
        } else if (browser.equalsIgnoreCase("firefox")) {
            driver = new FirefoxDriver();
        }
        driver.get(url);
    }
}

Groups in XML

<suite name="Group Suite">
    <test name="Smoke Tests">
        <groups>
            <run>
                <include name="smoke"/>
            </run>
        </groups>
        <classes>
            <class name="tests.LoginTest"/>
            <class name="tests.HomePageTest"/>
        </classes>
    </test>
</suite>

Test Groups

Groups allow you to categorize tests and run specific categories as needed.

Defining Groups

public class LoginTest {
    
    @Test(groups = {"smoke", "regression"})
    public void testValidLogin() {
        // Test logic
    }
    
    @Test(groups = {"regression"})
    public void testInvalidLogin() {
        // Test logic
    }
    
    @Test(groups = {"smoke"})
    public void testLogout() {
        // Test logic
    }
}

Group Dependencies

@Test(groups = {"database"})
public void testDatabaseConnection() {
    // Setup database
}

@Test(groups = {"api"}, dependsOnGroups = {"database"})
public void testAPI() {
    // API tests that need database
}

Class-Level Groups

@Test(groups = {"regression"})
public class CheckoutTest {
    
    @Test
    public void testAddToCart() {
        // All methods inherit "regression" group
    }
    
    @Test
    public void testCheckout() {
        // This is also in "regression" group
    }
}

Running Groups Programmatically

import org.testng.TestNG;

public class TestRunner {
    public static void main(String[] args) {
        TestNG testng = new TestNG();
        testng.setTestClasses(new Class[]{LoginTest.class});
        testng.setGroups("smoke");
        testng.run();
    }
}

Common Group Categories

  • smoke: Quick tests to verify basic functionality
  • regression: Comprehensive tests for all features
  • sanity: Tests after minor changes
  • critical: High-priority test cases
  • ui: User interface tests
  • api: API integration tests

TestNG Listeners

Listeners monitor test execution and perform actions at different stages like test start, success, failure, or skip.

ITestListener Implementation

import org.testng.ITestListener;
import org.testng.ITestResult;

public class TestListener implements ITestListener {
    
    @Override
    public void onTestStart(ITestResult result) {
        System.out.println("Test Started: " + result.getName());
    }
    
    @Override
    public void onTestSuccess(ITestResult result) {
        System.out.println("Test Passed: " + result.getName());
    }
    
    @Override
    public void onTestFailure(ITestResult result) {
        System.out.println("Test Failed: " + result.getName());
        // Capture screenshot on failure
        captureScreenshot(result.getName());
    }
    
    @Override
    public void onTestSkipped(ITestResult result) {
        System.out.println("Test Skipped: " + result.getName());
    }
    
    private void captureScreenshot(String testName) {
        // Screenshot logic here
    }
}

Using Listener with Annotation

import org.testng.annotations.Listeners;

@Listeners(TestListener.class)
public class LoginTest {
    
    @Test
    public void testLogin() {
        // Test logic
    }
}

Using Listener in XML

<suite name="Suite">
    <listeners>
        <listener class-name="listeners.TestListener"/>
    </listeners>
    
    <test name="Tests">
        <classes>
            <class name="tests.LoginTest"/>
        </classes>
    </test>
</suite>

IRetryAnalyzer for Failed Tests

import org.testng.IRetryAnalyzer;
import org.testng.ITestResult;

public class RetryAnalyzer implements IRetryAnalyzer {
    private int retryCount = 0;
    private static final int maxRetryCount = 2;
    
    @Override
    public boolean retry(ITestResult result) {
        if (retryCount < maxRetryCount) {
            retryCount++;
            System.out.println("Retrying " + result.getName() + 
                             " - Attempt " + retryCount);
            return true;
        }
        return false;
    }
}

// Using RetryAnalyzer
@Test(retryAnalyzer = RetryAnalyzer.class)
public void testFlaky() {
    // Test that might fail intermittently
}

Custom Report Listener

import org.testng.IReporter;
import org.testng.ISuite;
import org.testng.xml.XmlSuite;

public class CustomReporter implements IReporter {
    
    @Override
    public void generateReport(List<XmlSuite> xmlSuites, 
                                 List<ISuite> suites, 
                                 String outputDirectory) {
        // Custom report generation logic
        System.out.println("Generating custom report...");
    }
}

Parallel Test Execution

Run tests in parallel to reduce overall execution time.

Parallel at Test Level

<suite name="Parallel Suite" parallel="tests" thread-count="3">
    <test name="Test1">
        <classes>
            <class name="tests.LoginTest"/>
        </classes>
    </test>
    
    <test name="Test2">
        <classes>
            <class name="tests.CheckoutTest"/>
        </classes>
    </test>
</suite>

Parallel at Class Level

<suite name="Parallel Suite" parallel="classes" thread-count="2">
    <test name="All Tests">
        <classes>
            <class name="tests.LoginTest"/>
            <class name="tests.HomePageTest"/>
            <class name="tests.CheckoutTest"/>
        </classes>
    </test>
</suite>

Parallel at Method Level

<suite name="Parallel Suite" parallel="methods" thread-count="5">
    <test name="All Tests">
        <classes>
            <class name="tests.LoginTest"/>
        </classes>
    </test>
</suite>

Thread-Safe Driver Setup

public class BaseTest {
    protected static ThreadLocal<WebDriver> driver = new ThreadLocal<>();
    
    public static WebDriver getDriver() {
        return driver.get();
    }
    
    @BeforeMethod
    public void setUp() {
        WebDriver webDriver = new ChromeDriver();
        driver.set(webDriver);
        getDriver().manage().window().maximize();
    }
    
    @AfterMethod
    public void tearDown() {
        if (getDriver() != null) {
            getDriver().quit();
            driver.remove();
        }
    }
}

Parallel Modes

Mode        | Description                    | Thread Count
------------+--------------------------------+------------------
tests       | Runs test tags in parallel     | 2-5 recommended
classes     | Runs test classes in parallel  | 3-10 recommended
methods     | Runs test methods in parallel  | 5-20 recommended
instances   | Runs instances in parallel     | Varies
none        | No parallel execution          | 1

Test Reporting

TestNG generates detailed HTML reports automatically. You can also integrate third-party reporting tools.

Default Reports Location

test-output/
├── index.html          // Main report
├── emailable-report.html  // Email-friendly report
├── testng-results.xml  // XML results
└── Suite/
    ├── Test.html
    └── Test.xml

Extent Reports Integration

<!-- Add Extent Reports dependency -->
<dependency>
    <groupId>com.aventstack</groupId>
    <artifactId>extentreports</artifactId>
    <version>5.1.2</version>
</dependency>

Extent Reports Manager

import com.aventstack.extentreports.ExtentReports;
import com.aventstack.extentreports.ExtentTest;
import com.aventstack.extentreports.reporter.ExtentSparkReporter;

public class ExtentManager {
    private static ExtentReports extent;
    private static ThreadLocal<ExtentTest> test = new ThreadLocal<>();
    
    public static ExtentReports createInstance() {
        ExtentSparkReporter sparkReporter = 
            new ExtentSparkReporter("./reports/ExtentReport.html");
        extent = new ExtentReports();
        extent.attachReporter(sparkReporter);
        return extent;
    }
    
    public static ExtentTest getTest() {
        return test.get();
    }
    
    public static void setTest(ExtentTest extentTest) {
        test.set(extentTest);
    }
}

Using Extent Reports in Tests

public class LoginTest {
    ExtentReports extent = ExtentManager.createInstance();
    
    @BeforeMethod
    public void setup() {
        ExtentTest extentTest = extent.createTest("Login Test");
        ExtentManager.setTest(extentTest);
    }
    
    @Test
    public void testLogin() {
        ExtentManager.getTest().info("Starting login test");
        // Test logic
        ExtentManager.getTest().pass("Login successful");
    }
    
    @AfterMethod
    public void tearDown() {
        extent.flush();
    }
}

Report Features

  • TestNG default reports show pass/fail/skip statistics
  • Extent Reports provide rich HTML reports with screenshots
  • Allure Reports offer interactive test execution dashboards
  • Reports can be integrated into CI/CD pipelines

Best Practices

Complete Test Class Example

import org.testng.annotations.*;
import org.testng.Assert;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.By;

public class LoginTest {
    WebDriver driver;

    @BeforeClass
    public void setup() {
        driver = new ChromeDriver();
        driver.manage().window().maximize();
    }

    @BeforeMethod
    public void navigateToLogin() {
        driver.get("https://example.com/login");
    }

    @Test(priority = 1, description = "Verify login with valid credentials")
    public void testValidLogin() {
        driver.findElement(By.id("username")).sendKeys("testuser");
        driver.findElement(By.id("password")).sendKeys("password123");
        driver.findElement(By.id("loginBtn")).click();

        Assert.assertTrue(driver.getCurrentUrl().contains("dashboard"),
            "Login failed - Dashboard not displayed");
    }

    @Test(priority = 2, description = "Verify error message for invalid credentials")
    public void testInvalidLogin() {
        driver.findElement(By.id("username")).sendKeys("wronguser");
        driver.findElement(By.id("password")).sendKeys("wrongpass");
        driver.findElement(By.id("loginBtn")).click();

        String errorMsg = driver.findElement(By.className("error")).getText();
        Assert.assertEquals(errorMsg, "Invalid credentials");
    }

    @AfterMethod
    public void clearSession() {
        driver.manage().deleteAllCookies();
    }

    @AfterClass
    public void teardown() {
        if (driver != null) {
            driver.quit();
        }
    }
}

Key Takeaways

  • TestNG provides powerful test configuration with annotations
  • DataProvider enables data-driven testing with multiple data sets
  • TestNG XML allows flexible test suite configuration
  • Listeners enable custom actions like screenshots on failure
  • Parallel execution significantly reduces test execution time
  • Groups help organize and selectively run tests

Next Steps

Continue Your Learning Journey

  • Page Object Model (POM) - Design pattern for maintainable test automation
  • Data-Driven Testing - Parameterize tests with Excel, CSV or database
  • Selenium Grid - Parallel execution across multiple browsers and machines
  • Extent Reports - Generate rich HTML reports with screenshots
  • CI/CD Integration - Automate test execution with Jenkins or GitHub Actions
  • BDD with Cucumber - Behavior-driven development with TestNG