QTP/UFT – Display Test execution progress in Jenkins Console

16 Comments / Articles, Best Practices, CI / CD / DevOps, Framework, Jenkins, QTP / By vIns / May 6, 2015

We had seen the QTP/UFT integration & triggering QTP test scripts remotely using Jenkins in this article. (If you have not read that, I recommend you to read the article first).

In this article we will see how to display the QTP/UFT results in the Jenkins console.

I have more than 5000 automated test scripts which need to be executed very often remotely which takes more than 8 hours to execute. When I first used Jenkins to run my QTP test scripts in the remote slave machines, I had to wait for several hours to see the results. It was very annoying.

The setup was given below.

Solution:

Runner.vbs uses QTP’s Automation Object Model (COM API) to launch, open the test & run..etc.
Runner.vbs once launches QTP, QTP runs as a separate process -information is not exchanged between Runner.vbs & QTP.
As Runner.vbs uses QTP’s COM API, it can access QTP’s Environment object using which the information can be exchanged.

Updated Runner.vbs:

We will modify the Runner.vbs slightly to display the results in the console.

'Create QTP object
Set QTP = CreateObject("QuickTest.Application")
QTP.Launch
QTP.Visible = TRUE

'Open QTP Test
QTP.Open "QTP Test Path", TRUE 'Set the QTP test path

'Create Environment Variables to pass the results from QTP to runner.vbs
QTP.Test.Environment.Value("JenkinsFlag") = "N"
QTP.Test.Environment.Value("JenkinsTestCaseDescription") = ""
QTP.Test.Environment.Value("JenkinsTestCaseResult") = ""

'Set Result location
Set qtpResultsOpt = CreateObject("QuickTest.RunResultsOptions")
qtpResultsOpt.ResultsLocation = "Result path" 'Set the results location

'Run QTP test
QTP.Test.Run qtpResultsOpt, FALSE

'Write the result in the console
While QTP.Test.isRunning
	If QTP.Test.Environment.Value("JenkinsFlag") = "Y" Then
		QTP.Test.Environment.Value("JenkinsFlag") = "N"

		' Show TC ID and Description
		WScript.StdOut.WriteLine QTP.Test.Environment.Value("JenkinsTestCaseNumber") & " : " & QTP.Test.Environment.Value("JenkinsTestCaseDescription") & " : "

		'Wait till the test is executed & result is updated
		While (QTP.Test.Environment.Value("JenkinsTestCaseResult") = "" AND QTP.Test.isRunning)
				WScript.Sleep 1000
		Wend

		'Show the Result
		WScript.StdOut.Write QTP.Test.Environment.Value("JenkinsTestCaseResult")
	End If
	WScript.Sleep 1000
Wend

'Close QTP
QTP.Test.Close
QTP.Quit

Changes to be done in QTP:

We will also need to do some changes in QTP’s test to set those Environment variables.

Whenever the test starts – set these variables.

JenkinsFlag is used to tell jenkins that a new testcase starts.

JenkinsTestCaseDescription is used for the test case description.

Environment.Value("JenkinsFlag") = "Y" 
Environment.Value("JenkinsTestCaseDescription") = "Verify Google search results"

Runner.vbs will look for these variables continuously. As soon as JenkinsFlag is set to Y, it will display the testcase description in the console which user can see in Jenkins.

At the end of the test, update the result as given below.

Environment.Value("JenkinsTestCaseResult") = "PASS"

I am just giving you an idea here – I do not update this for each & every testcase. It is all part of my framework.

Jenkins Console:

Our Runner.vbs is modified to show the results & as given below – I do not have to wait to see the results anymore – as and when it executes, i will get the results immediately by accessing Jenkins job’s console.

Summary:

This post gives only a very high level idea about the implementation. Check the below article for more details and the scripts to download.

Github / SVN integration for UFT scripts

Happy Testing 🙂

16 thoughts on “QTP/UFT – Display Test execution progress in Jenkins Console”

Prasad says:

July 31, 2015 at 5:52 AM

Thanks

Reply
Guru says:

August 19, 2015 at 7:14 AM

Hi, My QTP test uses a Keyword driven framework. There is a driver script which reads each test case from excel and executes it. Excel sheet will be updated for Pass or Fail. Should i set the below Environment variables in each and every functions in my QTP test?

Environment.Value(“JenkinsFlag”)
Environment.Value(“JenkinsTestCaseDescription”)

Reply
1. vIns says:
  
  August 24, 2015 at 4:57 AM
  
  I am using a Hybrid framework & I want to report each testcase result as my automation suites has hundreds of testcases. Each testcase will have many steps. I do not need every step result in the jenkins console.
  
  But in your Keyword driven framework, as you write your entire testcases in a spreadsheet & driver is responsible for reading & executing it, You need to just include those environment variables in the driver script. Once a step is executed, you update these variables in driver before reading the next step.
  
  Reply
Marcelo Campusano says:

August 26, 2015 at 11:50 PM

Thanks.
One comment : QTP.Test.Environment.Value(“JenkinsTestCaseResult”) has the value placed on execution meanwhile the QTP.Test.isRunning. Your last execution could not have the correct value

Reply
1. vIns says:
  
  August 27, 2015 at 10:24 PM
  
  True. It happens for the last test. I have added a ‘wait 1’ after setting the value for QTP.Test.Environment.Value(“JenkinsTestCaseResult”) to make it work.
  
  Reply
Vasu says:

August 31, 2015 at 8:46 AM

Very Good Article

It is working fine with few issues

I am using Hybrid Framework and i have configured all the Environment Variables as mention by you. But my First Test Case is not displaying the Pass/Fail Status and for rest all test cases i am getting it successfully. Could you please guide me. Here is my Out Put

Building in workspace C:\Jenkins\workspace\ICL_Sanity_Test
[ICL_Sanity_Test] $ cmd /c call C:\WINDOWS\TEMP\hudson3178053962083818972.bat

C:\Jenkins\workspace\ICL_Sanity_Test>C:\Windows\SysWOW64\cscript.exe Sanity.vbs
Microsoft (R) Windows Script Host Version 5.8
Copyright (C) Microsoft Corporation. All rights reserved.

TC01 : Create Contact in CLC : .
TC02 : Update Contact in CLC : Passed.
TC03 : Add Contact to Location Relationship : Passed.
TC04 : Search Contacts in CLC : Passed.
TC05 : Create Location in CLC : Passed.
TC06 : Update Location in CLC : Passed.

HERE IS MY SANITY.VBS

‘Create UFT object

Set App = CreateObject(“QuickTest.Application”)
App.Launch
App.Visible = True
App.WindowState = “Maximized”
App.ActivateView “ExpertView”

‘Open UFT Test
App.Open “C:\Users\AB82164\Desktop\ICLAutomation\SanityDriver”, True

‘Create Environment Variables to pass the results from QTP to runner.vbs
App.Test.Environment.Value(“JenkinsFlag”) = “N”
App.Test.Environment.Value(“JenkinsTestCaseDescription”) = “”
App.Test.Environment.Value(“JenkinsTestCaseResult”) = “”

‘Set Result location
Set qtpResultsOpt = CreateObject(“QuickTest.RunResultsOptions”)
qtpResultsOpt.ResultsLocation = “Result path” ‘Set the results location

‘Run QTP test
App.Test.Run qtpResultsOpt, FALSE

‘Write the result in the console
While App.Test.isRunning
If App.Test.Environment.Value(“JenkinsFlag”) = “Y” Then
App.Test.Environment.Value(“JenkinsFlag”) = “N”

‘Show TC ID ,TC Name and Result
WScript.StdOut.WriteLine App.Test.Environment.Value(“JenkinsTestCaseNumber”) & ” : ” & App.Test.Environment.Value(“JenkinsTestCaseDescription”) & ” : ” & App.Test.Environment.Value(“JenkinsTestCaseResult”) & “.”

End If
Wend

App.Test.Close
App.Quit
Set App = Nothing

Reply
1. vIns says:
  
  August 31, 2015 at 7:40 PM
  
  How long each testcase takes to execute?
  Sanity.vbs & QTP are 2 different processes & they need to talk each other. You need to set enough ‘wait’.
  
  Reply
Vasu says:

September 1, 2015 at 10:40 AM

Thanks for your Reply bro.
Mine is a SOAP Services Framework and it takes less than 10 seconds for each test case

Reply
1. vIns says:
  
  September 2, 2015 at 1:55 AM
  
  See… You need 2 while loops. One outer loop which keeps executing as long as QTP runs.
  inner loop is for updating the result. Wait for App.Test.Environment.Value(“JenkinsTestCaseResult”) to be not null
  
  Reply
Vasu says:

September 2, 2015 at 5:14 AM

Thanks bro

I have update the script like below as per your comment

While App.Test.isRunning
If App.Test.Environment.Value(“JenkinsFlag”) = “Y” Then
App.Test.Environment.Value(“JenkinsFlag”) = “N”

‘Wait till the test is executed & result is updated
While (App.Test.Environment.Value(“JenkinsTestCaseResult”) = “” AND App.Test.isRunning)
WScript.Sleep 1000
Wend

‘Show TC ID ,TC Name and Result
WScript.StdOut.WriteLine App.Test.Environment.Value(“JenkinsTestCaseNumber”) & ” : ” & App.Test.Environment.Value(“JenkinsTestCaseDescription”) & ” : ” & App.Test.Environment.Value(“JenkinsTestCaseResult”) & “.”

End If
Wend

MY Result Comes like this. No First test case, only second one displays twice 🙁

TC02 : Update Contact in CLC : Passed.
TC02 : Update Contact in CLC : Passed.
TC03 : Add Contact to Location Relationship : Passed.
TC04 : Search Contacts in CLC : Passed.
TC05 : Create Location in CLC : Passed.

Reply
Sanjay Aggarwal says:

December 9, 2015 at 8:20 AM

Is there any way to create defects in Jira automatically for failed scripts. We want to implement in such a defects are automatically created for failed scripts. when we have a new build then if failed scripts gets passed then defects are auto updated accordingly.
Please suggest.

Regards,
Sanjay

Reply
1. vIns says:
  
  December 9, 2015 at 2:37 PM
  
  Jira has API exposed for most of the functionalities. So It is possible to create defects/update existing defects programmatically. You need to check with your Jira admin to access the API.
  
  Reply
TelcoGuy says:

January 18, 2016 at 8:27 PM

Hı mate,

I’m trying to initiate a similar scenario but in linux instead of windows. Could you please guide me? I need to execute several scripts and show the outputs of those scripts in console output or email report via email plugin and so on.. Biggest problem I am having at the moment is that, when I execute a shell script to check mount points lets say, build passes successfully but I do not see any results in console output. So this is basically my problem.. Any ideas would be appreciated very much!

Thanks in advance

Reply
1. vIns says:
  
  January 25, 2016 at 10:50 PM
  
  Sorry that i do not understand your question. If possible, please give more information.
  I would advise you to raise your question in StackOverFlow.com to get the help immediately.
  
  Reply
JM says:

September 2, 2016 at 12:44 AM

I have my UFT on VM and I am trying to hookup with Teamcity server on cloud , I am running into 2 issues
1. When I run above vb script(runner.vb) from command prompt , it does not displays output in command window but runs the test successfully and generate result.xml etc
2. When i call this script from teamcity server it fails where another scripts works fine .

Thanks in advance .

Reply
1. vIns says:
  
  September 2, 2016 at 11:34 AM
  
  Take a look at this [ http://www.testautomationguru.com/qtpuft-jenkins-github-svn-integration/ ] . I have shared the sample code in github to play around.
  
  Ensure that you use CScript command. CScript runner.vbs
  
  Reply