Curated by Vinay Prabhakar Minj
Gartner has estimated that by 2021, 50 percent of enterprises will leverage intelligent test automation driven by AI and machine learning
Today a lot of investment is going into digitisation of retail industry, resulting in the emergence of many e-commerce related websites. However, the industry statistics released by Forbes a few months back showed that about 90 percent of the sales still happen through stores and only 10 percent happen through online. It is even worse when it comes to grocery, with only two percent of the total sales happening through online channel.
On the other hand, satisfaction levels have come down for customers visiting retail stores, according to a recent American Customer Satisfaction Index (ACSI) report.
There are two parameters that need to be considered when you look at customer satisfaction from a store/retailer’s point of view: the quality of service and the speed of transaction. Based on these parameters, test automation is one of the solutions that can help in the overall improvement of customer satisfaction.
Gartner has also predicted that by 2023, 75 percent of IOT system integrators will have added comprehensive end-to-end IOT solution testing services to their portfolios, up from the current 30 percent.
Test automation trends
Test automation tools have been there for two decades, but today it has become more advanced.
As per the latest statistics by Forrester, 14 percent of customers the firm surveyed have achieved more than 80 percent of automation in the enterprise level.
But there are customers who are spending a huge amount of time and effort in building solutions that are built on Artificial Intelligence (AI) and Machine Learning. At the same time, there are a lot of organisations that have taken up digitisation and decided to adopt Agile and DevOps. Despite all these efforts, they still face challenges in automation. You won’t exactly get the benefits of Agile/DevOps if you don’t have enough test automation coverage within your landscape.
And it’s a bit worse in the retail segment primarily due to the challenges with respect to hardware.
According to latest World Quality Report, organisations are still unable to tap the full benefits of Agile/DevOps due to low levels of automation and challenges with test data/test environment. Many customers also do not have the right set of tools for testing.
Multiple devices to deal with for automation for POS system
A typical retail store today may have different types of devices. For instance, many retail stores in the U.S have mobile self-checkout solutions which enables you to scan items and make purchases through mobile. In some stores, one can also see fuel pumps and ATM machines where purchases can also be made. If you want to automate testing of point of sales, then you have to deal with different types of devices that are present in a retail store.
Apart from these devices, various flavours of point of sale (POS) software are coming into the market. Toshiba has an operating system called 4690 (initially called IBM 4690) which is one of the most popular operating systems for point of sale. And there are some new players like GK Software, NCR, PCMS, Wincor-Nixdorf and several others. There may be around 50 players in the market who have POS products.
There are at least 50 – 60 different automation tools available in the market, but these tools do not directly support automation of point of sales. This is one of the reasons why many customers are not able to achieve success in automating of POS.
The challenge here is that the product they apply in POS is going to be different as well as the operating system on which its runs. For example, NCR runs on Windows, Toshiba runs on its own custom-built operating system. So, you need to consider the various operating systems, as well as the applications when trying to automate point of sales.
Devices such as debit card readers, receipt printers, scanner scales, barcode scanners, display screen and handheld scanners that are used in a retail store must also be considered when doing test automation for a point of sale system.
Popular tools do not help automate POS
There are at least 50 – 60 different automation tools available in the market, both open-source based as well as commercial based, such as Tricentis, HP UFT, Microsoft’s Visual Studio, TestComplete, Selenium and Appium. However, these tools do not directly support automation of point of sales. This is one of the reasons why many customers are not able to achieve success in automating of POS.
Testing should be aligned with real world scenarios
Whenever you are doing testing, whether automated or manual, it is better to do it aligned with real scenarios, i.e. exactlylike a customer might do in a store. For example, you can use simulators that replicate real-world scenarios of using a magnetic strip/chip-based cards, scanning a barcode or weighing an item on a machine.
It is not just the point of sale system which needs to be tested. POS has numerous interfaces with in-house and third-party systems such as memberships, promotions, pricing, reporting, inventory and payments. None of these are built using the same technology. For example, the pricing engine might be mainframe-based system and the payment system might be a mobile/tablet-based application. Moreover, the technology can also be a mix of open source versus Microsoft stack or packaged application versus custom. This is another challenge that one may have to deal with in test automation.
So, testing of scenarios is required for an entire workflow where you buy/procure an item from the market, price it and set it to a point of sale. When the customer comes in, he/she will swipe his/her membership card to make a purchase and for which promotions will be applied with data going into the reporting system. This complete testing of the end-to-end workflow using your automation tool can be quite challenging.
To solve this problem, we will need an automation framework and a tool, a set of microcontrollers which will help to simulate or perform actions, a robotic arm, a KVM switch and a receipt comparison engine to validate the receipts which come out after the transaction is complete. An automation framework can also have other components like a reporting layer or reporting tool that helps report status to stakeholders.
Building blocks of POS test automation solution include:
- Microcontrollers for triggering events and KVM switches for event transmission
- Robotics arm for capturing receipts and performing actions on debit readers
- Automated validation of receipts using computer vison APIs
- Test reporting platform with predictive analytics for real time status reporting
OCR based automation tools are best suited for POS automation
Test automation tools can be classified into two categories: –
- Object-based test automation tools which take information about the UI elements within the screen.
- Tools that work on image recognition or Optical Character Recognition (OCR).
TestPlant and rhiscom are popular tools, which work with Optical Character Recognition that provides access to the machines that are running point of sales software.
KVM (Keyboard, Video, Mouse) switches are used to control the application/device. They are used for multiple purposes like sending command to the UI as well as receive updates from UI for remote application. KVM switches are primarily used in a networking environment consisting of 5-6 servers. It might not be required if you are using a Windows-based OS, but it is a must-have thing if are using Toshiba.
Microcontrollers are used for mimicking keyboard strokes. With help of microcontrollers, one can send keystrokes to a keyboard. Microcontrollers can also help in performing barcode scanning.
The robotic arm can be used to simulate the actions of a cashier/customer doing transactions such as swiping/inserting a card, entering of PIN number and signatures as well as receipt capture.
Receipt validation is very important. Once the receipt is printed on POS machine, you need a solution to validate the correctness of details printed on it. For this, you need a combination of a robotic arm and a camera. The purpose of the robotic arm is to basically hold the receipt in such a manner so that you can take a good image of the receipt and validate it. The photo is saved as JPEG and OCR comparison is done using computer vision APIs. The same solution can be used to compare labels.
Manual testers are still going to stay. In case of regression testing, 90-95 percent of the process can be automated, but for new enhancements you still need the involvement of manual testers.
Also, there should be a mechanism to report the status. You will basically have a lot of inputs coming in, for example data coming from the Automation Framework, tests cases from the Test Management Tool and defects from the Defect Management. All these inputs can be stored in a database and then by creating a dashboard using power BI, you can present it to your customer or stakeholders in multiple ways. The interface that has been created can be applied in three different ways :-
- Dashboard that runs on mobile
- Dashboard that can be hosted on a cloud, on-premise solution
- Chatbot to get the status of your release or application. It also gives insights about testing and test cases
Automation helps to improve customer experience and speed to market
By implementing this for customers, a lot of cycle time can be saved during regression testing (approx. 70 percent). Secondly, you can contain the defects going into production. This improves test coverage and reduce regression testing cost. While a manual tester can execute 30-40 regression test cases in a day, a single robotic arm can execute 100 test cases during the same time. Thus, one cycle of regression testing can be completed in a day, 24 x 7.
This 24 x 7 automation will not replace humans (testers) by any chance. As a tester, one needs to know what to test and to do this, artificial intelligence, machine learning or any other sort of technology can’t be implemented. Only when you have a test case or a scenario defined, automation can be applied. But rather than worrying about job losses due to it, it is very important to do upskilling such as learning programming languages like Java, C# or Python so that these can be applied to test automation.
Having said so, manual testers are still going to stay. In case of regression testing, 90-95 percent of the process can be automated, but for new enhancements you still need the involvement of manual testers.
About the author
This article is an extract from a speech presented by George Ukkuru, Head of Quality Engineering, UST Global, at IOTSHOW.IN 2019.