Testing in DevOps

Testing in DevOps A GUIDE FOR BUYERS

Full Page Ads_SDT054.qxp_Layout 1 11/12/21 4:54 PM Page 24

6RIWZDUH WHVWLQJ WRROV VXSSRUWLQJ D 'HY2SV DSSURDFK

8, 7HVWLQJ 3HUIRUPDQFH 7HVWLQJ $3, 7HVWLQJ 'DWD )DEULFDWLRQ 6HUYLFH 9LUWXDOL]DWLRQ

5HTXHVW D IUHH WULDO QRZ +&/VZ FR 2QH7HVW7ULDO

025-34_SDT054 for PDF.qxp_Layout 1 11/19/21 5:14 PM Page 25

www.sdtimes.com

December 2021

SD Times

Testing in DevOps

BY JAKUB LEWKOWICZ

T

esting in DevOps is as much about the people that are behind the tools as it is about the tools themselves. When they work in synchrony, organizations can see major benefits in the quality of their applications and SDLC process.

However some organizations still struggle with how to advance their DevOps testing initiatives because they are also implementing containerization, microservices, and other cloud-native methods that can sometimes complicate the environment.

In some organizations, those responsible for testing need to keep up with changes forced onto them by other teams, third-party applications, and platforms and also keep up with the growing list of regulatory compliance.

Since most of the applications rest on the cloud, businesses also must quickly react when cloud-based platforms receive updates.

The demand for speed and quality has prompted organizations to look towards a way to automate many of the facets of testing and changing the way that they define value.

“DevOps requires that testing is fast, accurate, meaning low false positive and low false negative rates, and runs without human intervention. Fast can be achieved with more compute power but for the tests to be accurate they need to handle the dynamic and evolving nature of modern applications,” said Gil Sever, co-founder and CEO of Applitools.

Traditional test automation requires frequent and human intervention to update the tests through assertions and

navigation, but AI has the ability to learn how the application behaves and respond appropriately, reducing the human intervention. “This makes AI essential for modern software development teams to keep pace with increased release frequency,” Sever added.

But shifting everything to the DevOps mentality of automation is not an overnight process and in some cases, the ideal delivery story won’t even apply to every company or any project, according to Marcus Merrell, Senior Director of Technology Strategy at Sauce Labs.

“Not all systems can do true

DevOps,” said Gareth Smith, general manager of Keysight Technologies. “If I am building a retail website, and it just requires a simple thing, then that’s fine.

But if I’m rolling out something that needs to work with various IoT connectors, then not all platforms are able to automate all that.”

QA brings all hands on deck for testing Quality engineering is being elevated because the C-level sees quality engineering as a key enabler. While developers used to throw things over the wall to QA, they’re bringing QA into the concontinued on page 4 >

3

025-34_SDT054 for PDF.qxp_Layout 1 11/19/21 5:14 PM Page 26

4

SD Times

December 2021

www.sdtimes.com

< continued from page 3

versation and the industry is seeing much more collaborative DevOps teams, where quality is a shared responsibility between developers and QA and even product owners, according to Dan Belcher, co-founder of mabl.

The interweaving of the maintenance and automation aspects of testing with the speed of DevOps has led to the new term QAOps. “Much in the same way that we would think of shifting left as looking at those defects early on because they are then cheaper to fix, now it’s a much greater level of having the whole structure of QA early on and throughout the DevOps cycle,” Belcher said.

Belcher added that now the CTOs are driving the transformations. “Now it’s a mandate coming from the C-level, to make investments in quality engineering to enable these transformations, whether it’s digital, or DevOps, or UX.” While many large organizations keep a central QA department, we’re seeing more and more of a shift to automation developers and manual testers being assigned to individual Squads, with a Center of Excellence to support the tools. This allows testers to remain focused on business needs and not worry so much about test infrastructure or tooling, according to Merrell.

While there are still people in the organizations who are responsible for testing as part of their job title, it has also become much more of an all-hands on deck approach in DevOps. In leading organizations, software quality has become everyone’s responsibility and has expanded beyond “does it work” to “is it the best customer experience”. Developers are increasingly involved, as well as others such as UI/UX designers and domain experts, to ensure the digital experience is not only working but that it is delivering on the goals of the business, according to Applitools’ Sever.

“This approach of having all hands on deck is beneficial because with the fast feedback cycles of DevOps, it’s much easier for a developer to understand the impact of a change that they’ve made, possibly before it’s gone

through a dedicated QA cycle,” said Chris Haggan, product management lead at HCL OneTest.

AI and automation are key components of testing in DevOps AI automation tools are necessary to provide insight by ingesting data from a plethora of data sources.

“Once you move to automated testing and a more integrated process, it enables you to check on things every step of the way and see whether you’re still on the right track,” said Joachim Herschmann, senior director and analyst on the Application Design and Development team at Gartner. “I can see the direct impact that my development, bug fixing and enhancements have whether they improve or make it worse.”

The more data that can be thrown at AI, the better the result is because it includes all of the subtle variants and different data from all the different sites that one connects it to.

“You can also use it right now to auto generate the test asset universe, what we refer to as the digital twin,” Keysight’s Smith said. Users of the ‘digital twin’ can define what type of test they want and the AI will work out what the best test scenario for that situation is.

Execution speed can be increased by assigning more resources to the problem, and the key benefit to AI is its ability to learn and improve the tests over time with minimal human intervention, Applitools’ Sever said.

There are several areas where AI has the potential to help with testing: smart crawling, although it is still in its infancy; self-healing, which is already well

established and understood; and visual validation.

“For visual validation to be effective, it must be accurate to ensure the team is not overwhelmed with false positives — a problem with the traditional pixel-based approach. It needs to handle dynamic content, shifting elements, responsive designs across different screen sizes and device/browser combinations — as well as provide developers and testers ways to optimize the review and maintenance of regressions,” Sever said.

Automation can also help with typically manual-centric types of tests such as UX testing. UX testing still requires manual input because here the outcomes of a test are subjective. However, testers don’t need to run the tests manually for every device because they can watch tests being run on a desktop app and then decide whether the quality is acceptable or not in an assisted manual testing fashion, mabl’s Belcher explained.

“A real simple example is if I’m halfway through entering my credit card details, and I talk to somebody, I roll forward my device, my device goes flat, it rotates and then I come back.

Now with that accidental rotation of the device and back, does that still work,” Keysight’s Smith said. “And in many cases, that particular use case or between those, between filling in field six and field seven on a form, then you rotate the device; no one will test that particular combination, but those happen in the real world. That’s where AI can help look at those different combinations as you’re going through the usual continuous tests.”

DevSecOps now a top priority One of the biggest trends of 2021 is that security became a top priority for testing in the wake of massive breaches that resulted in tremendous costs.

The Executive Order on cybersecurity that the Biden administration signed in May helped to put security awareness in the spotlight, according to Jeff Williams, the co-founder and CTO of Contrast Security. continued on page 8 >

Full Page Ads_SDT054.qxp_Layout 1 11/17/21 10:37 AM Page 27

Disrupting the Economics of Software Testing Through AI: An interview with Torsten Volk EMA (Enterprise Management Associates) recently released a report titled “Disrupting the Economics of Software Testing Through AI.” In this report, author Torsten Volk, Managing Research Director at EMA, discusses the reasons why traditional approaches to software quality cannot scale to meet the needs of modern software delivery. He highlights 5 key categories of AI and 6 critical pain points of test automation that AI addresses.

What’s wrong with the current state of testing?

Many software development teams are struggling to deliver on the promise of Agile / DevOps and are faced with numerous technical challenges, such as rising application complexity and explosion of browsers / devices. Multiply this by the constant drive for faster releases to deliver increased customer value — without the ability to hire additional quality control staff — and you can quickly see why the traditional approach to software testing can no longer scale to keep up. We need to optimize the process with AI to eliminate the mundane and repetitive tasks and ultimately ensure business success.

How does AI help with software testing?

There are five key capabilities AI provides to help improve human efficiency: smart test creation, self healing, coverage detection, anomaly detection, and visual inspection. In the report, I discuss the six critical pain points where these capabilities are delivering ROI today: false positives, test maintenance, inefficient feedback loops, rising application complexity, device sprawl, and tool chain complexity.

Of the capabilities available, AI-driven visual inspection has the broadest reach and highest impact. This discipline aims to provide test engineers with an additional “pair of eyes,” leaving the engineer to focus on activities that really need human intelligence. It provides humans with the contextual information needed to accelerate their test and remediation efforts, recommending solutions wherever necessary and remembering human decisions.

“The world’s top brands are already adopting AI to transform their software testing ... focusing on the ROI gained from the speed and scalability of leveraging AI in the development process.”

You talk about visual inspection having the highest impact. What pain points does it address?

Traditional pixel-based comparison attempts to perform visual inspection but is plagued with false positives. Training deep learning models to inspect an application through the eyes of the end-user removes a lot of the mundane repetitive tasks that cause humans to be inefficient. Due to its accuracy, AI-driven visual inspection can enable teams to create a stable suite of automated tests with reduced false positives. Visual inspection can be further leveraged to drive cross-browser / cross-device validation at scale.

How are people adopting AI? Do I need to hire AI experts or develop an AI practice?

The world’s top brands are already adopting AI to transform their software testing, but they’re focusing on the ROI gained from the speed and scalability of leveraging AI in the development process — not developing the AI itself. There are solutions on the market today that give you the ability to take advantage of fully trained models and several can be layered on top of your existing test automation. One of the solutions that seems widely adopted for visual inspection is Applitools, which is already trained on +1B images and delivers extremely high levels of accuracy.

How does the role of the developer / tester change with the introduction of AI? Developers and testers still need to make a decision about what and how something should be automated. AI minimizes the mundane and repetitive tasks, freeing the engineers to do more creative, interesting and valuable work.

For example, what happens when a tester is reviewing thousands of webpages for accuracy? Some pages have differences, but they don’t matter. AI can filter those out and highlight only the subset that needs review.

Auto-classification, grouping issues together, is another great example of being able to reduce your work and improve accuracy. If I’m looking at what looks like the same problem over and over again, I might miss a defect. Whereas if the AI categorizes and groups similar issues together it greatly reduces the possibility of a missed defect.

What’s the future state for software testing, and where do we start?

Autonomous testing is the vision for the future, but we have to ask ourselves, why don’t we have an autonomous car yet?

It’s because today, we’re still chaining together models, and models of models. We’re striving to get to the point where AI is taking care of all of the tactical and repetitive decisions and humans are thinking more strategically at the end of the process, where they are more valuable from a businessfocused perspective. I would recommend starting with the mature solutions available today in the areas of visual inspection and self healing — keeping an eye on the future and ongoing evolution of AI.

If you are interested in reading the full report you can download it here:

Full Page Ads_SDT054.qxp_Layout 1 11/12/21 4:55 PM Page 28

O Ik _

c k

c

_ |

_ A

c c

[ [ A 1

k 1 O I c

q _ 1 k

#S F B L U I S P V H I

"Q Q M J D B U J P O

4F D V S J U Z

G P S

4F S W F S M F T T

& O W J S P O NF O U T

$P OU S BT U

4F S W F S M F T T

"QQM J D B U J P O 4F D VS J U Z

J T

Q V S Q P T F

C V J M U

B T

Q B S U

P G

B

V O J m F E

T F D V S J U Z

Q M B U G P S N P G G F S J O H

C V J M U

U P

H F U

T F D V S F

D P E F

NP W J O H

U I S P V H I

U I F

F O U J S F

E F W F M P Q NF O U

Q J Q F M J O F

B O E

D P O U J O V P V T M Z

Q S P U F D U

B D S P T T

U I F

D P NQ M F U F

T P G U XB S F

M J G F D Z D M F

$P O U S B T U

T

E F W F M P Q F S

G S J F O E M Z

B Q Q S P B D I

U P

T F S W F S M F T T

B Q Q M J D B U J P O

T F D V S J U Z

U F T U J O H

J O D M V E F T

Q J Q F M J O F

O B U J W F

B V U P O P NZ

B O E

B V U P NB U J P O

0S H B O J [ B U J P O T

H B J O

D P NQ M F U F

T F D V S J U Z

W J T J C J M J U Z

G P S

" 84 B NC E B

G V O D U J P O T

XJ U I

O F B S

[ F S P

G B M T F

Q P T J U J W F T

6T F

$P O U S B T U

4 F S W F S M F T T

U P

B D I J F W F

$P NQ S F I F O T J W F

P C T F S W B C J M J U Z "V U P NB U F E

B Q Q M J D B U J P O

T F D V S J U Z 4F B NM F T T

S B Q J E

E F Q M P Z NF O U

B O E

NB O B H F NF O U

7J T J U

D P O U S B T U T F D V S J U Z

D P N T F S W F S M F T T

U P

M F B S O

NP S F

025-34_SDT054 for PDF.qxp_Layout 1 11/19/21 5:14 PM Page 29

www.sdtimes.com

December 2021

SD Times

How does your solution help organizations test applications in their DevOps environments?

Gil Sever, co-founder and CEO of Applitools Applitools is helping over 400 of the world’s top digital brands accelerate the delivery of visually perfect digital experiences across all browsers, devices and screens through AI-powered test automation.

Trained on 1B+ images to deliver 99.9999% accuracy, Applitools’ Visual AI mimics the human eye and brain to deliver reliable full page validation that integrates into your existing test automation — with 50+ SDKs supporting open source frameworks (such as Selenium, Cypress, Playwright, Appium, etc.) and integrations with commercial test automation tools.

Applitools Eyes provides users with the ability to perform complete validation of the end user experience with a single line of code. Tests utilizing Applitools are 5.8x faster to create, 3.8x more stable and catch 45% more defects.

The Applitools Ultrafast Test Cloud combines Applitools Eyes with the Applitools Ultrafast Grid to deliver a modern approach to cross-browser/cross-device testing that executes tests 18.2x faster than legacy cloud execution grids or device farms.

Applitools’ Visual AI modernizes critical test automation use cases — functional and visual regression testing, web and mobile UI/UX testing, cross browser/cross device testing, localization testing, PDF testing, digital accessibility and legal/compliance testing — to transform the way businesses deliver innovation at the speed of DevOps without jeopardizing their brand.

Jeff Williams, co-founder and CTO of Contrast Security We provide a platform of products that are designed to help companies become good at building secure code, doing it fast and reliably.

And we do it by giving instant feedback to the folks that need it through the tools they’re already using.

Unlike scanners that plow through your whole application portfolio, Contrast runs in the background, a lot like an APM tool. It gathers a ton of telemetry across all your applications in parallel — APIs, cloud-native, and serverless functions — brings that all together, and gives you dashboards to show you exactly what you need.

Most developers don’t really want another dashboard, what they’d really like, is their security results, right in JIRA or they’d like to fail a build or get Slack alerts or in their IDE. There’s a million ways to consume the data that we generate, but I think the most important thing is that we have super accurate data based on observing the actual application run. We’re not guessing about vulnerabilities.

We offer Contrast Assess, which runs within the application

and uses instrumentation to find vulnerabilities in your custom code and in your libraries.

We also have Contrast OSS, which finds the known vulnerabilities in all your open source so that you’re not using libraries that have known vulnerabilities.

Then we added Contrast Protect.

It’s the same instrumentation approach, but now we applied it in production so that it’s super high performance and it prevents vulnerabilities from being exploited.

We also added Contrast Scan, which is a static analysis tool with a new algorithm called demand-driven static analysis, making it much more efficient at finding vulnerabilities and you can run it in your pipeline. As a result of the tremendous uptake in serverless, we launched our first security for serverless offering for AWS Lambda.

Gareth Smith, general manager of Keysight Technologies Using artificial intelligence (AI), machine learning (ML) and real user data, Keysight’s Eggplant solution automates test creation and execution.

The Eggplant Digital Automation Intelligence (DAI) platform tests and monitors user interface (UI) performance to improve software development, enhance quality, and elevate the customer experience at DevOps speed.

Instead of testing the code, the DAI platform focuses on the end-to-end customer experience. It provides teams with unparalleled intelligence on where problems lie, significantly reducing the time to resolve these issues. This means organizations can meet customer experience demands and continuously deliver innovation faster while devising strategies to expand DevOps. Customers across aerospace and defense, automotive, education, financial services, healthcare, retail, and telecoms rely on the intelligent automation. The DAI platform automates over 95% of activities, including test-case design, test execution, and results analysis. This allows teams to rapidly accelerate testing and integrate with DevOps at speed.

As environments grow more complex and interconnected and with workers distributed, organizations need continuous intelligent test automation that is easy to integrate and scale. Keysight Technologies’ Eggplant automation helps businesses rapidly create products that delight users, test the entire customer experience across any technology, and predict the quantified impact of new product versions on the user before release.

By partnering with Keysight Technologies, enterprises can deliver better software at a faster pace that delights users. continued on page 8 >

7

025-34_SDT054 for PDF.qxp_Layout 1 11/19/21 5:14 PM Page 30

8

SD Times

December 2021

www.sdtimes.com

How does your solution help organizations test apps in DevOps? < continued from page 7 Chris Haggan, Product Management Lead, HCL OneTest A product that is rushed to market with little time for quality assurance can massively damage the reputation of even well-established organizations. Adopting new technologies, and the fastpaced work environment driven by users who expect more from the applications they work with, will not change. It is time to find testing solutions that evolve with changing landscapes.

HCL OneTest supports UI, performance and API testing along with synthetic data generation and service virtualization to help meet the challenge of testing highly-integrated and complex applications. It features a script-less, wizard-driven, test authoring environment, and supports more than 100 technologies and protocols. HCL OneTest helps with the connections and dependencies between services and components to plan integration test strategies, and generates coverage reports to help identify which processes and services require further testing. Together, these HCL OneTest components help automate and run tests earlier and more frequently to discover errors sooner when they are less costly to fix.

To achieve a successful DevOps strategy, software testing teams must automate regression testing to reduce the risk of deploying poor quality software into production. Effective test automation includes application programming interface (API) testing, user interface testing, and overall system testing.

Employing service virtualization in conjunction with test automation allows these tests to be executed earlier, while covering a wider range of scenarios.

Dan Belcher, co-founder of Mabl We’ve seen a profound shift in how organizations view software testing and quality assurance. Historically QA received less focus and investment than other functions, but that is changing: CTOs and engineering leaders are looking to quality engineering as a key enabler of DevOps and digital transformation, which require a broader mandate to ensure that quality is embedded deeply throughout the soft-

< continued from page 4

“I think it’s a real harbinger of better security for apps in the future that they require a minimum standard for AppSec testing, much-improved visibility into what you’ve done to secure your code, including things like security labels,” Williams said. “I look forward to a day when you can go to your online bank, insurance company, social media, or your election system and if you want to know a little bit about how that software was built, and how it was tested for security, it should be available to you; that should actually be a

ware delivery pipeline. Mabl is the only test automation platform designed to fulfill this new mandate in the enterprise.

Mabl features a low-code UI and framework that allows everyone, regardless of coding experience, to create automated tests with 80% less effort, spanning web UIs, APIs, and mobile browsers. Using artificial intelligence, mabl reduces test maintenance with autohealing, which detects and adapts to changes automatically. With functional test creation and maintenance streamlined, QE can spend time on broader quality attributes — including performance, accessibility, and UX — while keeping pace with DevOps. Mabl also integrates with popular tools such as Microsoft Teams, Slack, and Jira, so that users can incorporate testing information seamlessly into their workflows and benefit from rich diagnostic data from mabl. Rich reporting supports continuous improvement and improved collaboration across the software development pipeline by addressing one of the biggest inhibitors to DevOps: process changes. 1-in-3 development professionals cite the slow pace of change as their biggest DevOps challenge, making easy-to-adopt tools essential for success.

Marcus Merrell, Senior Director of Technology Strategy at Sauce Labs Continuous testing is a key enabler of digital confidence — the knowledge that you’re delivering the best possible user experience to your customers. Digitally confident organizations know that their web and mobile applications look, function and perform exactly as intended, every single time they’re used.

Sauce Labs gives companies the confidence to deliver a flawless digital brand experience to their customers. The Sauce Labs Continuous Testing Cloud is designed to quickly identify code errors, accelerating the ability to release and update web and mobile applications that look, function and perform exactly as they should on every browser, operating system and device, every single time. Sauce Labs dramatically reduces the time and effort required to discover and fix errors using automated or manual tests, multiple frameworks, leading operating systems, and on real or virtual devices for faster, cleaner releases and more successful, trusted customer experiences. z

fundamental right. If you’re trusting your life, or your healthcare, or your finances, or your government to a piece of software, I think you have the right to know a little bit about how it was tested for security.”

However, security isn’t always handled with the utmost care at organizations. A lot of this comes down to a lack of security expertise, according to Williams.

There’s never enough attention being paid to security, in testing, or in development. As hard as test/security vendors work to keep up, the bad actors

always seem to be one step ahead — aided by the fact that they’ve been every bit as institutionalized as the products they’re subverting, according to Sauce Labs’ Merrell.

Security testing has traditionally required a lot of expertise to run tools such as SaaS or desktop scanners, or even SCA scanning tools.

“You can’t just take tools designed for security experts and hand them to developers early in the process and just say ‘Go,’” Williams said. “They’ll end up with tons of false alarms and tons of wasted continued on page 12 >

Full Page Ads_SDT053.qxp_Layout 1 10/27/21 9:49 AM Page 11

NEW NEW

NEW NEW

Full Page Ads_SDT054.qxp_Layout 1 11/17/21 10:38 AM Page 32

, W

V

< R X U

6D O H V I R U F H { , W

V

1R W

/ L N H

$Q \

2W K H U V

7 K H

Q H Z ( J J S O D Q W

6D O H V I R U F H

6R O X W L R Q

D X W R

J H Q H U D W H V

W H V W

D V V H W V

W R

D F F H O H U D W H

\ R X U

6D O H V I R U F H

D S S O L F D W L R Q

U H O H D V H V

$V V H V V

< R X U

6 D O H V I R U F H

, Q V W D Q F H

7 R G D \

_

H J J S O D Q W V R I W ZD U H

F R P V D O H V I R U F H

V R O X W L R Q

www.sdtimes.com

December 2021

SD Times

A guide to DevOps testing tools n

FEATURED PROVIDERS n

n Applitools: Applitools is built to test all the elements that appear on a screen with a single line of code. Using Applitools’ Visual AI, you can automatically verify that your web or mobile app both functions correctly and that the digital experience is visually perfect across all devices, all browsers and all screen sizes. Applitools is designed to integrate with your existing test automation rather than requiring you to adopt a new tool and supports all major test automation frameworks and programming languages covering web, mobile, and desktop apps.

n HCL OneTest: HCL OneTest provides UI, API, and performance testing, as well as service virtualization and synthetic data fabrication, to support testers throughout the project lifecycle. It features a script-less, wizard-driven test authoring environment and support for more than 100 technologies and protocols. HCL OneTest belongs to the Secure DevOps portfolio of HCL Software, which is a division of HCL Technologies (HCL). HCL Software develops, markets, sells and supports more than 20 product families in the areas of Customer Experience, Digital Experience, Digital Solutions, Secure DevOps, Security, and Automation.

n Contrast Security: Contrast Security is the industry’s most comprehensive Application Security Platform, removing inefficiencies and empowering enterprises to write and release secure code faster. The Contrast platform automatically detects vulnerabilities while developers write code, eliminates false positives, and guides fast vulnerability remediation, which enables application and development teams to collaborate more effectively. This is why many of the world’s largest organizations rely on Contrast to secure their applications in development and in production.

n mabl: mabl is the intelligent test automation company that empowers high-velocity software development teams to integrate automated end-to-end testing into the entire development lifecycle. Mabl users benefit from a unified platform for easily creating, executing, and maintaining reliable tests that result in faster delivery of high quality, business critical applications. Learn more at https://www.mabl.com; follow @mablhq on Twitter and @mabl on LinkedIn.

n Keysight Technologies: Keysight Technologies Eggplant Digital Automation Intelligence (DAI) platform is the first AIdriven test automation solution with unique capabilities that make the testing process faster and easier. With DAI, you can automate 95% of activities, including test-case design, test execution, and results analysis. This enables teams to rapidly accelerate testing and integrate with DevOps at speed.

n Sauce Labs: Sauce Labs is the leading provider of continuous testing solutions that enable customers to deliver digital confidence. The Sauce Labs Continuous Testing Cloud delivers a 360-degree view of a customer’s application experience, ensuring that web and mobile applications look, function, and perform exactly as they should on every browser, OS, and device, every single time.

n Appvance is the inventor of AI-driven autonomous testing, which is revolutionizing the $120B software QA industry. The company’s patented platform, Appvance IQ, can generate its own tests, surfacing critical bugs in minutes with limited human involvement in web and mobile applications. n Digital.ai Continuous Testing (formerly Experitest) enables organizations to reduce risk and provide their customers satisfying, error-free experiences — across all devices and browsers. Digital.ai Continuous Testing provides expansive test coverage across 2000+ real mobile devices and web browsers, and seamlessly integrates with best-in-class tools throughout the DevOps/DevSecOps pipeline so developers can get test results faster and fix defects earlier in the process, allowing them to deliver secure, high-quality applications at-speed and atscale. Learn more at www.digital.ai/continuous-testing.

n HPE Software’s automated testing solutions simplify software testing within fast-moving agile teams and for continuous integration scenarios. Integrated with DevOps tools and ALM solutions, HPE automated testing solutions keep quality at the center of today’s modern applications and hybrid infrastructures. n IBM: Quality is essential and the combination of automated testing and service virtualization from IBM Rational Test Workbench allows teams to assess their software throughout their delivery life cycle. IBM has a market leading solution for the continuous testing of end-to-end scenarios covering mobile, cloud, cognitive, mainframe and more. n Micro Focus: Accelerate test automation with one intelligent functional testing tool for web, mobile, API and enterprise apps. AI-powered intelligent test automation reduces functional test creation time

and maintenance while boosting test coverage and resiliency. Users can test both the front-end functionality and back-end service parts of an application to increase test coverage across the UI and API. n Microsoft’s Visual Studio helps developers create, manage, and run unit tests by offering the Microsoft unit test framework or one of several third-party and opensource frameworks. The company provides a specialized tool set for testers that delivers an integrated experience starting from Agile planning to test and release management, on-premises or in the cloud. n Mobile Labs (acquired by Kobiton): Mobile Labs remains the leading supplier of in-house mobile device clouds that connect remote, shared devices to Global 2000 mobile web, gaming, and app engineering teams. Its patented GigaFox solves mobile device sharing and management challenges during development, continued on page 12 >

11

12

SD Times

December 2021

www.sdtimes.com

< continued from page 11 debugging, manual testing, and automated testing. n NowSecure: Through the industry’s most advanced static, dynamic, behavioral and interactive mobile app security testing on real Android and iOS devices, NowSecure identifies the broadest array of security threats, compliance gaps and privacy issues in custom-developed, commercial, and business-critical mobile apps. NowSecure customers can choose automated software on-premises or in the cloud, expert professional penetration testing and managed services, or a combination of all as needed. n Orasi is a leading provider of software testing services, utilizing test management, test automation, enterprise testing, Continuous Delivery, monitoring, and mobile testing technology. n Parasoft helps organizations continuously deliver quality software with its market-proven, integrated suite of automated software testing tools. Supporting the embedded, enterprise, and IoT markets, Parasoft’s technologies reduce the time, effort, and cost of delivering secure, reliable, and compliant software by integrating everything from deep code analysis and unit testing to web UI and API testing, plus service virtualization and complete code coverage, into the delivery pipeline. n Perfecto: Users can pair their favorite frameworks with Perfecto to automate advanced testing capabilities, like GPS, device conditions, audio injection, and more. It also includes full integration into the CI/CD pipeline, and continuous testing improves efficiencies across all of DevOps. With Perfecto’s cloud-based solution, you can boost test coverage for fewer escaped defects while accelerating testing. n ProdPerfect is an autonomous, end-toend (E2E) regression testing solution that continuously identifies, builds and evolves E2E test suites via data-driven, machineled analysis of live user behavior data. It addresses critical test coverage gaps, eliminates long test suite runtimes and costly bugs in production, and removes the QA burden that consumes massive engineering resources. n Progress: Telerik Test Studio is a test

automation solution that helps teams be more efficient in functional, performance and load testing, improving test coverage and reducing the number of bugs that slip into production. n SmartBear focuses on your one priority that never changes: quality. Whether it’s TestComplete, Swagger, Cucumber, ReadyAPI, Zephyr, or one of its other tools, SmartBear spans test automation, API life cycle, collaboration, performance testing, test management, and more. n SOASTA’s Digital Performance Management (DPM) Platform enables measurement, testing and improvement of digital performance. It includes five technologies: TouchTest mobile functional test automation; mPulse real user monitoring (RUM); the CloudTest platform for continuous load testing; Digital Operation Center (DOC) for a unified view of contextual intelligence accessible from any device; and Data Science Workbench, simplifying analysis of current and historical web and mobile user performance data. n Synopsys: A powerful and highly configurable test automation flow provides seamless integration of all Synopsys TestMAX capabilities. Early validation of complex DFT logic is supported through full RTL integration while maintaining physical, timing and power awareness through direct links into the Synopsys Fusion Design Platform. n testRigor helps organizations dramatically reduce time spent on test maintenance, improve test stability, and dramatically improve the speed of test creation.

This is achieved through its support of “plain English” language that allows users to describe how to find elements on the screen and what to do with those elements from the end-user’s perspective.

On top of it, testRigor helps teams deploy their analytics library in production that will make systems automatically produce tests reflecting the most frequently used end-to-end flows from production. n Tricentis Tosca, the #1 continuous test automation platform, accelerates testing with a script-less, AI-based, no-code approach for end-to-end test automation.

With support for over 160+ technologies and enterprise applications, Tosca provides resilient test automation for any use case. z

< continued from page 8

time, they won’t be able to tailor the tools properly, and they’ll end up really frustrated with security.”

This has created a need for tools that can be packaged in a way and in the right place for developers to use.

“There still is a role for expert-based pentesting and expert threat modeling and things like that. But they should work at the margin. Instead of trying to do everything with a pen test, including the stuff that your tools already did a great job at, have your pen testers focus on the things that are hard and difficult for tools,” Williams said.

Evolving testing in DevOps is primarily a people process Although tooling is necessary, testing in DevOps is also about a mindset shift on the part of the people in an organization and on making the process easier.

After all, they will still have a major part to play in testing in the near future.

Organizations are showing a strong preference for low code and test automation solutions as opposed to script-based solutions. They are also looking for unified quality engineering platforms, rather than best-of-breed point solutions for various aspects of testing, according to mabl’s Belcher.

Although AI is being applied to a growing number of use cases as part of testing in DevOps, some experts agree that there will always be humans in the loop and that the purpose of those underlying frameworks is to supercharge those people.

The next leap in the field is going to be autonomous testing where the team will steer the AI at a high level, review if the AI did the right thing and then spend most of their time focused on more strategic work, such as the usability of the application, according to Sever.

“AI is still an emerging technology, and its role in testing is evolving constantly. The most visible type of AI tooling we see is around AI-assisted automated test creation,” Merrell said.

“These tools, while extremely useful, are still no substitute for the human mind of a tester, nor do they take the place of a skilled test automation developer.” z

Full Page Ads_SDT054.qxp_Layout 1 11/12/21 4:55 PM Page 35

Intelligent Test Automation for Agile Teams TODAY, software development teams across the globe are facing the challenge of delivering high-quality web applications while keeping pace with business and customer demands. The risk of releasing bugs into production, impeded product velocity, and a diminished customer experience is too great.

Built for CI/CD, mabl integrates automated end-to-end testing into the entire development lifecycle. Creating, executing, and maintaining reliable tests has never been easier.

With mabl, teams can: Easily create automated UI tests - and save on test maintenance with the help of AI

Create automated end-to-end tests through the UI, capturing a true end-user perspective

Increase test coverage across applications and browsers with a single platform

Gain actionable insights from mabl’s rich application data for quicker issue resolution

ΖQWHJUDWH GLUHFWO\ LQWR \RXU ZRUNȵRZ ZLWK SODWIRUPV VXFK DV *LW+XE %LWEXFNHW Jira, and Gitlab

90%

3x

40%

Increase in test coverage

Faster test creation

Fewer bugs in production

Modern software development needs a modern testing solution. Try mabl free to see how easy it is to start testing.

START YOUR FREE TRIAL:

mabl.com/trial-registration