simplela.blogg.se

The same survey showed how flaky tests threaten the efficiency of CI: 47% of failed jobs that were manually restarted succeeded on the second run.ĭashlane understood this problem and wanted to ensure they reduced their flaky tests. About 41% of tests in Google and 26% in Microsoft were discovered to be flaky. Nearly 0 FlakyTestsĪ survey published in October 2021 shows that flaky tests haunt even giant corporations like Google or Microsoft. For example, HyperExecute could run about 70 test cases in 40 seconds, whereas their previous internal infrastructure took about 80 seconds.

With HyperExecute, they could finally shift left and scale their testing infrastructure.ĭuring their test run, Dashlane discovered that HyperExecute could significantly cut down their test execution time.

Another great benefit was the prompt and reliable customer support.ĭashlane decided to go ahead with LambdaTest’s HyperExecute as it outperformed their in-house setup as well as the alternatives they had put to test.ĭashlane wanted to run more tests in parallel and faster and adopt a shift left approach, so finding bugs earlier in the life cycle was necessary. Also, it drastically reduced the number of flaky tests thanks to the sanitized infrastructure. They did a detailed benchmarking with performance, cost, capabilities, stability, and support as the decision drivers.Īfter a head-to-head comparison of HyperExecute with their local infrastructure, the performance of individual tests on HyperExecute took nearly half the time compared to their local test infrastructure. Opting for the right tool was essential, or the quality engineers would have wasted their time maintaining the tests and working around the tool.ĭashlane had a checklist of all their testing requirements which includedĭashlane was looking for a test orchestration platform that could help them increase their velocity and eliminate developer friction and interruptions. The team developed a pipeline where they ran smoke tests for every commit, regression tests for every merge to master, and then multi-browser tests during nightly builds. They wanted to introduce the UI tests to be run along with the development roadmap.

They wanted to adopt shift left architecture. Though they were not very comfortable opting for a third party cloud platform for their testing needs, they finally decided it was better to utilize a platform to provide them with the infrastructure rather than create one in house, thereby saving both their resources and time. However, maintaining the infrastructure took a lot of resources and time. The team consisted of 40 quality engineers in Dashlane and had initially set up an in house testing infrastructure for all their testing needs.