QA has been a core part of Xtreme Labs and the team has grown from a team of two to a dedicated team of ten. Each week, builds are delivered to clients with new features and fixes that iterate towards the final product. We ensure that all of the great apps developed at Xtreme Labs are thoroughly tested for quality.
Weekly Testing and Release Candidate Testing
Before each build is delivered to our client, it first goes through QA. Weekly, we make sure all implemented features are working as expected and are run through our most updated test suites to ensure the build is in working condition before signing off. When it comes to a build that will be released into the market, we treat that as a Release Candidate (RC) build. Typically, the RC build is expected to contain mostly bug fixes instead of new features.
In these builds, we will go through the entire app and ensure all features of the app are working, in addition to the triaged and fixed bugs. If we find any bugs that will prevent the build from being released, we iterate quickly with the engineering team to fix these issues before signing off.
On RC builds, device testing is especially important. Thus, all of our builds are tested on the right matrix of devices to ensure coverage for all the required resolutions and supported devices. For weekly builds, we will test on several devices subject to project requirements (usually from different manufacturers as well as high, medium or low resolution devices).
For RC builds, we will test on all devices that we have in inventory that the project supports according to project requirements. As well, we encourage the engineers to sit and work with us during RC testing, as the team can communicate easier, resolve issues more quickly, and to make the overall process smoother and faster.
To make sure your testing is sound, you need an impressive collection of a wide range of mobile devices. We’re proud to say that as of October of this year we added our 500th device to our inventory. (In case you were wondering, device # 500 was a Samsung Galaxy Tab 2 7.0). If you’re surprised that it was a tablet, you’d probably be interested to know that our device inventory ranges from BlackBerry, Apple, Android, Windows Phone, Web OS products and more. We are always on the look for the latest product in the market. An inventory review is conducted every 2 weeks to ensure our devices are up-to-date. If new devices are released, we would try to get a hold of it ASAP so that we are always covering our bases with the latest technologies.
QA Meeting and Scheduling
One issue that we have had in the past was with the scheduling of weekly builds. In the past, weekly builds will come in at random intervals, which can cause a bottleneck in delivering builds to our clients as builds might end up reaching QA at the same time. Now, each project has scheduled build times that includes some buffer time in case something urgent needs to be tested right away. We coordinate with the engineers to ensure a build will be delivered to us in a timely manner. Right now, our team successfully tests between 20-30 builds on a weekly basis.
Xtreme Labs continues to grow as we continue to develop more mobile products. In order to ensure all products are tested thoroughly, every Monday morning, QA and the Product Managers meet to coordinate and schedule these builds to make sure the timing lines up with client expectations. In this meeting, we determine the specific projects that are to go through QA each day and what time each build is scheduled to be delivered. If we encounter multiple RC builds on a same day, we work to shuffle our build dates to ensure all projects receive the time needed for quality testing.
Product Usability Feedback
We have built up a lot of experience using and testing many applications on multiple platforms, so we have a good idea the best practices that are expected on different platforms. Therefore, while we perform testing on a weekly build, we will often provide feedback on UI scrubbing in addition to filing bugs related to delivered features. This additional feedback cycle helps to raise the level of polish on our apps.
In the past, testing has been performed manually by our team. Recently, we have begun integrating automated testing. Check out Daniel’s article on Automation to see the progress we have made on this front.
QA is Important to App Quality
We ensure that users have the best experience when they download and use one of our apps. The QA team is dedicated to giving every one of our apps the time and feedback it needs to be a great app before we sign off on a build. Look forward to more from us on how the QA team achieves this.