Early Software Testing for Samsung ARTIK Cloud
In 2017, South Korean tech giant Samsung turned to Performance Lab for testing of its internally developed Samsung ARTIK Cloud app. It was a cloud connector for IoT devices and cloud storage.
Samsung intended to expand this project and hired Performance Lab to test the app to minimize the number of critical defects in the product.
Our task was to check Samsung ARTIK Cloud’s operability and compatibility with several devices available in Performance Lab’s collection.
Performance Lab was hired to test the app to minimize the number of critical defects in the product.
Got a project in mind?
There is no better place for a QA solution than Performance Lab.Drop us a line to find out what our team can do for you.
What problems did the team face?
01 First problem: Tight deadlines.
The customer had already set a release date for Samsung ARTIK Cloud, so the early software testing had to be carried out at an accelerated pace.
02 Second problem: It turned out that the app did not support Android 4.
The contract signed with the customer was for a certain number of devices, including devices running Android 4.
We solved this problem as follows: We found out which phone models our office employees had and borrowed devices that were suitable for the project. This change was, of course, agreed with the customer.
03 Third problem: difficulty reading traffic on devices running Android 7 and higher.
From the developers of the Tinder app (one of our projects at that time), we learned how to add the necessary properties to the debug build and properly configure a traffic sniffer to view requests for the latest versions of Android.
For example, we added a debug-override to the build’s manifest in order to trust user certificates. In Charles Proxy, we set “Transparent Proxy: Enabled, port 9292” (in principle, you could use any port other than 8888, which is set by default). The certificate was saved as a binary and added to the phone for VPN and apps.
04 Fourth problem: The team had not previously worked with the WebSocket protocol.
This is a full duplex protocol (i.e. it can send and receive at the same time) on top of a TCP connection. It was designed for real-time messaging between a browser and web server.
To successfully complete the early testing in software, this technology had to be mastered in parallel with the project management.
What results were achieved?
We tested Samsung ARTIK Cloud on 19 different devices, which allowed us to find 43 bugs, including 8 critical defects.
It shouldn’t be surprising that we were able to find so many bugs: the Samsung app was in the early stages of development. After the project, the app began to be much more stable.
Performance Lab’s efforts made it possible to fix bugs in the app’s main processes, which allowed Samsung to release it on schedule. Additionally, the customer received recommendations on how to further improve Samsung ARTIK Cloud.
In turn, Performance Lab was able gain IoT expertise thanks to this joint project with Samsung. The company had not previously managed an IoT (Internet of Things) project. It was a useful experience with potential long-term benefits for us.