Identifying Trade-offs Associated with Cross-platform Mobile Development Tools

Abstract

Native app development is still the primary development method on Android and iOS, the two major platforms for mobile devices. The development process for building an app for more than one platform with the native approach is slow and expensive. Although cross-platform mobile development tools (CPDTs for short) allow building apps for more than one platform using a single codebase, application of such tools has traditionally required considerable sacrifices in terms of performance, resource use and user experience. In recent years, a new generation of CPDTs has emerged that once again raise hopes of achieving the "write once, run everywhere" ideal. However, few studies have systematically assessed trade-offs associated with such recent CPDTs.

This thesis sought to identify some of the practical trade-offs between data-driven apps built by modern CPDTs when compared amongst themselves and against their native counterparts. The thesis established and applied a quantitative evaluation framework for investigation of the trade-offs between modern CPDTs with respect to responsiveness, convenience of use, resource use, performance, and developer experience. The study first identified three modern CPDTs: Flutter, React Native and PWA (Progressive Web Apps) based on perceived developer interest. The thesis then developed a three-fold evaluation system. For each evaluation component, an app was designed and developed for each baseline comparator (i.e., Android native and iOS native). This work then developed an app implementing the same design for each CPDT (i.e., Flutter, React Native and PWA). For that evaluation component, such evaluation-specific apps were then compared to one another. The first component of the evaluation investigated trade-offs associated with responsiveness and convenience of use, using measurements of three defined metrics: launch time, navigation time and size of the installer. The second evaluation component investigated trade-offs related to resource use and performance by implementing a set of resource-intensive tests to measure relevant metrics. The third and final component of the evaluation focused on trade-offs associated with the developer experience by measuring three metrics: lines of code, cyclomatic complexity, and build time across multi-page data-driven apps developed separately using each development method.

The responsiveness and convenience of use evaluation found some trade-offs imposed by CPDTs, with those being especially notable on the iOS platform. The resource use and performance evaluation results demonstrated that while native approaches retain an edge in resource use and performance, modern CPDTs are shrinking the gap --- and, in some cases, outperforming native apps. The results of the developer experience evaluation showed that CPDTs could significantly improve the developer experience by requiring development and maintenance of just a single, shared body of source code that is less complex and requires fewer lines of code, in contrast to the per-platform sets of lower-level source code required in the native approach. This study conducted the first quantitative study investigating the trade-offs of modern CPDTs from the developer experience perspective, and provided empirical evidence regarding such trade-offs. Through the use of dedicated resource-intensive tests, this study also serves as one of the first quantitative studies to provide insight on the resource use and performance overhead imposed by modern CPDTs. All of the artifacts developed in this work, including the multi-page data-driven apps and dedicated resource-intensive tests, are accessible in public open-source repositories.