Performance of native and web-based mobile applications
Intro
Hello and welcome to my second blog post :). This blogpost is a summary of my master thesis. The topic of my thesis was the performance of mobile applications. In particular I am going to compare native and web-based applications. But at first I will tell a little bit about me. My name is Philipp Darkow and I finished last year (2013) my master diploma in software engineering at the University of Amsterdam. Before my master, I studied at a University of Applied Science in Venlo (Netherlands) and finished with a Bachelor in Software Engineering. During my master thesis I carried out a research at Trifork about the performance of mobile applications.
Because I love to write code and not just to read books about the performance I created two experiments. I start with of explaining what was compared with each other. Then I want to show you the experiments and how they were performed. Then it is time to see some code. I am going to show how a RESTful web service was created and how the different mobile prototypes communicate with it. That is be followed by a small section about the tools that I used to measure the time. After that I will share some results of the experiments and in the end I will give my conclusion.
Comparison
In this part I discuss what I compared. Next to native and web-based applications I also compared which data type XML or JSON is faster. I know, some of you will scream and say “Oh man of course JSON is faster because you don’t need to go through a document and read it”. And you are right, totally right. However I performed the comparison also with XML to find if there exists a difference between the sizes of the XML documents. As you already can think I decided to send various sizes of XML documents and JSON objects. Let us take a lock into the data and the size in bytes.
|
Size |
Description |
| Of 3JSON: 340 BytesXML: 422 Bytes | The values or tags in the object or document are
|
| Of 6JSON: 408 BytesXML: 503 Bytes | The values or tags in the object or document are
|
| Of 12JSON: 531 BytesXML: 666 Bytes | The values or tags in the object or document are
|
| Of 24JSON: 795 BytesXML: 1.01 Kilobytes | The values or tags in the object or document are
|
Taking a close look on the table you will ask yourself why are the objects and documents so small. And yes that is a good question but if you remember we are speaking about mobile applications and not about desktop applications. Why should I need a XML document and JSON objects with 100 of tags or key-value pairs? I just want my mobile user to login or to give him some information out of a database.
Native & Frameworks
The comparison was between iOS or Android with 4 mobile web-based frameworks. Those frameworks were:
- jQueryMobile
- One of the most famous web-based mobile frameworks. Comes from jQuery and is known to have a fast performance
- Sencha Touch 2
- A framework that is under open source and the guys of Sencha are pushing the boundaries of functionality
- The-M-Project
- Is based on jQueryMobile but supports more native features
- Appcelerator Titanium
- Appcelerator Titanium is a paid framework as soon your application comes life. But therefore you have access on nearly all or even all native features.
Experiments
As mentioned I performed two experiments. The experiments were aimed to measure the:
- Number of objects sent
- The number of objects sent is one of the first metrics to measure performance of applications. For this metrics different sizes of various data containers are sent and compared with each other.
- Average time of all object transmissions
- The average time of all object transmissions is calculated by performing the experiments about a certain time period. Each result is taken and added to the next one. In the end the total is divided by the time how often the experiment was made to calculate the average.
- Traffic speed
- The traffic speed as the name let suggest is about the traffic speed of a transaction.
Experiment One
In the first experiment I measured the time it took to send XML and JSON of different sizes. Below the figure is aimed to explains how the experiment was made.
So what is figure saying? We see a server unit in the left bottom corner, in the middle a bubble which represents a kind of network called the internet and in the right top corner a mobile device. The measurement was the time it took to send data from the server to the mobile device.
Experiment Two
The second experiment was covering a more realistic use case. I also created for that a figure which is presented below.
You should notice that the figure looks similar to the previous one. However the second experiment was aimed to measure the time to process the data of a object or document and to display it on a text field on the mobile device. Firstly, as in the previous experiment we are going to send the data from a server to a mobile device. As soon the data can be further used a timer starts to measure the time which is needed to insert the data in text fields on the mobile device. If that is done the timer stops and we have the actual time which was needed for using the data.
Setting up the Environment
To accomplish both experiments I created a RESTful web service. You will ask why a RESTful one and not a SOAP or another one? So to wrap it up the RESTful web service was used because it is proven that this technique fits the best for mobile applications. I created the web service with the Netbeans IDE. The web service had just the task to create a JSON object or XML document and to fill this object/document with data and to return it.
Here we see a method of the web service to return a JSON object with three key-value pairs. The code is very simple as you can see a JSON object is created and key-value pairs are put inside. In the end the created JSON object is returned.
As you can notice, you need more code to create the XML document with the exact same data as the JSON object. You need to create a document builder factory from that you get a document builder which is responsible of the creation of the document. After that each value will be created as an element and appended to the document. That was the code of the server side transaction of JSON and XML quite easy or?
The code to receive the data on the mobile device differs between native and web-based applications. As we know (I hope the most of you) iOS is using objective-c, Android is using Android for Java, and the web-based applications are most of the time a mix of HTML5, JavaScript, and CSS.
For this blog entry I am just going to present the code which was used to receive an JSON object. Below the figure contains the objective-c code.
The next code sample is of the native android code.
The four web-based code samples closely resemble each other. Below is the jQueryMobile code sample for creating an ajax request. As you can see the code is quite straightforward.
The next web-based code sample is of the Sencha Touch 2 framework.
The code sample of the The-M-Project is very similar to the one of the jQueryMobile example and presented in the figure below .
The last web-based code is of the Appcelerator Titanium framework and shown below. The Appcelerator framework has its own IDE and as default the background is set on black.
Tools
To measure the time in the experiments I used some tools. The frist tool was Charles. Another tool was an custom written timer.
Charles
Charles is a tool which can be used to measure the transfer of the data which was sent. Personally I think it is a pretty nice tool which does it job (thanks to Jettro for showing me this tool). Charles can be downloaded as a one month free version here http://www.charlesproxy.com/. It is a helpful tool to monitor the traffic of any application.
Timer
Unfortunately it was not possible to use Charles in the second experiment and it was not really a tool that titled my purpose (or i just couldn’t find one). So what did I do? I wrote an custom timer for the native and web-based applications.
Results
Instead of presenting you the boring results in a table, I will just give you my main findings from the experiments.
- JSON is faster than XML (yes we know that already but it is a result)
- As bigger the XML document is as faster it is on the mobile device (for example a 12 value document is faster than two times 6 value documents)
- Native are faster than web-based applications
- However the difference is not so significant and not recognizable for an end user
Okay the first result we know already. And the third one is (if you are a mobile developer) not a big surprise. But the second result could be interesting if you prefer XML over JSON (a big plus of XML is that you can structure the document better than an JSON object). If that is the case you should send a document which contains more data which the application might use (and save the data in the local storage, etc.) The last result is related to the third result as you can see. Is argues that the difference is not so significant. And yes the difference are just some milliseconds. Let me present one result as an example. Between the native iOS implementation and the jQueryMobile implementation is, with a JSON object, a difference of 0,07 ms (3 key-value pairs), 1,64 ms (6 key-value pairs), 2,75 ms (12 key-value pairs), and 2,58 ms (24 key-value pairs).
Conclusion
So what can we conclude from the results? First, we can conclude based on the results, that each mobile application should be developed native. Wait, each application native, that can’t be right! And it isn’t. Do you remember the comparison between native iOS and jQueryMobile with the JSON as data type. As mentioned there the difference will be not noticable for an end user.
Second, we can conclude (and that is more a fact) that performance is just one of more requirements which a mobile application can have. Other requirements can be usability, portability, capital etc. which can influence the decision between native and web-based. If a mobile application should have a high usability, native development is the right choice. On the other hand if portability and the capital are crucial requirements a web-based mobile application could be the right solution.
As you can see it is not an easy decision between a native or web-based application. You need to find out what you (or your customer) want(s) to accomplish and what your capabilities to reach that goal are. With capabilities I speak about if you want to have a native application and support iOS, Android, and Windows Phone you need to learn three different coding languages and be able to maintain them. As final conclusion I want to say, that each mobile application is an individual product and to be able to make a decision between a native and web-based implementation, the key requirements needs to be clear and worked out.
