Memory consumption: QtWebKit->compare(WebKit1, WebKit2, SystemMalloc, TCmalloc)
Last year I got a Nokia N9 on Qt Developer Days, it's a great mobile phone, I really like it, but I haven't published any memory measurement results about it yet. In my last post I did WebKit memory profiling on it with Valgrind, but it was all about heap usage. I think a number of persons are interested in the overall memory usage such as peak memory usage, the sizes of the used libraries, etc. During QtWebKit's life-cycle we switch from one library to another, usually because of performance considerations. In the past we do not take too much attention to memory consumption-related topics while switching between libraries, although it would be very vital to keep eye on this area as well.
Based on the data - which was produced - by Linux proc filesystem's smaps file, I compared WebKit1's (QtTestBrowser) memory usage with WebKit2 (MiniBrowser), with different memory manager setups. I used our Methanol benchmark suite with 18 locally mirrored unique sites from Alexa TOP500 and I used http://yvettesbridalformal.com/ for the single page test.
Some details of measuremented enviroment:
I did cross compilation for Qt5 and for WebKit based on the QtWebKit wiki page's description. (Side note: by default we compile QtWebKit with TCmalloc, but if you want to force the build to compile with system allocator you just need to pass --system-malloc option to build-webkit script.)
I have a PERL script what is capable of reporting about a process memory consumption with parsing of the smaps information file of the process. Furthermore, it is also capable of measuring the common memory consumption of two or more processes. The script analyzes the shared and the private library usage for the processes, the shared library memory consumption is counted only once of course. This tool is required to measure WebKit2's memory consumption, since it is working with two processes: MiniBrowser and QtWebProcess.
In the first part of this post I will concentrate on differences between System malloc and TCmalloc, on the second part I will focus on differences between WebKit1 and WebKit2. In the first measurement the Y axis of the chart shows the results of the maximum resident set size (maxRSS) in kilobytes of the WebKit2 based MiniBrowsers with different allocators (in WebKit the default memory allocator is the TCmalloc).
As I expected, TCmalloc still uses more memory than System allocator. This means more than 8 megabytes extra memory for TCmalloc in case of MiniBrowser. If memory consumption is really important for you then the system malloc could be a good choice, since we won 8 megabytes free memory during the loading of 18 sites.
I think I can show you more interesting things than pure values of maxRSS measurement, so in the first place I classified the memory consumption into 5 groups and made a chart with these groups.
The 5 groups are: HEAP usage, STACK usage, size of QtWebKit's, Qt's and finally the summary of every other used library's (Other) code+data segments.
I did not go deeply into an examination why there is a 2.7MB difference in the other library memory usage between the two allocators. It is definitely not related to the different allocator, rather there could be differences in loaded modules due to system drivers. Moreover, other libraries are mostly shared on the system amongst other processes as well, so most of them are in the memory anyway. The main purpose of this chart is to show the order of magnitude of the used libraries.
We have a very big QtWebkit library with ~20 megabytes resident set size, a ~13 megabytes Qt library, and a quite big >28 megabytes HEAP usage. The 5.4 megabytes difference in the HEAP usage is noticeable, this is what related to TCmalloc definitely.
Nowadays, we think about WebKit2 as a stable, accepted and regularly used browser engine. So now the biggest reformer is the Qt5, since there were enormous changes in Qt source since Qt4 and we have new kind of approaches in e.g. UI designing. In QtWebKit our Qt5-QtWebKit-WebKit2-API supports QML based UI over QWidget based one. Because of the changes in Qt5 I made table where I put a detailed list about the memory consumption of the code and the data segments of the used Qt modules.
As a next measurements I have loaded http://yvettesbridalformal.com into the testbrowsers. The site is very suitable for a memory profiling. ;-) Before the Qt5 library consumption details, let's see how WebKit1 (QtTestBrowser) and WebKit2 (MiniBrowser) consumes our memory on http://yvettesbridalformal.com with System allocator and with TCmalloc.
On the chart we can see the differences between using TCmalloc or System malloc and how do they consumes memory with WebKit1 and WebKit.
As I expected MiniBrowser (WebKit2) with TCmalloc setup consumes the most memory and QtTestBrowser (WebKit1) with the system allocator setup consumes the less. With these differences it makes sense to choose carefully, especially if you are working on an embedded platform with limited memory resources.
Here is the table what I mentioned above. It would be hard to make a nice chart of these results, so I hope you like this table form.
Both QtTestBrowser and MiniBrowser use the same Qt libraries, the differences are grouped by library. As we take a look at the numbers we see implicitly that MiniBrowser uses less memory for QtWidgets than QtTestBrowser, since it is QML based and vice versa (QtTestBrowser is QWidgets based).
I set out pvrsrvkm kernel module the biggest consumer with its 10 megabytes memory usage from other library usage, it's significant part of the maxRSS value.
It would be nice make a profiling about HEAP usage and make these separations for HEAP usage as well. If somebody is interested in this, I suggest to use Freya tool which is the proper tool for these kind of profiling.
Thanks for your attention!