Monthly Archives: May 2014

Performance Comparison from Delphi 2010 to Delphi XE6 (Part 5 – Hello World Mobile)

Hello World Mobile Execution (Speed) Tests

Now that we have finished comparing execution on Win32, Win64, and OSX, we are finally going to compare performance on mobile devices!  We created a mobile app version of the Hello World project.  It is directly comparable to the desktop version, using TMemo, TListBox, TButtons, and TLabels (see Figure)

Hello World Mobile App

Hello World Mobile App

For testing iOS apps, we used an iPod Touch with 32GB.  For testing Android apps, we used a Nexus 7 (2013) with 32GB.

Because the speed of the mobile devices is so much slower than our desktop machines, we tested adding 10, 100, and 1000 items or lines instead of 100, 1000, and 10000 like the desktop versions.  This was only necessary when the BeginUpdate and EndUpdate methods were not called.

Please notice that the FMX charts have a logarithmic y-axis for execution time.  With FMX, execution times vary so greatly between a small number of items and a large number of items that filling a TListBox or a TMemo with 1000 items would swamp out the other results and make them appear tiny and indistinguishable.

Hello World, Mobile FMX, iOS

Our first tests are with filling the TListBox on an iOS device.  Since Delphi XE4, Delphi has supported deploying to iOS devices.  Note that the Delphi XE4 app was compiled with debug.  Delphi XE5 and XE6 can deploy release versions directly to the device without requiring certificates.  However, this was not true with Delphi XE4 but as we will see, it doesn’t seem to have slowed the app down.

Comparison of execution speed for filling a TListBox in the Mobile FMX Hello World (iOS) with Delphi XE4 to XE6

Comparison of execution speed for filling a TListBox in the Mobile FMX Hello World (iOS) with Delphi XE4 to XE6

Delphi XE6 does well when filling a TListBox on an iOS device.  For 10 items, it manages to be over 2x faster (416 ms) than the slowest version, Delphi XE4, when BeginUpdate/EndUpdate are not called.  At 100 items, it is still 1.5x faster (11.9 secs) than XE4.  Only at 1000 items is it slowest, however, “slowest” is relative here as Delphi XE5 which is fastest still took over 16 minutes.  When BeginUpdate/EndUpdate are used while filling a TListBox, Delphi XE5 is the fastest.  Delphi XE6 is last in this case, but the differences are minor as it fills a TListBox with 100 items in 189 ms compared to XE5’s 102.3, with 1000 items in 3.1 secs compared to XE5’s 2.2 seconds, and with 10000 all of the Delphi’s are slow, taking 2.8 minutes for XE5 and 3.19 minutes for XE6.

Comparison of execution speed for filling a TMemo in the Mobile FMX Hello World (iOS) with Delphi XE4 to XE6

Comparison of execution speed for filling a TMemo in the Mobile FMX Hello World (iOS) with Delphi XE4 to XE6

It must be mentioned that Delphi XE4 did well filling a TListBox when the BeginUpdate and EndUpdate methods are called, nearly equaling XE5.  This is important as Delphi XE4 completely dominates when filling a TMemo, with or without calls to BeginUpdate/EndUpdate.  Delphi XE6 is in second managing to beat XE5 in every case except when filling a TMemo with 10 lines (51 ms vs 33 ms).

Hello World, Mobile FMX, Android

Our last platform is Android.  Note that only Delphi XE5 and XE6 can target this platform.

Comparison of execution speed for filling a TListBox in the Mobile FMX Hello World (Android) between Delphi XE5 and XE6

Comparison of execution speed for filling a TListBox in the Mobile FMX Hello World (Android) between Delphi XE5 and XE6

No clear winner emerges for the Android platform, though Delphi XE6 is the all-around better Delphi for Android.  If you look at the filling TListBox chart, Delphi XE6 is faster at filling a TListBox with less than 1000 items and no BeginUpdate/EndUpdate.  However, when the BeginUpdate and EndUpdate methods are called, Delphi XE5 is faster (157 ms at 100 items, 941 ms at 1000 items, and 41 seconds at 10000 items.

Comparison of execution speed for filling a TMemo in the Mobile FMX Hello World (Android) between Delphi XE5 and XE6

Comparison of execution speed for filling a TMemo in the Mobile FMX Hello World (Android) between Delphi XE5 and XE6

When filling a TMemo, Delphi XE5 and Delphi XE6 are essentially tied except for one important caveat.  Delphi XE5 is almost twice as fast as XE6 at adding 10 lines (and no BeginUpdate/EndUpdate); however, that does not mean much as it is only a difference of 22 ms.  The glaring discrepancy is that Delphi XE6 is over 7x faster than XE5 in filling a TMemo with 1000 lines (and no BeginUpdate/EndUpdate).  It takes 20 seconds for XE6; over 2 minutes later XE5 finishes.

 

Stay tuned for more next week… 🙂

That’s it for the Hello World project execution speed results.  The Hello World project is a simple application/app that provided some interesting comparisons between the different Delphi versions on various platforms.  It showed the work that needs to be done to improve the FMX versions.  However, the Hello World project is limited and truly only tested a couple “standard” controls (TListBox and TMemo).

Next week, we are going to show speed results from sample applications using the Inference Engine Component Suite (IECS).  The IECS is a large component library (96K lines of engine code and another 48K LOC for dialogs) for developing expert systems with Delphi.  It uses lots of interfaces, generics, parsing/string manipulation, and 100s of classes.  It works from Delphi 2010 through XE6 and for all platforms that Delphi supports.  It should provide very interesting non-visual execution results.

We are going to follow that up with tests using the RiverSoftAVG SVG Component Library (RSCL).  The RSCL will only test Delphi XE2 through XE6.  However, it renders SVG files using low-level canvas operations (gradients, paths, text, etc) and is available for VCL and FMX.  It uses GDI+ for VCL (which is a software renderer with comparable features to the FMX TCanvas) so it should provide a good test of the FMX DirectX and OpenGL hardware-based rendering.  The RSCL is also able to build SVGs using the FMX shape primitives (TRectangle, TPath, TText, etc); it should reveal any optimizations (or lack thereof) Embarcadero has been doing with the low-level primitives from which most FMX controls are built.

That’s it for this week.  I hope you are enjoying this series of blog posts.  Happy CodeSmithing!

Performance Comparison from Delphi 2010 to Delphi XE6 (Part 4 – Hello World Win64 and OSX)

Hello World Execution (Speed) Tests

We’re back to continue this string of blog posts into the performance differences between Delphi 2010 through Delphi XE6, and between all platforms.  This week we continue our speed tests from last week by finishing up our performance tests for the Hello World project from the introduction.

Last week, we saw that performance differences in the VCL are minor.  With the Win32 version of the FMX Hello World application, it was more interesting.  When filling a TListBox, at small number of items, the XE2 version is fastest.  Surprisingly, at higher numbers of items, Delphi XE3 is fastest.  For filling a TMemo, Delphi XE6 rocked and managed to be faster than every other version of Delphi except XE4 when not using BeginUpdate/EndUpdate, and faster than all other versions of Delphi when using BeginUpdate/EndUpdate (even the VCL versions, though this comparison is not really valid as the VCL TMemo, when BeginUpdate/EndUpdate methods are not used, draws every string as it is added while the FMX TMemo does not)!

This week, we are going to continue our tests with Win64 and then investigate non-windows platforms.

As we noted in our update last post, please notice that the FMX charts have a logarithmic y-axis for execution time.  With FMX, execution times vary so greatly between a small number of items and a large number of items that filling a TListBox or a TMemo with 10000 items would swamp out the other results and make them appear tiny and indistinguishable.

Hello World, FMX, Win64

The FMX Win64 speed tests for filling a TListBox hold no new surprises and closely mirror the Win32 tests.  With the FMX Hello World application, at small number of items, the XE2 version is fastest using the TListBox.  However, as mentioned in the update to last week’s post, the Delphi XE2 TListBox is crippled when clearing items (which is not reflected here) and shows exponential times in clearing the list box.  At higher numbers of items, Delphi XE3 is fastest.  Delphi XE6 manages to be insignificantly faster than Delphi XE4 and XE5 when not using the BeginUpdate/EndUpdate methods.  Using BeginUpdate and EndUpdate, Delphi XE6 is slower than every prior version until we get up to 10000 items where Delphi XE2 is the slowest.

Comparison of execution speed for filling a TListBox in the FMX Hello World (Win64) with Delphi XE2 to XE6

Comparison of execution speed for filling a TListBox in the FMX Hello World (Win64) with Delphi XE2 to XE6

Ratio of Win64 to Win32 execution speed for TListBox in Delphi XE2 to XE6 (FMX)

Ratio of Win64 to Win32 execution speed for TListBox in Delphi XE2 to XE6 (FMX)

Comparing Win64 FMX to Win32 FMX execution, surprisingly only Delphi XE2 manages to be faster in Win64 on a Windows 64-bit OS and machine (though even it cannot do it when using BeginUpdate/Endupdate methods at 10000 points).  Delphi XE3, at a small number of items is faster filling a TListBox in Win64 than Win32.  Delphi XE4, XE5, and XE6 for some reason are all slower filling a TListBox in Win64 vs Win32 when executed on a Windows 64-bit OS and machine.

Comparison of execution speed for filling a TMemo in the FMX Hello World (Win64) with Delphi XE2 to XE6

Comparison of execution speed for filling a TMemo in the FMX Hello World (Win64) with Delphi XE2 to XE6

Filling a TMemo, the picture changes.  Because the FMX version of TMemo does not draw each line as it is added, the Delphi versions manage to maintain a speedy pace. Delphi XE4 is fastest when filling a TMemo with no BeginUpdate/EndUpdate method calls.  Delphi XE6 is strong again and is the fastest when using the BeginUpdate/EndUpdate methods.

Ratio of Win64 to Win32 execution speed for TMemo in Delphi XE2 to XE6 (FMX)

Ratio of Win64 to Win32 execution speed for TMemo in Delphi XE2 to XE6 (FMX)

For all versions of Delphi except for XE2 in some cases, the 64-bit version of the FMX Hello World application is faster than its 32-bit FMX counterpart when filling a TMemo.

Hello World, FMX, OSX

Now for the really exciting tests, how do the different versions of Delphi do on other platforms?  Our first tests will be the Hello World application running on OSX.  Since Delphi XE2, there has been support for creating applications for the Mac.  For our tests, we use a 2.3 GHz Intel Core I7 Mac Mini with 4GB 1600 MHz DDR3 RAM running OSX 10.9.2.

Comparison of execution speed for filling a TListBox in the FMX Hello World (OSX) with Delphi XE2 to XE6

Comparison of execution speed for filling a TListBox in the FMX Hello World (OSX) with Delphi XE2 to XE6

The results for filling a TListBox in OSX is interesting.  In general, Delphi XE3 is the fastest when BeginUpdate/EndUpdate are not called.  When BeginUpdate and EndUpdate are called, Delphi XE5 has a very strong showing, beating out all other versions except where Delphi XE3 manages to eke out a win by 1/10 of a second for 10000 items.

Delphi XE6 gives competent but not winning numbers.   It is faster or practically equal to Delphi XE4 and XE5 when not using the BeginUpdate/EndUpdate methods.  If BeginUpdate and EndUpdate are called, Delphi XE6 is faster than XE2 and XE3 for 1000 items and under, and almost tied with XE3-XE5 at 10000 items.

Comparison of execution speed for filling a TMemo in the FMX Hello World (OSX) with Delphi XE2 to XE6

Comparison of execution speed for filling a TMemo in the FMX Hello World (OSX) with Delphi XE2 to XE6

When filling a TMemo, Delphi XE4 is particularly strong on OSX.  Delphi XE4 is faster than every other version.  Delphi XE6 gives an excellent showing and is in second place for all TMemo tests, and actually equalling XE4 with 100 items and BeginUpdate/EndUpdate calls.

Hello World, FMX, Win32 vs OSX Win32

Unfortunately, the Windows machine used for testing is not exactly comparable to the Mac Mini.  However, the Windows Machine specs (Windows 7 64-bit Intel I7 930 @ 2.8 GHz CPU, and 6 GB RAM) are *close* to the Mac Mini specs (OSX 64-bit Intel I7 @ 2.3 GHz with 4GB RAM), so it will be very interesting to do comparisons of the FMX Win32 vs FMX OSX Hello World.

Ratio of Win32 to OSX execution speed for TListBox in Delphi XE2 to XE6 (FMX).  Note that test machines are not exactly comparable.

Ratio of Win32 to OSX execution speed for TListBox in Delphi XE2 to XE6 (FMX). Note that test machines are not exactly comparable.

Surprisingly, adding items to a TListBox is faster on a Mac when the BeginUpdate and EndUpdate methods are ***not*** called.  Once, BeginUpdate and EndUpdate are called, we get the expected results and the Windows box wins.

Ratio of Win32 to OSX execution speed for TMemo in Delphi XE2 to XE6 (FMX).  Note that test machines are not exactly comparable.

Ratio of Win32 to OSX execution speed for TMemo in Delphi XE2 to XE6 (FMX). Note that test machines are not exactly comparable.

Adding lines to a TMemo is a rout and the Windows application wins across the board and for all versions of Delphi.

Next…

In our next post, we are finally going to compare execution speed for mobile apps

Until then, Happy CodeSmithing!

Performance Comparison from Delphi 2010 to Delphi XE6 (Part 3)

Execution (Speed) Tests

Finally we get to the interesting part of these series of blog posts.  In the previous post, we looked at compilation speed and EXE size for the Hello World project from the introduction.  In this section, we are going to look at the execution speed for the Hello World project in Delphi 2010 to Delphi XE6, for Win32, Win64 (Delphi XE2-XE6 only).  Next week, we will finish the Win64 FMX tests and then also show OSX (Delphi XE2-XE6 only), iOS (Delphi XE4-XE6 only), and Android (Delphi XE5-XE6 only). The Hello World project fills a TListBox or a TMemo with a number of strings with every click of a button.  In our speed tests, we tested adding the string “Hello World” with a number appended after it.  The button click adds 10, 100, 1000, or 10000 strings at a time.  We are also going to test adding strings inside a BeginUpdate/EndUpdate, which should be significantly faster unless something has been broken. Note that the charts display the execution time on the y-axis in milliseconds.  The number of strings added as well as if BeginUpdate/EndUpdate were used are on the x-axis.  Except for the VCL charts, the y-axis is logarithmic as the amount of time to add strings goes up exponentially with FMX applications.

Hello World, VCL, Win32

The first test is with the VCL version of Hello World in Win32.  Both the TListBox and TMemo pass their control handling to the underlying Windows control so unless Embarcadero broke something, we should see little change between the different versions of Delphi.  And that is what we see.  Every version of Delphi is able to fill a TListBox with up to 10000 strings in under a second.  Note that using BeginUpdate and EndUpdate methods speed up the execution by over 30x!  Every version of Delphi is able to fill a TMemo with up to 10000 strings in just around 6 seconds with the BeginUpdate/EndUpdate tests finishing in a sixth of the time.  

Comparison of execution speed for filling a TListBox in the VCL Hello World (Win32) with Delphi 2010 to XE6

Comparison of execution speed for filling a TListBox in the VCL Hello World (Win32) with Delphi 2010 to XE6

Comparison of execution speed for filling a TMemo in the VCL Hello World (Win32) with Delphi 2010 to XE6

Comparison of execution speed for filling a TMemo in the VCL Hello World (Win32) with Delphi 2010 to XE6

Hello World, VCL, Win64

For the 64-bit Hello World application, we can only test with Delphi XE2 through XE6.  There are also no significant execution speed differences between the Delphi versions in Win64.  It is, however, gratifying to see that the promise of Win64 Delphi applications running native in a Windows 64-bit OS and 64-bit CPU is realized as the Win64 versions run around 10-25% faster.  Even though 64-bit applications are larger and consume more memory, it can be worth it in order to see these speed increases.

Comparison of execution speed for filling a TListBox in the VCL Hello World (Win64) with Delphi XE2 to XE6

Comparison of execution speed for filling a TListBox in the VCL Hello World (Win64) with Delphi XE2 to XE6

Comparison of execution speed for filling a TMemo in the VCL Hello World (Win64) with Delphi XE2 to XE6

Comparison of execution speed for filling a TMemo in the VCL Hello World (Win64) with Delphi XE2 to XE6

Ratio of VCL Hello World execution speed of Win64/Win32 running with Windows 7 64-bit and 64-bit CPU

Ratio of VCL Hello World execution speed of Win64/Win32 running with Windows 7 64-bit and 64-bit CPU

Hello World, FMX, Win32

From here on out, we are going to be testing FMX applications.  Everything… every pixel, every string, every button… is drawn by the FMX library and does not use OS controls.  It is expected that the FMX applications will be, not only bigger (as we saw in the last post as all the drawing and interaction code must be compiled into the application), but may also draw GUI controls slower.  Not only can Delphi applications not count on using OS native controls, but in general FMX controls are much, much richer than their VCL counterparts.  The FMX TListBox implementation will be slower than the VCL implementation for this reason.   However, this blog post is about exploring the execution differences between the various Delphi versions.  What we hope to see is that the execution speed will improve with each new version of Delphi, especially with the latest Delphi XE6 and its mantra of “Quality, Performance, and Stability.”

(Update: Note the FMX charts have a logarithmic y-axis for execution time.  We used this because the execution times vary so greatly between a small number of items and a large number of items.  Filling a TListBox or a TMemo with 10000 items would swamp out the other results and make them appear tiny and indistinguishable.)

Comparison of execution speed for filling a TListBox in the FMX Hello World (Win32) with Delphi XE2 to XE6

Comparison of execution speed for filling a TListBox in the FMX Hello World (Win32) with Delphi XE2 to XE6

With the Win32 version of the FMX Hello World application, at small number of items, the XE2 version is fastest using the TListBox.  Surprisingly, at higher numbers of items, Delphi XE3 is fastest.  Delphi XE6 manages to be faster than Delphi XE4 and XE5 when not using the BeginUpdate/EndUpdate methods.  Using BeginUpdate and EndUpdate, Delphi XE6 is slower than every prior version until we get up to 10000 items where Delphi XE2 is the slowest.   (Update: I forgot to mention an important point about Delphi XE2 TListBox performance: the Clear method is broken in XE2.  It doesn’t seem to use BeginUpdate/EndUpdate internally so emptying the list goes exponential as well causing any use of Delphi XE2’s TListBox to be very painful)

Ratio of FMX to VCL execution speed for TListBox in Delphi XE2 to XE6

Ratio of FMX to VCL execution speed for TListBox in Delphi XE2 to XE6

Comparing FMX execution time to their VCL counterparts, the TListBox is much slower (there is a reason that Embarcadero recommends using TListView for lots of items).  At its best (100 items using BeginUpdate and EndUpdate), the FMX TListBox is 3x (XE5) to 6.76x (XE6) times slower.  Even using the BeginUpdate and EndUpdate methods, the FMX TListBox is much slower when adding 10000 items, from 72x (XE3) to a staggering 2258x (XE2, without BeginUpdate/EndUpdate) slower.

Comparison of execution speed for filling a TMemo in the FMX Hello World (Win32) with Delphi XE2 to XE6

Comparison of execution speed for filling a TMemo in the FMX Hello World (Win32) with Delphi XE2 to XE6

Filling a TMemo, the story is completely different: Delphi XE6 is faster than every other version of Delphi except XE4 when not using BeginUpdate/EndUpdate, and faster than all other versions of Delphi when using BeginUpdate/EndUpdate (even the VCL versions)!  Delphi XE5 is the big loser when the BeginUpdate/EndUpdate methods are not used.  If they are used, Delphi XE3 has the slowest FMX TListBox.

Ratio of FMX to VCL execution speed for TMemo in Delphi XE2 to XE6

Ratio of FMX to VCL execution speed for TMemo in Delphi XE2 to XE6

As stated, Delphi XE6 FMX TMemo is even faster than the VCL TMemo, even without using BeginUpdate/EndUpdate; however, the comparison is not fair as the VCL TMemo draws every string as it is added while the FMX TMemo does not.  As anyone who has used the FMX TMemo can attest, the FMX TMemo draws very slowly, especially compared to the VCL TMemo.

Coming up next week…

Next week, we will look at the Win64, OSX, iOS, and Android speed comparisons.  I apologize that you cannot look at them this week, but the FMX versions of the applications can take a frustratingly long time as has been shown.  I am still running speed tests and will report more when I have finished.  For now, Happy CodeSmithing!

Performance Comparison from Delphi 2010 to XE6 (Part 2)

Compilation Tests

For our first tests, I wanted to see how much Embarcadero has been improving the various compilers (Win32, Win64, OSX, iOS, and Android) for each version of Delphi, both in terms of compilation speed and final executable size.  I know that this is a trivial part of the performance comparison but I thought it is important to look at every aspect of the application making process.  I took the Hello World project described in the introduction and ran it through the various Delphi compilers: 2010 to XE6, Win32, Win64, OSX, iOS, and Android.

Hello World, VCL, Win32

Comparison of compilation times for VCL Hello World Application (Win32) with Delphi 2010 to XE6

Comparison of compilation times for VCL Hello World Application (Win32) with Delphi 2010 to XE6

For the Win32 test, all compilers from Delphi 2010-XE6 were used.  Every compiler was blazingly fast and with such a small application (101 lines of code), compilation was performed so fast (less than 1/4 of a second) that the difference between compilers looks like system noise.  It must also be noted that the Delphi 2010 and Delphi XE compilers were run from the IDE which explains their slower execution.  The interesting result, however, is EXE size.  It has been apparent for a long time that each version of Delphi has been increasing the EXE size (i.e., code bloat).  The results are backed up with our tests.  Every version has increased the resulting EXE in size from 914KB in Delphi 2010 to over 2 MB (2258KB) in Delphi XE6.  The biggest jumps were from XE to XE2 and XE2 to XE3.  Note that this code bloat is not unexpected as every version of Delphi has been adding to the run-time library.  Whether you find this code bloat good or bad is a matter of opinion as adding features is IMO a good thing.  However, it is a shame that the Delphi linker has not kept pace with development and been aggressive in discarding unused code.

Comparison of EXE Sizes for VCL Hello World Application (Win32) with Delphi 2010 to XE6

Comparison of EXE Sizes for VCL Hello World Application (Win32) with Delphi 2010 to XE6

Hello World, VCL, Win64

Comparison of compilation times for VCL Hello World Application (Win64) with Delphi 2010 to XE6

Comparison of compilation times for VCL Hello World Application (Win64) with Delphi 2010 to XE6

With the introduction of Delphi XE2, both Win64 and OSX support were added.  For my next tests, I compiled the Hello World application with the Delphi Win64 compilers.  Again, compilation was blazingly fast (less than 1/4 second) though each version of the compiler is very slightly slower than the previous one.  However, EXE size has ballooned to about a MB more than the Win32 version so compilation differences are probably just hard drive limited.  Again, the EXE sizes are increasing with every version of Delphi.  Delphi XE did not have a Win64 compiler so we cannot see the EXE size increase between those versions, but the jump from XE2 to XE3 is bad.

Comparison of EXE Sizes for VCL Hello World Application (Win64) with Delphi 2010 to XE6

Comparison of EXE Sizes for VCL Hello World Application (Win64) with Delphi 2010 to XE6

Hello World, FMX, Win32

Comparison of compilation times for FMX Hello World Application (Win32) with Delphi XE2 to XE6

Comparison of compilation times for FMX Hello World Application (Win32) with Delphi XE2 to XE6

The compilers for Win32 and Win64 are the same as that used for the VCL application.  The difference is the visual component library used (FMX instead of VCL).  The Hello World FMX application uses TListBox, TMemo, TEdit, TCheckBox, etc, but instead of using Windows managed controls, FMX does all of the work itself.  As expected, FMX EXEs are significantly larger than their VCL counterparts.  The surprise is by how much (over 2x as large) and how bad Delphi XE3 EXE sizes are (almost 4x larger).  Delphi XE4 dramatically improved EXE sizes over XE3 but was not able to reduce the EXE size to XE2 levels.  Since then, the EXE size has been slowly creeping up. Compilation speeds seem to track very closely to the EXE size so my guess is that we are seeing HD read/write times.

Comparison of EXE Sizes for FMX Hello World Application (Win32) with Delphi XE2 to XE6

Comparison of EXE Sizes for FMX Hello World Application (Win32) with Delphi XE2 to XE6

EXE Size increase between VCL and FMX Hello World applications (Win32) for Delphi XE2 to XE6

EXE Size increase between VCL and FMX Hello World applications (Win32) for Delphi XE2 to XE6

Hello World, FMX, Win64

The Win64 results closely mirror the Win32 results.

Comparison of compilation times for FMX Hello World Application (Win64) with Delphi XE2 to XE6

Comparison of compilation times for FMX Hello World Application (Win64) with Delphi XE2 to XE6

Comparison of EXE Sizes for FMX Hello World Application (Win64) with Delphi XE2 to XE6

Comparison of EXE Sizes for FMX Hello World Application (Win64) with Delphi XE2 to XE6

EXE Size increase between VCL and FMX Hello World applications (Win64) for Delphi XE2 to XE6

EXE Size increase between VCL and FMX Hello World applications (Win64) for Delphi XE2 to XE6

Hello World, FMX, OSX

Comparison of compilation times for FMX Hello World Application (OSX) with Delphi XE2 to XE6

Comparison of compilation times for FMX Hello World Application (OSX) with Delphi XE2 to XE6

Finally, we get to compare a completely new platform!  With the release of Delphi XE2, OSX support was added.  We see the same growth pattern of EXE size from Delphi XE2 to XE6, with Delphi XE3 again being the outlier.  In this case, Delphi XE6 is very slightly faster than XE5 though I wouldn’t read too much into it.  An interesting comparision is to compare the size of Delphi FMX applications on OSX (Win32) to their Windows counterparts (Win32).  OSX applications are even larger than their Windows versions, approximately 1.5x to 2x larger.  However, the OSX version from XE3 is not as comparatively huge as other Delphi versions. Also, it is interesting to see that the OSX compiler is as fast as the WinXX compilers.  I believe that this is because the compiler was written by Embarcadero and does not use LLVM.

Comparison of EXE Sizes for FMX Hello World Application (OSX) with Delphi XE2 to XE6

Comparison of EXE Sizes for FMX Hello World Application (OSX) with Delphi XE2 to XE6

EXE Size increase between Win32 and OSX Hello World applications for Delphi XE2 to XE6

EXE Size increase between Win32 and OSX Hello World applications for Delphi XE2 to XE6

Hello World, Mobile, iOSSim and iOSDevice

In Delphi XE4, support was added for compiling and deploying apps to an iOS device (I am going to ignore the free pascal compiler from XE2).  This next generation compiler creates ARM code and then uses XCode running on the Mac to deploy to an iOS device.  I had difficultly compiling a release version of the Hello World Mobile app for my iPod Touch as I did not want to create a certificate for the app; I had to settle for compiling in debug for Delphi XE4.  Note that these tests do NOT include the XCode part of the compile and deploy equation.

Compilation time increase between Win32 and iOS Hello World apps for Delphi XE4 to XE6

Compilation time increase between Win32 and iOS Hello World apps for Delphi XE4 to XE6

Comparison of EXE Sizes for Mobile FMX Hello World App (iOS) with Delphi XE4 to XE6

Comparison of EXE Sizes for Mobile FMX Hello World App (iOS) with Delphi XE4 to XE6

As predicted and even without the deployment step, iOS apps take significantly longer to compile than Win32, Win64, and OSX. While it is awesome that we can finally compile iOS apps, the compilation process is much slower (over 10x slower than compiling the same app for Win32) and much, much, MUCH slower when you finally deploy to device.  The move to XE6 (ignoring the spurious XE4 results) has not reduced compilation time or EXE size.

Comparison of compilation times for Mobile FMX Hello World App (iOS) with Delphi XE4 to XE6

Comparison of compilation times for Mobile FMX Hello World App (iOS) with Delphi XE4 to XE6

Hello World, Mobile, Android

In Delphi XE5, support was finally added for compiling and deploying apps to an Android device.  This next generation compiler creates ARM code and an APK file and then directly deploys it to an Android device.  Note these tests do not include the deployment times.  They are also executed directly in the IDE instead of the command line.

Comparison of compilation times for Mobile FMX Hello World App (Android) with Delphi XE5 to XE6

Comparison of compilation times for Mobile FMX Hello World App (Android) with Delphi XE5 to XE6

Comparison of EXE Sizes for Mobile FMX Hello World App (Android) with Delphi XE5 to XE6

Comparison of EXE Sizes for Mobile FMX Hello World App (Android) with Delphi XE5 to XE6

Interestingly, without the deployment step (a big caveat admittedly 🙂 ), iOS and Android EXE/APK sizes are comparable and compile in the same amount of time.  Like the iOS compilations, Android apps take significantly longer (almost 12x) to compile than Win32, Win64, and OSX. The move to XE6 has not reduced compilation time or EXE size over XE5.

Compilation time increase between Win32 and Android Hello World apps for Delphi XE5 to XE6

Compilation time increase between Win32 and Android Hello World apps for Delphi XE5 to XE6

Test Results Conclusion

Embarcadero has not been doing a lot of work in increasing compilation speed or reducing EXE size.  Except for Delphi XE3 which seems an outlier, EXE sizes have been going up slowly but steadily as more is added to the RTL.  The addition of FMX greatly increases EXE sizes.  The ARM compilers are 10x-12x slower than the Win32 compiler even without the lengthy deployment step. Delphi XE6 does not improve either compilation speed or EXE size.  However, this is not the most important of Delphi performance.

That is all for this week.  Next week, we will get to the real speed comparisons and see how Hello World execution speed compares between the different Delphi versions.

Happy CodeSmithing!

Performance Comparison from Delphi 2010 to Delphi XE6 (Introduction)

(Update: there is a PDF version of this series of blog posts now available: Performance Comparison from Delphi 2010 to XE6)

Introduction

With the release of Delphi XE6 and Embarcadero’s emphasis on Quality, Performance, and Stability (QPS), I wanted to see for myself the level of improvement, especially in performance.  Delphi XE6 is definitely faster and more responsive than the last few versions, especially in FMX, but I wanted to see if I could quantify the performance improvement.  There have been a couple recent articles and posts about XE6’s speed (see http://www.dewresearch.com/news/232-rad-studio-xe6-lo-and-behold-  and http://www.delphitools.info/2014/05/07/a-look-at-improved-inlining-in-delphi-xe6/).  This series of blog posts document my explorations into the performance differences between Delphi 2010, XE, XE2, XE3, X4, XE5, and XE6. Before starting, I made some predictions about what I would see when comparing Delphi 2010-XE6.

  • EXE size will probably increase with every version of Delphi.  This was based on the fact that every version of Delphi has been adding to the runtime library (RTL).  After using Borland/CodeGear/Embarcadero products for almost 2 decades, I would be surprised if the linkers have improved much to remove unused code.
  • FMX executables will be larger than VCL executables.  This is completely expected as FMX controls are non-native controls (i.e., they don’t use OS level equivalents) so all the drawing and interaction code must be compiled into the executable.
  • FMX executables will be slower than VCL executables, though each new version of Delphi for a platform should improve.  See second bullet above.  However, I expect Embarcadero has been working hard on improving FMX execution so I would expect every version to be slightly faster than the previous.
  • Win32 and Win64 compilation should be faster than other platforms.  Embarcadero has a long history with developing for the Windows platform so these versions should be far superior to other platforms.  Also, with the use of LLVM compiler and linker, I expect the compilation to be MUCH slower as now Object Pascal becomes a at least 2-pass compiler: compilation of object pascal into LLVM bytecode and then the compilation and linking of that code into the final platform executation.  As LLVM bytecode from what I understand is language independent, I would expect it does 2 pass compilation like C.
  • Windows FMX executables will be faster than other platforms’ FMX executables.  I have much less confidence in this prediction as the LLVM compilers for each platform are not under Embarcadero’s control so it is possible that a platform vendor will optimize LLVM code much better than Embarcadero.  However, I base this prediction on the fact that it is far easier to debug and optimize on Delphi’s native platform than other platforms so I expect that Windows will be where Embarcadero has put in most of their optimization efforts.

Methodology

For the tests in these blog posts, I created sample applications and compiled them in Release configuration for each version of Delphi 2010 to XE6 (if applicable).  All compilation tests (including for OSX, iOS, and Android targets) were performed on my Windows 7 box, and, if possible, compiled from the command line to avoid Delphi IDE overhead (this was not possible with some of the mobile tests).  Windows testing was performed on this box (Microsoft Windows 7 64-bit, Intel I7 930 @ 2.8 GHz CPU, and 6 GB RAM) with all applications except Microsoft Excel (to record test results) closed.  iOS applications were tested on an iPod touch deployed directly to the device in debug mode.  Android app were tested on a Nexus 7 (2013) deployed directly to the device in release mode.  iOS applications by necessity were in Debug configuration in order to be able to compile and deploy them to my iOS device. Every test was performed a minimum of 3 times.  The best 3 times were averaged to produce a final value.

Hello World

Hello World FMX Application Screenshot

Hello World FMX Application Screenshot

My first test was to create a slightly complex Hello World Delphi application for VCL, FMX, and Mobile (iOS and Android).  This application is a variant of the classic Delphi application of a TEdit, TListBox, and TButton.  Instead of one add of the TEdit constant every click of the button, each click would fill a TListBox or a TMemo with some number of repetitions.  The TListBox.Items.BeginUpdate/EndUpdate and TMemo.Lines.BeginUpdate/EndUpdate could be turned on or off.  In addition, a number could be optionally be appended to each TEdit string (to avoid any efficiencies with using the same string each addition).  The Hello World source code for VCL, FMX, and Mobile can be downloaded here.

Tests

Each week, I will add a new blog post detailing different performance tests.

  1. Hello World Compilation Tests (Speed and Size)
  2. Hello World Execution Tests (VCL Win32 and Win64, FMX Win32)
  3. Hello World Execution Tests (FMX Win64, OSX)
  4. Hello World Mobile Execution Tests (iOS and Android)
  5. Inference Engine Component Suite Execution Tests
  6. RiverSoftAVG SVG Component Library Drawing Tests 
  7. RiverSoftAVG SVG Component Library FMX Primitives Tests
  8. Conclusion