Author Archives: tgrubb

Performance Comparison from Delphi 2010 to Delphi XE6 (Introduction)

(Update: there is a PDF version of this series of blog posts now available: Performance Comparison from Delphi 2010 to XE6)

Introduction

With the release of Delphi XE6 and Embarcadero’s emphasis on Quality, Performance, and Stability (QPS), I wanted to see for myself the level of improvement, especially in performance.  Delphi XE6 is definitely faster and more responsive than the last few versions, especially in FMX, but I wanted to see if I could quantify the performance improvement.  There have been a couple recent articles and posts about XE6’s speed (see http://www.dewresearch.com/news/232-rad-studio-xe6-lo-and-behold-  and http://www.delphitools.info/2014/05/07/a-look-at-improved-inlining-in-delphi-xe6/).  This series of blog posts document my explorations into the performance differences between Delphi 2010, XE, XE2, XE3, X4, XE5, and XE6. Before starting, I made some predictions about what I would see when comparing Delphi 2010-XE6.

  • EXE size will probably increase with every version of Delphi.  This was based on the fact that every version of Delphi has been adding to the runtime library (RTL).  After using Borland/CodeGear/Embarcadero products for almost 2 decades, I would be surprised if the linkers have improved much to remove unused code.
  • FMX executables will be larger than VCL executables.  This is completely expected as FMX controls are non-native controls (i.e., they don’t use OS level equivalents) so all the drawing and interaction code must be compiled into the executable.
  • FMX executables will be slower than VCL executables, though each new version of Delphi for a platform should improve.  See second bullet above.  However, I expect Embarcadero has been working hard on improving FMX execution so I would expect every version to be slightly faster than the previous.
  • Win32 and Win64 compilation should be faster than other platforms.  Embarcadero has a long history with developing for the Windows platform so these versions should be far superior to other platforms.  Also, with the use of LLVM compiler and linker, I expect the compilation to be MUCH slower as now Object Pascal becomes a at least 2-pass compiler: compilation of object pascal into LLVM bytecode and then the compilation and linking of that code into the final platform executation.  As LLVM bytecode from what I understand is language independent, I would expect it does 2 pass compilation like C.
  • Windows FMX executables will be faster than other platforms’ FMX executables.  I have much less confidence in this prediction as the LLVM compilers for each platform are not under Embarcadero’s control so it is possible that a platform vendor will optimize LLVM code much better than Embarcadero.  However, I base this prediction on the fact that it is far easier to debug and optimize on Delphi’s native platform than other platforms so I expect that Windows will be where Embarcadero has put in most of their optimization efforts.

Methodology

For the tests in these blog posts, I created sample applications and compiled them in Release configuration for each version of Delphi 2010 to XE6 (if applicable).  All compilation tests (including for OSX, iOS, and Android targets) were performed on my Windows 7 box, and, if possible, compiled from the command line to avoid Delphi IDE overhead (this was not possible with some of the mobile tests).  Windows testing was performed on this box (Microsoft Windows 7 64-bit, Intel I7 930 @ 2.8 GHz CPU, and 6 GB RAM) with all applications except Microsoft Excel (to record test results) closed.  iOS applications were tested on an iPod touch deployed directly to the device in debug mode.  Android app were tested on a Nexus 7 (2013) deployed directly to the device in release mode.  iOS applications by necessity were in Debug configuration in order to be able to compile and deploy them to my iOS device. Every test was performed a minimum of 3 times.  The best 3 times were averaged to produce a final value.

Hello World

Hello World FMX Application Screenshot

Hello World FMX Application Screenshot

My first test was to create a slightly complex Hello World Delphi application for VCL, FMX, and Mobile (iOS and Android).  This application is a variant of the classic Delphi application of a TEdit, TListBox, and TButton.  Instead of one add of the TEdit constant every click of the button, each click would fill a TListBox or a TMemo with some number of repetitions.  The TListBox.Items.BeginUpdate/EndUpdate and TMemo.Lines.BeginUpdate/EndUpdate could be turned on or off.  In addition, a number could be optionally be appended to each TEdit string (to avoid any efficiencies with using the same string each addition).  The Hello World source code for VCL, FMX, and Mobile can be downloaded here.

Tests

Each week, I will add a new blog post detailing different performance tests.

  1. Hello World Compilation Tests (Speed and Size)
  2. Hello World Execution Tests (VCL Win32 and Win64, FMX Win32)
  3. Hello World Execution Tests (FMX Win64, OSX)
  4. Hello World Mobile Execution Tests (iOS and Android)
  5. Inference Engine Component Suite Execution Tests
  6. RiverSoftAVG SVG Component Library Drawing Tests 
  7. RiverSoftAVG SVG Component Library FMX Primitives Tests
  8. Conclusion

Creating Icons and Launchers for Delphi Mobile Applications

If you have ever tried to create the icons, spotlights, settings and launchers for Delphi Android and iOS applications, you have quickly realized that the process is fairly ridiculous.  There are 2 different platforms (iOS and Android), 3 different device types (iPhone, iPad, Android), and 7 different ratios (1:1, 1.33:1, etc) of icon/launcher graphic sizes all combining to require a whopping total of 28 different png files to be created (See Figure 1).  Talk about taking the Rapid out of Rapid Application Development! 🙂   ProjectOptions

After doing this once, I realized that I never want to do that again.  For my own sanity and yours, I have created a small little utility for quickly creating these 30 different png files and for helping fill in the Application Options page.

The utility works by allowing you to specify base image(s) for each graphic ratio, optionally defining what portion of the image to extract for each ratio, and then generating the png files.  You have the option of creating iPhone, iPad, and/or Android files.  In addition, the utility will even make the .optset files which Delphi uses to fill in the blanks (Click Apply… for each configuration and select the .optset file).

To use the Mobile Gfx Setup tool:

First, set up your images on the Graphics Tab

GraphicsTab

  • For each image ratio, enter or browse to an image file
  • Select the part of the image to be used for the format (shown in red)
  • Move the image selection around by Ctrl Dragging the red rectangle

 

 

Next, set up the output options from the Setup tab

  • SetupTabEnter a base filename for the generated images.  The tool will append the Width and Height of an image to this filename (e.g., ‘c:\junk.png’ becomes ‘c:\junk114x114.png’
  • Select which devices you want to generate images for
  • Finally, check the ‘Generate .optset file(s)’ checkbox if you want the .optset file generated which you can then later import into Delphi

Finally, generate your images from the Generate tab

  • When you go to the Generate tab, the tool will verify you have entered everything correctlyGenerateTab
  • If validation has passed, click the Generate button to generate the images and, optionally, the .optset files (which will be called basename.android.optset and basename.ios.optset)

Finished!

If you generated the .optset file, do the following steps:

  • Load your Delphi project
  • Select Project->Options to show the project options (See Figure 1)
  • Click Application on the left to show application options
  • Change the Target configuration
  • Click the Apply… button
  • Browse to the .optset file
  • Modify the configuration and click OK!

I hope this is as useful to you as it is to me.  Source and compiled executable: MobileGFXSetup.  Requests/comments/bug fixes can be sent to me at tgrubb AT RiverSoftAVG DOT com.

Happy CodeSmithing!

Delphi Cross-platform XML: Beware the XML parser differences

When you are doing cross-platform Delphi with XML, be very careful about the XML parser.  I just got bit by a treacherous behavior difference between XML parsers.  It works perfectly fine in Windows (using the default MS XML parser), but fails miserably on other platforms (using the ADOM XML parser).

It turns out that the ADOM XML parser has a bug (well, I think it is a bug) that its implementation of the IXMLNode.HasAttribute and IXMLNode.Attributes[named string] functions fail when the attributes have namespaces (for example, ‘xlink:href’).  I had code in my software that looked for attributes by name:

if Node.HasAttribute(Name) then
  result := Node.Attributes[Name]

This code works perfectly using the MS XML parser but fails miserably with the ADOM XML parser and attributes with namespaces.  When you call HasAttribute and Attributes[Name] using the ADOM XML parser, it assumes a namespace equal to ”.  Then, its code to find the attribute checks the namespace and attribute name and fails.  For example, ‘xlink:href’ will fail because it compares a blank namespace string with ‘http://www.w3.org/1999/xlink’.  Note that this means that you will fail even if you only passed in ‘href’.

Ideally, the ADOM implementation would either ignore the namespace if you are not asking for the namespace, or it would detect that your name contains a namespace and would find the NamespaceURI.  However, until then,to make it work, you need to look for the NamespaceURI yourself:

if NamespaceTag <> '' then
   NamespaceTag := Node.FindNamespaceURI(NamespaceTag);
 if Node.HasAttribute(Name, NamespaceTag) then
  result := Node.GetAttributeNS(Name, NamespaceTag)

So beware of XML parser differences!

Working on Windows for debugging is just so much faster and smoother, I recommend testing the Windows version of your XML application using the ADOM XML parser.  The following code will force the Windows version to use the ADOM XML parser:

uses
  xmlintf, adomxmldom, xmldom;
var
  XML: TXMLDocument;
<span style="font-size: 0.857142857rem; line-height: 1.714285714;">begin
</span>  XML := TXMLDocument.Create(nil);
  XML.DOMVendor := GetDOMVendor(adomxmldom.sAdom4XmlVendor);
  XML.LoadFromFile(Filename);
&lt;snip&gt;

Hope this helps someone else!  Happy CodeSmithing!

 

Delphi for iOS and Android: The Natives are restless

One of the major marketing points from Embarcadero is that Delphi will create applications that are “native” to your mobile platform.  One of the major complaints that I see from people trying Delphi for mobile platforms is the disappointment about the speed and appearance of these “native” applications.  This is a classic case of a company overselling their product, inflating the expectations of their customers, and then disappointing them.

So is Delphi creating “native” applications for iOS and Android?  The answer is yes, yes-ish, and not at all. 🙂  What you need to realize is that there are many ways that an application can be native (or not) to a platform.  I am going to talk about 3 ways to help you understand what Delphi is good and not-so-good for.  There is native to the hardware, native to the OS, and native look and feel.

Native to the hardware

Native to the hardware means that your code is compiled down to actual machine instructions of the hardware.  This is the area that Embarcadero has done a great job and is really what their marketing department is selling you.  Unlike Java or Python or php, Delphi’s code is not interpreted by some engine or virtual machine.  After the compiling and linking are finished, that code that adds 2 numbers goes directly to the hardware without any layer in-between.  This is true for Windows, for MacOS, for iOS, and for Android.

Note that this doesn’t mean that you cannot write fast interpreted code or slow compiled code.  The algorithms you write and the optimizations that the interpreter/compiler does and does not perform can greatly affect execution speed.  It does mean that Delphi has an in-built advantage over interpreted code here though.

Note 2: Native to the Hardware could also mean direct access to the hardware (such as GPS, camera, etc) of the device, but unless you are writing device drivers, noone does direct access to that kind of hardware.  It is more properly a function of the OS APIs which is discussed in the following section.

Native to the OS

What I mean by native to the OS is does a program have direct access to the API and controls of the Operating System of the device?  For Delphi and iOS, this is an *equivocal* yes.  For Delphi and Android, this is a *good enough*.  Almost by definition, unless your development language is the same language used by the OS, you will always have to use a hopefully thin application programming interface to the OS and its controls (this is one area where C++Builder can do better than Delphi on iOS).  Since iOS’ “native” language is objective-C and (even worse from Delphi’s standpoint), Android’s “native” language is Java, Delphi is not “native” to the OS.  However, that does not mean it is a second class citizen for development.

You see, Delphi, at least the Delphi that everyone thinks about, has never had direct access to the API and controls of the Operating System.  One of the things that made Delphi great is its true Rapid Application Development of applications using its VCL.  The VCL is not direct access to the Window’s API.  At best from the standpoint of direct access, you had to use C headers to access Window’s services and at worst, the VCL was a thin wrapper around Window’s controls.  The TListBox is not native to Windows – it is a beautifully logical wrapper around a Window’s object.  Delphi sacrificed native access to the API and controls to give you Rapid Application Development (RAD).  The goal, however, has been to keep the layer’s between Delphi and the OS as thin and fast as possible, while keeping RAD and now adding cross-platform development.

So Native to the OS is a qualified yes.  For Windows, OSX, and iOS (as they are all based on C/C++), access to OS services through APIs is fast and clean.  For Android, the access is not so good as Delphi needs to use a JNI (Java Native Interface) layer, which is expensive relative to C APIs.

Delphi’s cross-platform library, Firemonkey or FMX, wrap services like Location and the Camera in what I believe is a very good way for cross-platform development.  These classes help hide the complexity of the underlying service and abstracts it for all platforms that Delphi supports.  They are thin but good layers to the underlying OS services.

However, access to the native OS controls is another story and is the subject of my next section.

Native Look and Feel

Native Look and Feel is the area where everyone is getting disappointed.  Delphi mobile apps can look slightly wrong, feel slightly sluggish, and act in a non-standard way (relative to OS controls).  This is the consequence of the path Embarcadero chose to implement cross-platform controls like TListBox, TMemo, etc.

The VCL, unlike FMX, is a thin wrapper around native Window’s controls and so shares Windows look and feel and smooth responsiveness (insert Mac fanboy disparaging comments here ;-)).  When you click and drag your mouse on Windows, you are clicking and dragging on Window’s controls.  When the application runs on a new Windows OS, it inherits (mostly) the new look and feel.  The VCL communicates through Windows messages with the underlying OS controls and everything just works.

FMX emulates all controls of the underlying OS.  It draws every pixel itself and handles every user interaction.  This is the cause of the slightly wrong look and feel.  This is why a Delphi TListBox will have different bugs than Android’s list box.  This is why it will refresh slower and respond slower than the iOS list box.  This is also why, when a new OS is published, Delphi apps do not update their look and feel and can look out of date.

So, despite Embarcadero’s claims of pixel-perfect compatibility, Delphi does not have NATIVE look and feel.  FMX still provides the RAPID in RAD, just not native look and feel.

I wrote that this is a consequence of Embarcadero’s decision, and it is.  And I really cannot blame them for the decision.  With the VCL, Embarcadero/Borland chose the thin wrapper over the OS approach.  It works beautifully, but it ties the component library to the OS.  Embarcadero could have chosen this approach for FMX but it is really, really hard and we would still be waiting for cross-platform compatibility.  They would have had to develop a cross-platform abstraction for every control in FMX (a herculean task by itself and maybe not possible).  Then, for every platform they support, they would have to develop and maintain an FMX version.  Embarcadero chose to “get in the game” as fast as possible.  I believe it was the right decision, but it does mean that they have to deal with the consequences of their choice.

Note: the bugginess and responsiveness will improve (and God do I know that there are bugs!).  Every version Embarcadero has been improving the FMX library.  While it will never have native look and feel, improved look and feel will occur.

Note 2: you can have native look and feel on Windows and iOS.  Choose the VCL for Windows, and choose something like D.P.F Delphi iOS Native Components (http://sourceforge.net/projects/dpfdelphiios/) or TMS’s iCL (http://www.tmssoftware.com/site/tmsicl.asp), which do the same thing for iOS.  Of course, you sacrifice the cross-platform development… though you get to keep using your Delphi skillset and with proper modularization can hopefully keep the incompatibilities to a minimum.

Delphi’s Strengths and Weaknesses

So what does all this mean for developers?  It means that Delphi has its strengths and weaknesses, and as long as you are aware of them, you can choose Delphi for the right jobs.

If you need cross-platform compatibility and want to deal with only one code base (mostly), Delphi is an excellent choice.  It provides nice abstractions of the OS and its services, a relatively pretty GUI library, and native (very fast) access to the CPU.  Delphi also has excellent DB connectivity, web services connectivity, and networking in general.  This means Delphi is a good choice for

  • Enterprise developers, who want to provide mobile access and don’t really care about pixel-perfect look and feel
  • Scientific and number crunching developers, who need fast processing and a way to nicely display their results
  • Game developers, surprisingly enough, who want to develop cross-platform games which don’t have “native” interfaces anyway and where FMX can provide fast-enough graphics.  (In other words, not Madden level graphics but Angry Birds).
  • “Light” Apps which don’t use a bazillion controls to interact with the user and don’t need “pixel-perfect” responsiveness
  • Compelling apps where the user will forgive some idiosyncrasies because the app is so good.  I mention this because Delphi allows you to concentrate on the app.  Delphi provides the RAD and the cross-platform compatibility, you can concentrate on making the killer app.

Hopefully, this article helps you understand the “nativeness” of Delphi.  I believe if you choose Delphi with eyes open to its strengths and limitations and how it can help you solve your problem, it is an excellent choice.

Happy CodeSmithing!

AUTOREFCOUNT and the Free Pattern

The more I use the Delphi mobile compiler and its automatic reference counting (ARC), the more I am coming to the conclusion that Embarcadero chose the wrong implementation pattern for the Free code.  Now I am not getting into the pros and cons of ARC here, I am just referring to how the Free method in the ARC compiler is mapped to nil in mobile code.  It should have stayed a Free/DisposeOf method instead.

Why do I think it was mistake?

1.  If Free is used in ARC code, the developer is porting code from other compilers.  He/she wants this behavior.  If the developer is writing from scratch for mobile (and only mobile), he will omit the Free call (or use DisposeOf).  Even worse, for the component developer, the DisposeOf option is not available.  We have to leave the Free call in as the code must work in earlier versions of Delphi.

2. The ARC compiler’s unfortunate decision to output a hint when it encounters a Free call means I usually end up mapping the Free to DisposeOf with very ugly IFDEFs anyway to get rid of the hint, e.g.,

aObject.{$IFDEF AUTOREFCOUNT}DisposeOf;{$ELSE}Free;{$ENDIF}

3. Mapping Free to nil makes destruction of objects non-deterministic from the developer’s point of view.  From the point of view of the program, of course, it is completely deterministic: destruction occurs when there are no more references to the object.  However, from the developer’s point of view, mapping Free to reference = nil is not deterministic, as the call to Free may not do anything!  It is especially bad as the behavior from non-ARC to ARC changes.  Often, I depend on the destruction of an object to notify other objects.  If that destruction doesn’t happen or happens at a different time, bad things can happen.

Objections

Now, there are plenty of objections you can make to this.

ARC is supposed to make our code more robust, and the Free method bypasses that safety.  However, Embarcadero supplies the DisposeOf method to get around that safety anyway!  Why couldn’t the Free call have stayed the method for forcing destruction?

Disposing of the object when there are still references can cause access violations!  Well, maybe… and maybe not.  The TComponent memory management architecture (using notification) means that there can be multiple references to a component, but that the destruction of a component can clean them all up.  And I would argue that I *want* to see these problems otherwise.  I am porting from a non-memory managed compiler.  If I haven’t cleaned this up properly on non-ARC, it is a bug.  It should be a bug here.

What happens in practice, except for simple classes, is that the Destroy code gets called at an unexpected moment and then causes access violations or other unexpected behavior that the developer did not expect.  It is very difficult to debug this late destruction behavior.

So to close, the Free method should have stayed Free! (I feel very patriotic for some reason right now 🙂 )  Not only would it have made the look of the code much better (without ugly IFDEFs), the behavior of the code between compilers would have stayed more alike and easier to debug. 

 

Listening is Learning

Quick note: On August 14th, Ray Konopka had his Seeing is Believing: Data Visualization in Multi-Device Apps with RAD Studio.  I highly recommend watching this webinar.  It is one of best webinars for Delphi and FireMonkey you can watch.  Unlike some webinars that are more marketing than material (or that the title of this webinar even suggests 🙂 ), the webinar is dense with information and presented in a very clear and logical format.  Ray wrote one of the best early Delphi books and it shows.  This guy knows his stuff.  I know I learned a lot about FireMonkey component development from watching this webinar.  Highly recommended, thumbs up, and five stars! 🙂

Disclaimer: I have worked with Ray on Raize Software’s Inspex component. 

Using Primitive Types with TStrings in iOS

Problem

One of the unwelcome surprises with coding in Delphi for iOS is that you cannot store primitive types (integer, string, single, etc) in TStrings.Objects.  Usually, I am storing TObjects in the TStrings but occasionally it needs to be a primitive type (such as a score).   I would never do this for components but for programs where I know exactly what I place in the TStrings, I find nothing wrong with this and it is incredibly useful.

But with Delphi for iOS, when you try:

ListBox1.Items.AddObject('Tom', TObject(100));

It blows up.  In retrospect (and with a little digging with the debugger), it is obvious why you cannot.  With an automatic reference counting compiler, any new references to an object means that the compiler inserts calls to __ObjAddRef when you add an object to the Listbox’s list.  Since the TObject(100) is not a real object, kaboom!

Now if Delphi was a “rooted” language (i.e., even primitive types have a TObject equivalent, similar to .NET or java), this would not be a problem.  The primitive would automatically be boxed and unboxed as needed to store it inside a TObject.  However, Delphi is not a rooted language, at least not yet.   Well, the technique is so useful I was unwilling to give it up.

Solution

By using generics and class operators, I have come up with a generic class for boxing and unboxing primitive types (actually any type) which I find pretty slick.  Before I get into the details, I want to show how you would use it.  To add a primitive to a TStrings, just use the TBoxXXX cast where XXX is the primitive type name:

ListBox1.Items.AddObject('Hello', TBoxInteger(10));
ListBox1.Items.AddObject('World', TBoxString('It is a nice day'));
ListBox1.Items.AddObject('How', TBoxDouble(20.23334));

To unbox back to the primitive type, it is a little more complicated.  First, you cast your TObject back to its TBoxXXX type and then you can cast it either explicitly or implicitly back to the primitive type:

var
  i: Integer;
begin
  i := TBoxInteger(ListBox1.Items.Objects[0]);
  ShowMessage(i.ToString);
end;

Easy as can be! 🙂  The one wrinkle is that the IndexOfObject method will no longer work because the method compares the TObject to the TBoxInteger object and not its value.  To handle that case, I have added an IndexOf class method:

ShowMessage(TBoxInteger.IndexOf(ListBox1.Items, 100).ToString);

Code

So how does the code work?  The whole trick is to define a generic boxing class.  This class contains a private field containing the primitive value.  It then defines implicit and explicit conversions from the primitive to the boxing object and back.  It adds the previously mentioned IndexOf method and defines the Equal class operator to make it work.  The full declaration is thus:

TRSBoxPrimitive&lt;T&gt; = class(TObject)
{ Purpose: To provide a simple class for boxing and unboxing a primitive type
Note: this class is only for use by AUTOREFCOUNT compilers }
private
  { private declarations }
  FValue: T;
protected
  { protected declarations }
public
  { public declarations }
  constructor Create( const V: T );
  class operator Equal(a,b: TRSBoxPrimitive&lt;T&gt;): Boolean;
  class operator NotEqual(a,b: TRSBoxPrimitive&lt;T&gt;): Boolean;
  class function IndexOf( const Strings: TStrings; a: T ): Integer;
  class operator Implicit(a: T): TRSBoxPrimitive&lt;T&gt;; 
  class operator Implicit(a: TRSBoxPrimitive&lt;T&gt;): T;
  class operator Explicit(a: T): TRSBoxPrimitive&lt;T&gt;; 
  class operator Explicit(a: TRSBoxPrimitive&lt;T&gt;): T; 
end; { TRSBoxPrimitive&lt;T&gt; }

When you type TBoxInteger(100), the Explicit(a: T): TRSBoxPrimitive<T> method is called:

class operator TRSBoxPrimitive&lt;T&gt;.Explicit(a: T): TRSBoxPrimitive&lt;T&gt;;
begin
  result := TRSBoxPrimitive&lt;T&gt;.Create(a);
end;

The line assigning the boxed value back to an integer calls the Implicit(a: TRSBoxPrimitive<T>): T method:

class operator TRSBoxPrimitive&lt;T&gt;.Implicit(a: TRSBoxPrimitive&lt;T&gt;): T;
begin
  result := a.FValue;
end;

The final code creates primitive boxing types from the generic type:

TBoxInteger = TRSBoxPrimitive&lt;Integer&gt;;
TBoxString = TRSBoxPrimitive&lt;String&gt;;
TBoxDouble = TRSBoxPrimitive&lt;Double&gt;;
...

The code takes advantage of the fact that the TBoxXXX classes are reference counted and will be automatically destroyed.  Otherwise, this code would leak like a sieve!  A cool bonus with this code is that the boxing allows primitive types that would not usually fit in the size of TObject.

Hopefully, this code is as useful to you as it is to me.  The full code is available here for you to use as you see fit (but please keep the attribution and let me know if you improve it)

Happy CodeSmithing!

 

TStringList.Assign… Bug, Feature, or Documentation Error?

I was recently revisiting my improved implementation of TStringList (see http://www.riversoftavg.com/efficiencytlistntstringlist.htm).  Now this code is very old, but its point about the efficiencies to be gained when copying sorted TStringLists applies.  Unfortunately, my stringlist, and Delphi’s TStringList, has a bug.

Suppose you have 2 TStringLists and the following code:

Strings1.Assign(Strings2);
if not Strings1.Equals(Strings2) then
  raise EListError.Create('Copy unsuccessful');

Do you think the exception will ever occur?  If you are like me, you would say no, and you would be wrong.

Here is the TStringList.Assign method:

procedure TStringList.Assign(Source: TPersistent);
begin
 inherited Assign(Source);
 if Source is TStringList then
 begin
   FCaseSensitive := TStringList(Source).FCaseSensitive;
   FDuplicates := TStringList(Source).FDuplicates;
   FSorted := TStringList(Source).FSorted;
 end;
end;

The inherited Assign clears the TStrings, copies some properties over, and then copies the strings using AddStrings.  The problem is subtle, but the error is that TStringList sets the Self properties after the strings have been copied (To be precise, the trouble is that the Self TStringList does not set the properties properly before copying the strings.  But more on that in a moment).

If Self has its Sorted property set to True, things start to take a wrong turn.  Basically, 3 errors can occur:

  • If the Source TStringList is not sorted, the 2 TStringLists are guaranteed to not be equal, e.g.,
Source := TStringList.Create;
Source.Add('Hello');
Source.Add('World');
Source.Add('Hello');
Dest := TStringList.Create;
Dest.Sorted := True;
Dest.Assign(Source);
if not Dest.Equals(Source) then raise exception.create('oops');
  • If the Source TString is sorted but the CaseSensitive property is not the same, the 2 TStringLists could possibly not be equal as they sort the strings differently. 
  • If the Source TStringList is sorted but contains duplicates, the 2 TStringLists could still not be equal.  The default value for the Duplicates property is dupIgnore, which means that the Self TStringList could be just throwing those duplicates away.  So not only are they not in the same order, but the 2 TStringLists don’t even contain the same number of items!

So how should the Assign method look?  The first thought is to put the TStringList property assigns before the inherited call, e.g.,

procedure TStringList.Assign(Source: TPersistent);
begin
 if Source is TStringList then
 begin
   FCaseSensitive := TStringList(Source).FCaseSensitive;
   FDuplicates := TStringList(Source).FDuplicates;
   FSorted := TStringList(Source).FSorted;
 end;
 inherited Assign(Source);
end;

This is better, but there is still a bug.  The problem is that even if the Source TStringList is Sorted and Duplicates = dupIgnore, there still may be duplicates in the Source TStringList!  Setting the Duplicates property does not actually do anything – it only works when strings are added.  So the Source TStringList could have added duplicate strings and then set its Duplicates property to dupIgnore or dupError afterwards.

The solution is to temporarily set the Self TStringList to unsorted and Duplicates to dupAccept.  Copy the strings and then copy the actual property values.  So, our final code looks like this:

procedure TStringList.Assign(Source: TPersistent);
begin
 if Source is TStringList then
 begin
   FDuplicates := dupAccept;
   FSorted := False;
 end;
 inherited Assign(Source);
 if Source is TStringList then
 begin
   FCaseSensitive := TStringList(Source).FCaseSensitive;
   FDuplicates := TStringList(Source).FDuplicates;
   FSorted := TStringList(Source).FSorted;
 end;
end;

One thing I am unsure about.  What should be done if the Source is a TStrings but is not implemented by a TStringList?  In that case, should the strings be sorted or not as they are added to Self?  My decision was that the Self TStringList should keep its own property values, even if that means the 2 objects, Self and Source, will not be equal.

Postscript

So will this “bug” ever be fixed?  My instinct says that it won’t.  This behavior has been around way too long and has become the expected behavior.  After all, the current behavior is a quick way to strip duplicate strings from a TStrings.  Changing the behavior now may break user’s code.  So that is why the title of this post asks is it a bug, is it a feature, or is it just a documentation error?

Happy CodeSmithing!

Delphi Component Writers… Descend from TPersistent!

I have been busy converting the RiverSoftAVG code to support Delphi XE4.  I promise to discuss this process in more detail later, but right now, I have to talk about a pet peeve of mine.  I guarantee my recommendation is controversial with some people, and perhaps, I should avoid it for my very first real blog post, but I just have to go with it…

Descend your objects from TPersistent, not TObject

It is amazing how useful the TPersistent class is, and how little it is actually used.  Embarcadero is especially guilty of this.  The TPersistent class is not just about persisting a class, it is arguably more useful to use the class to copy from one class to another.  Unfortunately, Embarcadero leaves large swaths of the Vcl/FMX library descending from TObject instead of TPersistent.

One of my favorite tricks when developing a class is to use the AssignTo protected method as a method to assign my class to a TStrings object.  I override the AssignTo method, detect if the destination object is TStrings and then fill the TStrings with appropriate text.

For example, my handy-dandy string some type of list/collection class could have:

procedure TMyStringCollection.AssignTo(Dest: TPersistent);
begin
  if Dest is TStrings then begin
    TStrings(Dest).BeginUpdate;
    try
      TStrings(Dest).Clear;
      for i := 0 to Count - 1 do
        TStrings(Dest).AddObject(MyString[i], MyItem[i]);
    finally
      TStrings(Dest).EndUpdate;
    end;
  end
  else begin
    inherited AssignTo(Dest);
  end;
end;

This little trick makes it easy to quickly see the contents of my list in a TListBox, TMemo,… you name it.  You just do code like:

ListBox1.Items := MyStringCollection1;

If Embarcadero had made TObjectList descend from a persistent TList, that AssignTo could easily have been:

procedure TObjectList.AssignTo(Dest: TPersistent);
begin
  if Dest is TStrings then begin
    TStrings(Dest).BeginUpdate;
    try
      TStrings(Dest).Clear;
      for i := 0 to Count - 1 do
        TStrings(Dest).AddObject(Items[i].ToString, Items[i]);
    finally
      TStrings(Dest).EndUpdate;
    end;
  end
  else begin
    inherited AssignTo(Dest);
  end;
end;

Wouldn’t that be incredibly useful for quickly dumping a TObjectList?  Unfortunately, they didn’t.  TList doesn’t descend from TPersistent; it descends from TObject.  Even worse, TList<T> (the generics version of TList) and TObjectList<T> don’t descend from TPersistent (and this is where my latest aggravation comes from).  With generics lists, Embarcadero has no idea what I will put in the generic.  If they had descended from TPersistent, I could have overridden AssignTo and done whatever I want.  Not just assign to TStrings, but any other class.

In order to support mobile, I need to get away from using TLists and casting Pointers around.  Embarcadero recommends replacing TLists with generics.  Now I have a hard choice of using their generic classes and losing TStrings compatibility or making my own and losing compatibility with their generics.  Most likely, I will descend from their generics classes and graft on an Assign and AssignTo method, but it is far from ideal.Because I also need my lists/tables to persist (the IECS overrides the hash table/dictionary so that they can be read and write properties), I will have to make my own basic generics classes.

The Counter-arguments

Granted, there is the argument to be made that Embarcadero should have put the Assign/AssignTo code at the TObject level and left TPersistent for classes that truly persist themselves.  And I can completely agree with that, unfortunately they didn’t.  Perhaps in the future they will do so, it is not the first time that they added important new methods to TObject after many years of leaving them out.  I had Equals and ToString in all my objects almost a decade before they got around to adding it.  If they add them, I will rejoice (and for goodness sake, add in Clone as well).  (At the same time, I must admit I will be cursing them because now I have to handle 2 inheritance hierarchies in supporting legacy code 🙂 ).

There is also an argument that object purists will make that classes should not know about other classes.  And this one, I totally understand and I must admit it does make me a little uncomfortable.  However, I consider TStrings a basic class, only one level below TObject and TPersistent (actually, anything in Classes.pas is basic (such as TCollection) to my mind and can be supported).  And the Assign/AssignTo paradigm was expressly designed to avoid this.  TStrings does not need to know about my class to be assigned back and forth to it.  My class just needs to know about TStrings.

So the next time you are tempted to descend your incredibly useful class from TObject instead of from TPersistent, just don’t.

That’s all for now from the Delphi CodeSmith.

Greetings from a Delphi CodeSmith!

Welcome to the inaugural post of a Delphi CodeSmith!

To start, I wanted to let you know a little bit about myself and my background.  I have been using Delphi since almost the beginning (since Delphi 1 though I was not a beta tester).  I have gone through the various trials and tribulations of Delphi (16-bit to 32-bit, VCL.NET, Unicode Support, 64-bit, FMX and cross-platform, and now mobile) and its parent companies (Borland to Inprise to Borland to Codegear to Embarcadero).  Through the years I have had a lot of occasions to curse Delphi, but I have never lost that sense of wonder that I got the very first time I saw how easy it was to create visual applications.

I have been developing Delphi components since Delphi 2.  First for myself and friends, and finally for everyone with RiverSoftAVG.com.  Even more importantly, I have been maintaining Delphi components since Delphi 2, which requires a lot of patience, ingenuity and hard work. Hopefully, my experience with Delphi component development will make what I have to write worthwhile to you.

They say that “If you only have a hammer, you tend to see every problem as a nail.”  My coding toolbox has always had more than a hammer, but in my experience most desktop software would really benefit from the Delphi hammer 🙂  Seriously, Delphi makes coding fun.  I always love when the best solution is Delphi and I can convince the powers that be that I should use my hammer.

For my day job, I have been working at NASA since 1988, which I started right after college.  I have always been a software developer.  However, I have rarely been able to use Delphi at NASA, though not for lack of trying 🙂  I have developed in Fortran, C/C++, Ada, perl, CLIPS, Delphi, java, php, and javascript.  Currently, I work on the General Mission Analysis Tool (GMAT) project (it calculates burns and stuff to help spacecraft achieve their orbits and missions) and with the Earth Science Technology Office (ESTO).

That’s it for now.  Take care and Happy CodeSmithing!