Category Archives: Coding

Delphi for iOS and Android: The Natives are restless

One of the major marketing points from Embarcadero is that Delphi will create applications that are “native” to your mobile platform.  One of the major complaints that I see from people trying Delphi for mobile platforms is the disappointment about the speed and appearance of these “native” applications.  This is a classic case of a company overselling their product, inflating the expectations of their customers, and then disappointing them.

So is Delphi creating “native” applications for iOS and Android?  The answer is yes, yes-ish, and not at all. 🙂  What you need to realize is that there are many ways that an application can be native (or not) to a platform.  I am going to talk about 3 ways to help you understand what Delphi is good and not-so-good for.  There is native to the hardware, native to the OS, and native look and feel.

Native to the hardware

Native to the hardware means that your code is compiled down to actual machine instructions of the hardware.  This is the area that Embarcadero has done a great job and is really what their marketing department is selling you.  Unlike Java or Python or php, Delphi’s code is not interpreted by some engine or virtual machine.  After the compiling and linking are finished, that code that adds 2 numbers goes directly to the hardware without any layer in-between.  This is true for Windows, for MacOS, for iOS, and for Android.

Note that this doesn’t mean that you cannot write fast interpreted code or slow compiled code.  The algorithms you write and the optimizations that the interpreter/compiler does and does not perform can greatly affect execution speed.  It does mean that Delphi has an in-built advantage over interpreted code here though.

Note 2: Native to the Hardware could also mean direct access to the hardware (such as GPS, camera, etc) of the device, but unless you are writing device drivers, noone does direct access to that kind of hardware.  It is more properly a function of the OS APIs which is discussed in the following section.

Native to the OS

What I mean by native to the OS is does a program have direct access to the API and controls of the Operating System of the device?  For Delphi and iOS, this is an *equivocal* yes.  For Delphi and Android, this is a *good enough*.  Almost by definition, unless your development language is the same language used by the OS, you will always have to use a hopefully thin application programming interface to the OS and its controls (this is one area where C++Builder can do better than Delphi on iOS).  Since iOS’ “native” language is objective-C and (even worse from Delphi’s standpoint), Android’s “native” language is Java, Delphi is not “native” to the OS.  However, that does not mean it is a second class citizen for development.

You see, Delphi, at least the Delphi that everyone thinks about, has never had direct access to the API and controls of the Operating System.  One of the things that made Delphi great is its true Rapid Application Development of applications using its VCL.  The VCL is not direct access to the Window’s API.  At best from the standpoint of direct access, you had to use C headers to access Window’s services and at worst, the VCL was a thin wrapper around Window’s controls.  The TListBox is not native to Windows – it is a beautifully logical wrapper around a Window’s object.  Delphi sacrificed native access to the API and controls to give you Rapid Application Development (RAD).  The goal, however, has been to keep the layer’s between Delphi and the OS as thin and fast as possible, while keeping RAD and now adding cross-platform development.

So Native to the OS is a qualified yes.  For Windows, OSX, and iOS (as they are all based on C/C++), access to OS services through APIs is fast and clean.  For Android, the access is not so good as Delphi needs to use a JNI (Java Native Interface) layer, which is expensive relative to C APIs.

Delphi’s cross-platform library, Firemonkey or FMX, wrap services like Location and the Camera in what I believe is a very good way for cross-platform development.  These classes help hide the complexity of the underlying service and abstracts it for all platforms that Delphi supports.  They are thin but good layers to the underlying OS services.

However, access to the native OS controls is another story and is the subject of my next section.

Native Look and Feel

Native Look and Feel is the area where everyone is getting disappointed.  Delphi mobile apps can look slightly wrong, feel slightly sluggish, and act in a non-standard way (relative to OS controls).  This is the consequence of the path Embarcadero chose to implement cross-platform controls like TListBox, TMemo, etc.

The VCL, unlike FMX, is a thin wrapper around native Window’s controls and so shares Windows look and feel and smooth responsiveness (insert Mac fanboy disparaging comments here ;-)).  When you click and drag your mouse on Windows, you are clicking and dragging on Window’s controls.  When the application runs on a new Windows OS, it inherits (mostly) the new look and feel.  The VCL communicates through Windows messages with the underlying OS controls and everything just works.

FMX emulates all controls of the underlying OS.  It draws every pixel itself and handles every user interaction.  This is the cause of the slightly wrong look and feel.  This is why a Delphi TListBox will have different bugs than Android’s list box.  This is why it will refresh slower and respond slower than the iOS list box.  This is also why, when a new OS is published, Delphi apps do not update their look and feel and can look out of date.

So, despite Embarcadero’s claims of pixel-perfect compatibility, Delphi does not have NATIVE look and feel.  FMX still provides the RAPID in RAD, just not native look and feel.

I wrote that this is a consequence of Embarcadero’s decision, and it is.  And I really cannot blame them for the decision.  With the VCL, Embarcadero/Borland chose the thin wrapper over the OS approach.  It works beautifully, but it ties the component library to the OS.  Embarcadero could have chosen this approach for FMX but it is really, really hard and we would still be waiting for cross-platform compatibility.  They would have had to develop a cross-platform abstraction for every control in FMX (a herculean task by itself and maybe not possible).  Then, for every platform they support, they would have to develop and maintain an FMX version.  Embarcadero chose to “get in the game” as fast as possible.  I believe it was the right decision, but it does mean that they have to deal with the consequences of their choice.

Note: the bugginess and responsiveness will improve (and God do I know that there are bugs!).  Every version Embarcadero has been improving the FMX library.  While it will never have native look and feel, improved look and feel will occur.

Note 2: you can have native look and feel on Windows and iOS.  Choose the VCL for Windows, and choose something like D.P.F Delphi iOS Native Components (http://sourceforge.net/projects/dpfdelphiios/) or TMS’s iCL (http://www.tmssoftware.com/site/tmsicl.asp), which do the same thing for iOS.  Of course, you sacrifice the cross-platform development… though you get to keep using your Delphi skillset and with proper modularization can hopefully keep the incompatibilities to a minimum.

Delphi’s Strengths and Weaknesses

So what does all this mean for developers?  It means that Delphi has its strengths and weaknesses, and as long as you are aware of them, you can choose Delphi for the right jobs.

If you need cross-platform compatibility and want to deal with only one code base (mostly), Delphi is an excellent choice.  It provides nice abstractions of the OS and its services, a relatively pretty GUI library, and native (very fast) access to the CPU.  Delphi also has excellent DB connectivity, web services connectivity, and networking in general.  This means Delphi is a good choice for

  • Enterprise developers, who want to provide mobile access and don’t really care about pixel-perfect look and feel
  • Scientific and number crunching developers, who need fast processing and a way to nicely display their results
  • Game developers, surprisingly enough, who want to develop cross-platform games which don’t have “native” interfaces anyway and where FMX can provide fast-enough graphics.  (In other words, not Madden level graphics but Angry Birds).
  • “Light” Apps which don’t use a bazillion controls to interact with the user and don’t need “pixel-perfect” responsiveness
  • Compelling apps where the user will forgive some idiosyncrasies because the app is so good.  I mention this because Delphi allows you to concentrate on the app.  Delphi provides the RAD and the cross-platform compatibility, you can concentrate on making the killer app.

Hopefully, this article helps you understand the “nativeness” of Delphi.  I believe if you choose Delphi with eyes open to its strengths and limitations and how it can help you solve your problem, it is an excellent choice.

Happy CodeSmithing!

AUTOREFCOUNT and the Free Pattern

The more I use the Delphi mobile compiler and its automatic reference counting (ARC), the more I am coming to the conclusion that Embarcadero chose the wrong implementation pattern for the Free code.  Now I am not getting into the pros and cons of ARC here, I am just referring to how the Free method in the ARC compiler is mapped to nil in mobile code.  It should have stayed a Free/DisposeOf method instead.

Why do I think it was mistake?

1.  If Free is used in ARC code, the developer is porting code from other compilers.  He/she wants this behavior.  If the developer is writing from scratch for mobile (and only mobile), he will omit the Free call (or use DisposeOf).  Even worse, for the component developer, the DisposeOf option is not available.  We have to leave the Free call in as the code must work in earlier versions of Delphi.

2. The ARC compiler’s unfortunate decision to output a hint when it encounters a Free call means I usually end up mapping the Free to DisposeOf with very ugly IFDEFs anyway to get rid of the hint, e.g.,

aObject.{$IFDEF AUTOREFCOUNT}DisposeOf;{$ELSE}Free;{$ENDIF}

3. Mapping Free to nil makes destruction of objects non-deterministic from the developer’s point of view.  From the point of view of the program, of course, it is completely deterministic: destruction occurs when there are no more references to the object.  However, from the developer’s point of view, mapping Free to reference = nil is not deterministic, as the call to Free may not do anything!  It is especially bad as the behavior from non-ARC to ARC changes.  Often, I depend on the destruction of an object to notify other objects.  If that destruction doesn’t happen or happens at a different time, bad things can happen.

Objections

Now, there are plenty of objections you can make to this.

ARC is supposed to make our code more robust, and the Free method bypasses that safety.  However, Embarcadero supplies the DisposeOf method to get around that safety anyway!  Why couldn’t the Free call have stayed the method for forcing destruction?

Disposing of the object when there are still references can cause access violations!  Well, maybe… and maybe not.  The TComponent memory management architecture (using notification) means that there can be multiple references to a component, but that the destruction of a component can clean them all up.  And I would argue that I *want* to see these problems otherwise.  I am porting from a non-memory managed compiler.  If I haven’t cleaned this up properly on non-ARC, it is a bug.  It should be a bug here.

What happens in practice, except for simple classes, is that the Destroy code gets called at an unexpected moment and then causes access violations or other unexpected behavior that the developer did not expect.  It is very difficult to debug this late destruction behavior.

So to close, the Free method should have stayed Free! (I feel very patriotic for some reason right now 🙂 )  Not only would it have made the look of the code much better (without ugly IFDEFs), the behavior of the code between compilers would have stayed more alike and easier to debug. 

 

Listening is Learning

Quick note: On August 14th, Ray Konopka had his Seeing is Believing: Data Visualization in Multi-Device Apps with RAD Studio.  I highly recommend watching this webinar.  It is one of best webinars for Delphi and FireMonkey you can watch.  Unlike some webinars that are more marketing than material (or that the title of this webinar even suggests 🙂 ), the webinar is dense with information and presented in a very clear and logical format.  Ray wrote one of the best early Delphi books and it shows.  This guy knows his stuff.  I know I learned a lot about FireMonkey component development from watching this webinar.  Highly recommended, thumbs up, and five stars! 🙂

Disclaimer: I have worked with Ray on Raize Software’s Inspex component. 

Using Primitive Types with TStrings in iOS

Problem

One of the unwelcome surprises with coding in Delphi for iOS is that you cannot store primitive types (integer, string, single, etc) in TStrings.Objects.  Usually, I am storing TObjects in the TStrings but occasionally it needs to be a primitive type (such as a score).   I would never do this for components but for programs where I know exactly what I place in the TStrings, I find nothing wrong with this and it is incredibly useful.

But with Delphi for iOS, when you try:

ListBox1.Items.AddObject('Tom', TObject(100));

It blows up.  In retrospect (and with a little digging with the debugger), it is obvious why you cannot.  With an automatic reference counting compiler, any new references to an object means that the compiler inserts calls to __ObjAddRef when you add an object to the Listbox’s list.  Since the TObject(100) is not a real object, kaboom!

Now if Delphi was a “rooted” language (i.e., even primitive types have a TObject equivalent, similar to .NET or java), this would not be a problem.  The primitive would automatically be boxed and unboxed as needed to store it inside a TObject.  However, Delphi is not a rooted language, at least not yet.   Well, the technique is so useful I was unwilling to give it up.

Solution

By using generics and class operators, I have come up with a generic class for boxing and unboxing primitive types (actually any type) which I find pretty slick.  Before I get into the details, I want to show how you would use it.  To add a primitive to a TStrings, just use the TBoxXXX cast where XXX is the primitive type name:

ListBox1.Items.AddObject('Hello', TBoxInteger(10));
ListBox1.Items.AddObject('World', TBoxString('It is a nice day'));
ListBox1.Items.AddObject('How', TBoxDouble(20.23334));

To unbox back to the primitive type, it is a little more complicated.  First, you cast your TObject back to its TBoxXXX type and then you can cast it either explicitly or implicitly back to the primitive type:

var
  i: Integer;
begin
  i := TBoxInteger(ListBox1.Items.Objects[0]);
  ShowMessage(i.ToString);
end;

Easy as can be! 🙂  The one wrinkle is that the IndexOfObject method will no longer work because the method compares the TObject to the TBoxInteger object and not its value.  To handle that case, I have added an IndexOf class method:

ShowMessage(TBoxInteger.IndexOf(ListBox1.Items, 100).ToString);

Code

So how does the code work?  The whole trick is to define a generic boxing class.  This class contains a private field containing the primitive value.  It then defines implicit and explicit conversions from the primitive to the boxing object and back.  It adds the previously mentioned IndexOf method and defines the Equal class operator to make it work.  The full declaration is thus:

TRSBoxPrimitive<T> = class(TObject)
{ Purpose: To provide a simple class for boxing and unboxing a primitive type
Note: this class is only for use by AUTOREFCOUNT compilers }
private
  { private declarations }
  FValue: T;
protected
  { protected declarations }
public
  { public declarations }
  constructor Create( const V: T );
  class operator Equal(a,b: TRSBoxPrimitive<T>): Boolean;
  class operator NotEqual(a,b: TRSBoxPrimitive<T>): Boolean;
  class function IndexOf( const Strings: TStrings; a: T ): Integer;
  class operator Implicit(a: T): TRSBoxPrimitive<T>; 
  class operator Implicit(a: TRSBoxPrimitive<T>): T;
  class operator Explicit(a: T): TRSBoxPrimitive<T>; 
  class operator Explicit(a: TRSBoxPrimitive<T>): T; 
end; { TRSBoxPrimitive<T> }

When you type TBoxInteger(100), the Explicit(a: T): TRSBoxPrimitive<T> method is called:

class operator TRSBoxPrimitive&lt;T&gt;.Explicit(a: T): TRSBoxPrimitive&lt;T&gt;;
begin
  result := TRSBoxPrimitive&lt;T&gt;.Create(a);
end;

The line assigning the boxed value back to an integer calls the Implicit(a: TRSBoxPrimitive<T>): T method:

class operator TRSBoxPrimitive&lt;T&gt;.Implicit(a: TRSBoxPrimitive&lt;T&gt;): T;
begin
  result := a.FValue;
end;

The final code creates primitive boxing types from the generic type:

TBoxInteger = TRSBoxPrimitive&lt;Integer&gt;;
TBoxString = TRSBoxPrimitive&lt;String&gt;;
TBoxDouble = TRSBoxPrimitive&lt;Double&gt;;
...

The code takes advantage of the fact that the TBoxXXX classes are reference counted and will be automatically destroyed.  Otherwise, this code would leak like a sieve!  A cool bonus with this code is that the boxing allows primitive types that would not usually fit in the size of TObject.

Hopefully, this code is as useful to you as it is to me.  The full code is available here for you to use as you see fit (but please keep the attribution and let me know if you improve it)

Happy CodeSmithing!

 

TStringList.Assign… Bug, Feature, or Documentation Error?

I was recently revisiting my improved implementation of TStringList (see http://www.riversoftavg.com/efficiencytlistntstringlist.htm).  Now this code is very old, but its point about the efficiencies to be gained when copying sorted TStringLists applies.  Unfortunately, my stringlist, and Delphi’s TStringList, has a bug.

Suppose you have 2 TStringLists and the following code:

Strings1.Assign(Strings2);
if not Strings1.Equals(Strings2) then
  raise EListError.Create('Copy unsuccessful');

Do you think the exception will ever occur?  If you are like me, you would say no, and you would be wrong.

Here is the TStringList.Assign method:

procedure TStringList.Assign(Source: TPersistent);
begin
 inherited Assign(Source);
 if Source is TStringList then
 begin
   FCaseSensitive := TStringList(Source).FCaseSensitive;
   FDuplicates := TStringList(Source).FDuplicates;
   FSorted := TStringList(Source).FSorted;
 end;
end;

The inherited Assign clears the TStrings, copies some properties over, and then copies the strings using AddStrings.  The problem is subtle, but the error is that TStringList sets the Self properties after the strings have been copied (To be precise, the trouble is that the Self TStringList does not set the properties properly before copying the strings.  But more on that in a moment).

If Self has its Sorted property set to True, things start to take a wrong turn.  Basically, 3 errors can occur:

  • If the Source TStringList is not sorted, the 2 TStringLists are guaranteed to not be equal, e.g.,
Source := TStringList.Create;
Source.Add('Hello');
Source.Add('World');
Source.Add('Hello');
Dest := TStringList.Create;
Dest.Sorted := True;
Dest.Assign(Source);
if not Dest.Equals(Source) then raise exception.create('oops');
  • If the Source TString is sorted but the CaseSensitive property is not the same, the 2 TStringLists could possibly not be equal as they sort the strings differently. 
  • If the Source TStringList is sorted but contains duplicates, the 2 TStringLists could still not be equal.  The default value for the Duplicates property is dupIgnore, which means that the Self TStringList could be just throwing those duplicates away.  So not only are they not in the same order, but the 2 TStringLists don’t even contain the same number of items!

So how should the Assign method look?  The first thought is to put the TStringList property assigns before the inherited call, e.g.,

procedure TStringList.Assign(Source: TPersistent);
begin
 if Source is TStringList then
 begin
   FCaseSensitive := TStringList(Source).FCaseSensitive;
   FDuplicates := TStringList(Source).FDuplicates;
   FSorted := TStringList(Source).FSorted;
 end;
 inherited Assign(Source);
end;

This is better, but there is still a bug.  The problem is that even if the Source TStringList is Sorted and Duplicates = dupIgnore, there still may be duplicates in the Source TStringList!  Setting the Duplicates property does not actually do anything – it only works when strings are added.  So the Source TStringList could have added duplicate strings and then set its Duplicates property to dupIgnore or dupError afterwards.

The solution is to temporarily set the Self TStringList to unsorted and Duplicates to dupAccept.  Copy the strings and then copy the actual property values.  So, our final code looks like this:

procedure TStringList.Assign(Source: TPersistent);
begin
 if Source is TStringList then
 begin
   FDuplicates := dupAccept;
   FSorted := False;
 end;
 inherited Assign(Source);
 if Source is TStringList then
 begin
   FCaseSensitive := TStringList(Source).FCaseSensitive;
   FDuplicates := TStringList(Source).FDuplicates;
   FSorted := TStringList(Source).FSorted;
 end;
end;

One thing I am unsure about.  What should be done if the Source is a TStrings but is not implemented by a TStringList?  In that case, should the strings be sorted or not as they are added to Self?  My decision was that the Self TStringList should keep its own property values, even if that means the 2 objects, Self and Source, will not be equal.

Postscript

So will this “bug” ever be fixed?  My instinct says that it won’t.  This behavior has been around way too long and has become the expected behavior.  After all, the current behavior is a quick way to strip duplicate strings from a TStrings.  Changing the behavior now may break user’s code.  So that is why the title of this post asks is it a bug, is it a feature, or is it just a documentation error?

Happy CodeSmithing!

Delphi Component Writers… Descend from TPersistent!

I have been busy converting the RiverSoftAVG code to support Delphi XE4.  I promise to discuss this process in more detail later, but right now, I have to talk about a pet peeve of mine.  I guarantee my recommendation is controversial with some people, and perhaps, I should avoid it for my very first real blog post, but I just have to go with it…

Descend your objects from TPersistent, not TObject

It is amazing how useful the TPersistent class is, and how little it is actually used.  Embarcadero is especially guilty of this.  The TPersistent class is not just about persisting a class, it is arguably more useful to use the class to copy from one class to another.  Unfortunately, Embarcadero leaves large swaths of the Vcl/FMX library descending from TObject instead of TPersistent.

One of my favorite tricks when developing a class is to use the AssignTo protected method as a method to assign my class to a TStrings object.  I override the AssignTo method, detect if the destination object is TStrings and then fill the TStrings with appropriate text.

For example, my handy-dandy string some type of list/collection class could have:

procedure TMyStringCollection.AssignTo(Dest: TPersistent);
begin
  if Dest is TStrings then begin
    TStrings(Dest).BeginUpdate;
    try
      TStrings(Dest).Clear;
      for i := 0 to Count - 1 do
        TStrings(Dest).AddObject(MyString[i], MyItem[i]);
    finally
      TStrings(Dest).EndUpdate;
    end;
  end
  else begin
    inherited AssignTo(Dest);
  end;
end;

This little trick makes it easy to quickly see the contents of my list in a TListBox, TMemo,… you name it.  You just do code like:

ListBox1.Items := MyStringCollection1;

If Embarcadero had made TObjectList descend from a persistent TList, that AssignTo could easily have been:

procedure TObjectList.AssignTo(Dest: TPersistent);
begin
  if Dest is TStrings then begin
    TStrings(Dest).BeginUpdate;
    try
      TStrings(Dest).Clear;
      for i := 0 to Count - 1 do
        TStrings(Dest).AddObject(Items[i].ToString, Items[i]);
    finally
      TStrings(Dest).EndUpdate;
    end;
  end
  else begin
    inherited AssignTo(Dest);
  end;
end;

Wouldn’t that be incredibly useful for quickly dumping a TObjectList?  Unfortunately, they didn’t.  TList doesn’t descend from TPersistent; it descends from TObject.  Even worse, TList<T> (the generics version of TList) and TObjectList<T> don’t descend from TPersistent (and this is where my latest aggravation comes from).  With generics lists, Embarcadero has no idea what I will put in the generic.  If they had descended from TPersistent, I could have overridden AssignTo and done whatever I want.  Not just assign to TStrings, but any other class.

In order to support mobile, I need to get away from using TLists and casting Pointers around.  Embarcadero recommends replacing TLists with generics.  Now I have a hard choice of using their generic classes and losing TStrings compatibility or making my own and losing compatibility with their generics.  Most likely, I will descend from their generics classes and graft on an Assign and AssignTo method, but it is far from ideal.Because I also need my lists/tables to persist (the IECS overrides the hash table/dictionary so that they can be read and write properties), I will have to make my own basic generics classes.

The Counter-arguments

Granted, there is the argument to be made that Embarcadero should have put the Assign/AssignTo code at the TObject level and left TPersistent for classes that truly persist themselves.  And I can completely agree with that, unfortunately they didn’t.  Perhaps in the future they will do so, it is not the first time that they added important new methods to TObject after many years of leaving them out.  I had Equals and ToString in all my objects almost a decade before they got around to adding it.  If they add them, I will rejoice (and for goodness sake, add in Clone as well).  (At the same time, I must admit I will be cursing them because now I have to handle 2 inheritance hierarchies in supporting legacy code 🙂 ).

There is also an argument that object purists will make that classes should not know about other classes.  And this one, I totally understand and I must admit it does make me a little uncomfortable.  However, I consider TStrings a basic class, only one level below TObject and TPersistent (actually, anything in Classes.pas is basic (such as TCollection) to my mind and can be supported).  And the Assign/AssignTo paradigm was expressly designed to avoid this.  TStrings does not need to know about my class to be assigned back and forth to it.  My class just needs to know about TStrings.

So the next time you are tempted to descend your incredibly useful class from TObject instead of from TPersistent, just don’t.

That’s all for now from the Delphi CodeSmith.