Going multi-platform, part 5 : Finishing touches

phase-2-linux After weeks of hard work and learning a lot about the differences between coding on windows and linux, the current source revision of Phase 2 is completly functional on linux and playing the same as a windows version. So if you’d put them side-by-side in fullscreen mode you wouldn’t notice any differences!

Currently I’m putting the finishing toches on the linux release and only a very few minor things have to be fixed and adjusted before I’ll release the linux version into the wild, so you guys can play with it and provide me with feedback, the linux world is far more fractured than the windows world, so I’d love to know how the game runs on different distributions.

Since the last posting I did the following for the linux source path of the game :

  • Switched to the BASS audio system
    In my last posting I wrote about the problems finding an fmod linux library that would work with my game and free pascal. The problem here is that delphi/fpc support has long been dropped from fmod, and since a lot of declarations have changed, using the last delphi/fpc header with a recent fmod library wouldn’t work. So I switched over to BASS, a sound library that I already used several years ago. It still supports delphi/fpc, is leightweight, updated on a regular basis, free for non-commercial use and supports many platforms. You can get it from un4seen.
    Since my different sound systems (fmod, OpenAL) are all abstracted via TSoundSystem and TMusicPlayer classes, adding in BASS only took me 15 minutes, so the game is now running on a different sound API and the linux version now also plays music and samples.
  • Added resolution selection / switchting
    This was a lot harder than I intially thought. For windows this is pretty simple (as I’ve been using it for years), but for linux I had to do a lot of research before I started implementing it. In the end I went with xrandr and you can now change resolutions and windowed / full screen on linux like on windows.
  • Implemented performance fixes
    I hinted at this in one of my recent postings. There are several performance differences between free pascal and delphi, so code that ran pretty fast on delphi made the game crawl on free pascal. One example was the transfer market. On delphi I had all texture names for all available staffer picutes in a stringlist and on drawing each staffer entry in the selection listbox, I looked up the texture ID of the staffer’s face via this stringlist. If that image was already loaded into VRAM I draw it, if not it get’s loaded from the virtual file system. On free pascal that code made the game go down below 10 fps when opening up the transfer market window. So now I load up all staffer pictures when loading the game (only takes a few ms) and directly assing the texture ID to a staffer. Sicne this is faster in general, this will also make the delphi version run faster.

So the linux version is looking fine, and I hope to have a public release ready within the next two weeks, so the linux crowd is finally able to play my game outside of wine or virtual machines. And though it was a hard piece of work (partially making me wonder if it was worth all the hours of coding) I think it was worth it as I learned a lot about coding on linux. So for my future projects I’ll take multi platform support into account from the very beginning.

Going multi-platform, part 5 : Finishing touches

Going multi-platform, part 3 : First throwback

Last weekend I worked on getting the lazarus / free pascal build of “Phase 2” to look and play exactly the same as the delphi build. I had to change some small things here and there, but after a few hours you can now no longer distinguish between the two builds, maybe except for performance. It looks that some stuff that’s pretty fast with delphi is kinda slow with fpc, for example accessing stringlists and stuff. But that’s nothing that can’t be easily optimized, so the first goal of getting the different compiler builds on par is done. This also means that the game is fully playable under linux.

But during my playtestings on linux (I ditched my ubuntu installation with it’s horrible ubuntu one interface btw., and now use mint) I noticed that all descriptions of nations, units, technologies, etc. were missng, as well as the whole credits text. And after hours of debugging, checking encodings, creating test applications, etc. I found out that NativeXml is not able to load text contained within XML nodes on linux. So a node like this :

[xml]<unit name="test">
<description>This is a test description</description>

Will return an empty node value on Linux. No matter what I tried, accessing it directly, via the first childnode, etc. it was always empty. At first I thought I’d be doing it wrong (though it worked fine on windows), but then I did a very simple test that failed : I loaded up the XML with NativeXml and directly saved it to disk afterwards. And guess what? All texts within the xml nodes were removed by the library!

So it seems that I can’t use NativeXml for linux due to this severe bug. Their forums seem dead (registered there the weekend, but haven’t even been unlocked there so I couldn’t post the problem), and finding people that use it for linux is almost impossible. So all the thousands of lines of code I replaced to get the xml stuff to work on windows were just a waste of time, and I’ll have to remove this library again. I took a look at other xml libraries, but most either only supported delphi or were too bloated (dozens of units, huge source files, different dependencies, etc.). But well, that’s part of the learning process that makes programming so much fun (and sometimes so frustrating), and for the next external library I’m going to use (I prefer to write most of the stuff myself anyway) I’ll first check and see if it works on all my intented platforms.

And what now? Is the linux port in danger? No, rest assured, it’s not! Instead of relying on an external library for loading and saving xml I have written my own xmlwrapper that abstracts free pascal’s and delphi’s different xml interfaces into a single structure that I can use in my game (and coming projects). So now I load up an xml document via my own wrapper and compiler directives control wether delphi’s xml interface or free pascal’s DOM is used to fill this dynamic xml structure from the given file. The syntax is leaned towards the style of delphi’s xml implementation and parts of the game have already been changed to use my new xmlwrapper. So in my code it looks like this to get values from the xmlwrapper :

[delphi]VFS.LoadXML(‘data\xml\nationinfo.xml’, TmpXML);
for i := 0 to High(Nation) do
Nation[i].History := TmpXML.Root.NodesByIndex[i].Nodes[‘history’].NodeValue;
for j := 0 to LocalizationDB.LangID.Count-1 do
Nation[i].Name[j] := TmpXML.Root.NodesByIndex[i].Attributes[‘name_’+LowerCase(LocalizationDB.LangID[j])].AsString;
Nation[i].Stronghold := TmpXML.Root.NodesByIndex[i].Attributes[‘stronghold’].AsInteger;
with TmpXML.Root.NodesByIndex[i].Nodes[‘basefactor’] do
Nation[i].BaseFactor.Growth := Attributes[‘growth’].AsSingle;
Nation[i].BaseFactor.Loyality := Attributes[‘loyality’].AsSingle;
Nation[i].BaseFactor.Ressources := Attributes[‘ressources’].AsSingle;


So much for an open source xml library that was supposed to work with linux. So I guess this weekend will pretty much look like the second last one with me replacing lots of lines of xml-related code 🙁

Going multi-platform, part 3 : First throwback

Going multi-platform, part 2 : First compile and run

I’ve been working to get the current code of “Phase 2” to compile and work under linux for the whole last weekend (and my eyes kinda hurt, again sitting in front of a monitor ;)). And things went much faster than I expected, so I was able to compile and run the game under linux for the very first time ever!

First step was to add code for all things that are specific to an operating system. So in addition to all the IFDEFs that differntiate between free pascal and Delphi, there are now also IFDEFs that differntiate beween windows and linux. Luckily MacOS and linux are pretty much the same when it comes to OS specific functions, so most of the IFDEFs for linux should also work there. This includes creation and management of the render context (WGL, GLX), getting the user’s home directory (to store logs, saves, screenshots, etc.) and much more. Currently there are still several functions missing from the linux version, e.g. getting a list of available screen resolutions and the possibility to change them via the settings like on windows. I also had to install an additional package for MESA (the software OpenGL implementation running in the VM) so it supports S3TC texture compression. That’s a must for the game cause almost all textures are stored as DDS and with compression.

Second step was debugging and stepping through the code. For some reasons it seems that there are small differences between FPC (2.6.) on windows and linux. For example I’m using a THashedStringList for getting texture indices by name from my texture manager. On Delphi this worked fine, whereas on linux I only got blank white quads. In the first place I thought that the OpenGL implementation in my VM wasn’t able to load the textures, but after some (unstable, gdb often lost trace) debugging I finally found out that the THashedStringList was case-sensitive by default, and setting this parameter to false oddly had no effect at all. So I just changed my texturemanager, it now inserts and compares with uppercase texture names all the time. Same goes for the localization database, which was suffering from the same problem.

In the third step I had to remove the sound system (for now). The game uses a rather old version of FMOD for it’s soundsystem, and though FMOD works with Delphi and FPC and also win, linux, macOS, they dropped Delphi/FPC-Support some time ago. And since I don’t have the last linux package version of FMOD that I used for the windows releae, I just couldn’t get it to work. I haven’t yet found that version anywhere (not even in FMODs archives), so I guess I’ll have to go for a different path on linux. I actually once wrote a soundsystem based on OpenAL, so I may take a look at it and use that one for linux.

After these steps I was finally able to compile, link and run the game under linux, for the very first time in the long development history of the game! Yeah, that’s kind of a historic milestone for me and was worth spending a whole weekend in front of the IDE.

As for a public release it’s still a way to go. First I need to get sound in again, some OS specific stuff is still missing and I also need to setup a real linux somewhere, as the VM is nice to test but I wanna see the release running on a real linux before I’ll spread it to the public.

But yes, linux support is finally coming, and with it maybe also a MacOS release somewhere in the future!

Going multi-platform, part 2 : First compile and run

Going multi-platform, part 1 : The setup

As mentioned in my agenda for 2013 (and in several older posts), I’ve been wanting to release “Phase 2” of Projekt W on more platforms than just windows. Especially in the developer community linux has a big fan base, and the bigger the audience for the game, the better.

So shortly after the last post I updated my lazarus installation to the latest one (along with a current FPC compiler) and started to make the code compile with it. I already hinted at the fact that this required changing thousands of lines of code to make the game compileable with free pascal (which is the first step to getting it onto multiple platforms, Delphi isn’t a choice for that). And that’s exactly what I did. So for the complete last week (and weekend) I’ve put all my spare time in getting the code ready for lazarus, and considering that I’m coding 8h a day for living it was a bit tiresome, but I’m the kind of guy that likes to get things done once started.

The biggest part was to replace the all of Delphi’s xml functionality cause it’s based on windows xml libraries  and it also has a different syntax than the xml implementation of free pascal. But since I want my code to compile with Delphi and FPC (I’d like to keep all options open, e.g. Embarcadero’s announced mobile studio with android support) I opted for NativeXML instead, an open source library that’s self-contained, works with Delphi and FPC and works on different operating sytems. And it’s a lot faster than the implementation of Delphi, so the next release of the game should load a bit faster. Other than that there wasn’t much to do. Some IFDEFs here and there for slight syntax differences between FPC and Delphi (e.g. in the generics implementation) and different units to be included.

And so I was finally able to compile the game with free pascal and with a few small fixes it’s now completely playable and performance is on par with the Delphi version . There are still some visual glitches and some of the text labels aren’t filled with language specific texts, but that’s nothing that can’t be fixed within a day or two.

After I got it to compile and run with free pascal under windows the first big step towards multi-platform support was completed. And just before writing this post I’ve set up a linux and got lazarus to work there. I must admit though that it was a real pain compared to getting Lazarus set up for windows. The user interface of Ubuntu 12 is just horrible (I’m actually shocked at it’s bad usability) and getting Lazarus to work there had me install a total of 10 packages, and none of the required vesions could be installed by the package manager. So compared to the 3~4 minutes you need under windows, it took me almost an hour to get it up and running. But the virtual machine is saved, so no need to do this again.

So now everything is set up to start porting the game to Linux (with other platforms to follow), so hopefully I’ll be able to release a current snapshot of the game for two platforms, with more platforms (Mac OSX will be next) following sometime in the future.

Going multi-platform, part 1 : The setup