Map Database  •  FAQ  •  RSS  •  Login

Linux Porting Thread

<<

H.A.H.

Lance Carrier

Posts: 69

Joined: 26 Dec 2012, 23:51

Location: The Netherlands

Post 20 Jun 2014, 00:38

Linux Porting Thread

Hi all,


In some effort porting KaM Remake to Linux x86_64, I've encountered a problem I do not know how to solve. Perhaps you guys can help. I'm somehow missing the GUIMainH.rx and Tiles.rx files that are required by the RXXPacker. They were not in my KAM-TPR /data/gfx/res folder. Where can I get them?

Here is a patch that fixes some bugs (which is also inconveniently clobbered by Lazarus' window state). It also requires Linux users to rename some files:
./SpriteResources/trees.rx to Trees.rx
./SpriteResources/gui.rx to GUI.rx
./SpriteResources/guimain.rx to GUIMain.rx
./SpriteResources/houses.rx to Houses.rx
./SpriteResources/units.rx to Units.rx

http://pastebin.com/qvqgx0hw


Cheers,

Hans
<<

sado1

User avatar

Council Member

Posts: 1430

Joined: 21 May 2012, 19:13

KaM Skill Level: Skilled

Post 20 Jun 2014, 08:37

Re: Linux Porting Thread

I remember it wasn't a problem when I was playing around with compiling it. It was either already in the source or something like that... Maybe copy it over from Remake for the time being?

Would be nice to have a native Linux build of the Remake; I think you'd have to promise the developers that you will support it, though - I doubt they'd want to have a one-off Linux release, and then not have another one when a next next Remake version is released. If you plan to provide the support, though, keep in mind that it'd be nice to have it ready for RCs. I'll try to play around with your patch later on to see if I can find any bugs around.
<<

Krom

User avatar

Knights Province Developer

Posts: 3280

Joined: 09 May 2006, 22:00

KaM Skill Level: Fair

Location: Russia

Post 20 Jun 2014, 08:47

Re: Linux Porting Thread

As far as I remember, GUIMainH is unused/empty. Tiles.rx gets generated solely from SpriteResource folder.

Porting to x64 is another problem in itself, since there might be a lot of places where pointer manipulation takes place and we expect them to take exactly 4 bytes.

All in all I see little reason to make a *nix port, since that means a lot of effort in making and supporting it. However if the porting process is interesting experience for you and it wont break/overcomplicate Win version - we are fine with it.
Knights Province at: http://www.knightsprovince.com
KaM Remake at: http://www.kamremake.com
Original MBWR/WR2/AFC/FVR tools at: http://krom.reveur.de
<<

Lewin

User avatar

KaM Remake Developer

Posts: 3822

Joined: 16 Sep 2007, 22:00

KaM Skill Level: Skilled

ICQ: 269127056

Website: http://lewin.hodgman.id.au

Yahoo Messenger: lewinlewinhodgman

Location: Australia

Post 20 Jun 2014, 09:38

Re: Linux Porting Thread

As Krom says, our code is currently designed only for 32 bit. There are many places in the code where we assume that Integer, Cardinal, Boolean, object pointers, etc. are all 4 bytes (or at least the same size on every PC running the game). The first task would be to change EVERY usage of those types, which is obviously a massive task and very disruptive to any ongoing work. The best way to go about it would be to define our own versions of the types we need, so we can fix them as 4 bytes. For example:
type KInteger = LongInt; KCardinal = LongWord;
(K for KaM) Then everywhere we use Integer and Cardinal where size matters (especially where it will eventually be use in IO like save files, networking, scripting, etc.) then replace it with the "K" versions. This means the size of those types is only defined in one place rather than scattered everywhere in the code. Intger and Cardinal are the only "native" types in Delphi which will be different on 32/64 machines. Byte, Word, etc. will be the same on both 32 and 64 bit.

NOTE: This is just an idea, not a plan. Please don't do this without discussing it with both Krom and I because obviously we need to fully agree with such a massive change (and if you do it without asking we won't merge your changes). It will probably also introduce a lot of bugs so it might be best to do it after the next release (we don't like to do any structural changes when we're close to a new release, because of unexpected bugs).

On the other hand, a 32 bit Linux port should be fairly straight forward. I've successfully compiled the game on Linux several times by commenting out stuff that doesn't work for Linux yet (for example the sound library, taskbar flashing, etc.). You can use compiler directives to have separate code for Linux/Windows, which we've already done in some places.

If you are serious about making a port then I suggest you make a 32 bit Linux port first (which should not be hard, a lot of the work is already done), then later make our code 64 bit compatible (which is a massive task). Any patches you submit for a 32 bit Linux port should be fairly easy to merge and unlikely to cause bugs in the Windows version (since you will mostly be working within {$IFDEF UNIX} compiler directives). But if you want to do any big changes, it's best to ask us first.
<<

H.A.H.

Lance Carrier

Posts: 69

Joined: 26 Dec 2012, 23:51

Location: The Netherlands

Post 20 Jun 2014, 18:32

Re: Linux Porting Thread

Great advice, thank you! I will look into the 64-bit compatibility and how things work out for Linux. I think I will support 64-bit OS but let the code run under 32-bit mode, as suggested, but I need to do further research to be sure of this.

And as for my motivation, I am enthusiastic about K&M and am in love with the Remake since I've known it, but moved from Windows to Linux recently. Wine is an emulation layer for getting the game running, however it feels as a workaround solution. I suppose it is a great exercise and fun project to work this out in Linux without Wine. Also, it could be a first step towards supporting platforms other than PC, which generally are UNIX-like platforms, as it is a possible that in the long run more mobile-orientated and non-PC devices are increasingly more used: we should keep K&M around.
<<

sado1

User avatar

Council Member

Posts: 1430

Joined: 21 May 2012, 19:13

KaM Skill Level: Skilled

Post 20 Jun 2014, 21:22

Re: Linux Porting Thread

Running 32-bit binaries on 64-bit Linux isn't rocket science. You need compability libraries (lib32-* in Arch Linux for example), and while they occupy quite some space, some programs and most Steam games need them, for example, so unless you really want to avoid installing them, I cannot see many advantages of having an 64-bit Remake build. However, to compile the Remake, you'd probably need a 32bit system, 32bit chroot configured with X support, or a virtual machine for an 32bit system. Sadly I don't think there is other way to force Lazarus to produce 32bit binaries, but I might be wrong since my knowledge is much more Linux than programming oriented.

By the way, what do I need to do to be able to compile Remake in Lazarus? I applied your patch selectively (avoiding stuff which was clearly 64-bit related) but for some reason Lazarus complains about not having a lazopenglcontext package, and when I try to add it then recompile Lazarus with support for it, it complains about not having that package yet again. Is this even a proper way to do this? I am using Lubuntu 32bit in a Virtualbox.
<<

Lewin

User avatar

KaM Remake Developer

Posts: 3822

Joined: 16 Sep 2007, 22:00

KaM Skill Level: Skilled

ICQ: 269127056

Website: http://lewin.hodgman.id.au

Yahoo Messenger: lewinlewinhodgman

Location: Australia

Post 21 Jun 2014, 07:05

Re: Linux Porting Thread

By the way, what do I need to do to be able to compile Remake in Lazarus? I applied your patch selectively (avoiding stuff which was clearly 64-bit related) but for some reason Lazarus complains about not having a lazopenglcontext package, and when I try to add it then recompile Lazarus with support for it, it complains about not having that package yet again. Is this even a proper way to do this? I am using Lubuntu 32bit in a Virtualbox.
Try without using his patch, I don't think you need any of those changes for 32 bit. I just got it to compile and run on 32 bit Fedora from the trunk with just removing one "uses Windows" (I've committed a fix for that so now it should compile on 32 bit Linux without any changes). You need the folders "data\Sprites" and "data\defines" (take them from the nightly). Rebuilding the sprite packs (.rxx files) yourself is possible but probably requires changes to RXXPacker, so for now just take them from the nightly release.

Something is going wrong with loading sprite masks, so team colors aren't applied properly (everyone grey), so that needs fixing. Also sound/music isn't working, but network is. I can't judge performance/resource usage because I'm using a VM (obviously it will be better with compiler optimization enabled and debugging options disabled)
<<

H.A.H.

Lance Carrier

Posts: 69

Joined: 26 Dec 2012, 23:51

Location: The Netherlands

Post 21 Jun 2014, 23:29

Re: Linux Porting Thread

The patch is mainly concerned with RXXPacker. After rebuilding Lazarus to include LazOpenGLContext, you will also need to add the dependency into the project. Either do this using the GUI, or by editing the .lpi file:
  Code:
+++ Utils/RXXPacker/RXXPacker.lpi (working copy) @@ -29,22 +29,23 @@ <FormatVersion Value="1"/> </local> </RunParams> - <RequiredPackages Count="1"> + <RequiredPackages Count="2"> <Item1> + <PackageName Value="LazOpenGLContext"/> + </Item1> + <Item2> <PackageName Value="LCL"/> - </Item1> + </Item2> </RequiredPackages>
<<

Lewin

User avatar

KaM Remake Developer

Posts: 3822

Joined: 16 Sep 2007, 22:00

KaM Skill Level: Skilled

ICQ: 269127056

Website: http://lewin.hodgman.id.au

Yahoo Messenger: lewinlewinhodgman

Location: Australia

Post 22 Jun 2014, 04:12

Re: Linux Porting Thread

We avoid using non-standard packages/components in Delphi and Lazarus because it makes it harder for other people to start working on the project, because first they have to spend time configuring their environment correctly. That's why we include the OpenGL context stuff that we need in our Common folder (same with our network library and a few other things). The only reason the RXXPacker couldn't find it was that the include paths were setup incorrectly for that project. I've committed a fix so now the RXX Packer will compile with an untouched version of Lazarus (only uses standard packages). I didn't test whether it creates the RXX files correctly, the image libraries we use (BGRABitmap for Lazarus) might have issues on Linux.

So now RXXPacker and KaM Remake can be compiled on 32 bit Linux you can start fixing problems and adding features missing in Linux :)

Note that you need to edit KaM_Remake.inc and change
{$DEFINE USESECUREAUTH}
to
{.$DEFINE USESECUREAUTH}

So it doesn't complain out the network authentication unit being missing.
<<

sado1

User avatar

Council Member

Posts: 1430

Joined: 21 May 2012, 19:13

KaM Skill Level: Skilled

Post 22 Jun 2014, 09:22

Re: Linux Porting Thread

Thanks to your posts and fixes I managed to build it in a VM (I'm gonna try in chroot now though since it's easier and somehow VM compiled one doesn't work on host for me...). Anyway, I got a question. If HAH decided to maintain it, what are the looks and technical difficulties of having an (semi?)official Linux version that would play well with the Windows one? Would it be a problem to deal with "player X has different game files" etc.?
<<

Lewin

User avatar

KaM Remake Developer

Posts: 3822

Joined: 16 Sep 2007, 22:00

KaM Skill Level: Skilled

ICQ: 269127056

Website: http://lewin.hodgman.id.au

Yahoo Messenger: lewinlewinhodgman

Location: Australia

Post 22 Jun 2014, 11:16

Re: Linux Porting Thread

If we can make a stable Linux build then I'd like to make it official. You're right, at the moment it won't let you play with others due to having a different executable file. I can figure out a solution to that. However, the Linux builds for official releases will need to be created by me or Krom since we don't want to give anyone else our network authentication unit.

Things that need to be done:
- Fix whatever issue is causing sprite team colours to not be working
- Test that performance/resource usage is reasonable
- Test that it runs stable without crashing (it crashed a few times for me while I was testing it, I'm not sure why)
- Test that networking is stable. The server code is well tested but the client code for Linux is barely tested at all
- Test that the 32 bit Linux build can be run on a variety of Linux distros, both 32 and 64 bit (Ubuntu, Fedora, Mint, etc.)
- Make sure we are using PathDelim everywhere not hard coded "\", and check that our path names all have consistent upper/lower case (since on Windows it doesn't matter)
- Get music working (low priority, we can go without this if necessary)
- Figure out a way to test for original KaM TPR (what's the point in a Linux build if you still have to install TPR with Wine and install KaM Remake with an EXE installer?)
- Probably other stuff I haven't thought of
<<

Esthlos

User avatar

Knight

Posts: 676

Joined: 23 Jun 2013, 16:02

KaM Skill Level: Beginner

Post 22 Jun 2014, 11:45

Re: Linux Porting Thread

- Figure out a way to test for original KaM TPR (what's the point in a Linux build if you still have to install TPR with Wine and install KaM Remake with an EXE installer?)
Could it check and use directly the CD or the gog.com package?
Just when you think you know something, you have to look at it in another way, even though it may seem silly or wrong. You must try! - John Keating, "Dead Poets Society"
<<

Lewin

User avatar

KaM Remake Developer

Posts: 3822

Joined: 16 Sep 2007, 22:00

KaM Skill Level: Skilled

ICQ: 269127056

Website: http://lewin.hodgman.id.au

Yahoo Messenger: lewinlewinhodgman

Location: Australia

Post 25 Jun 2014, 04:00

Re: Linux Porting Thread

I fixed a few more issues, now the 32 bit version compiles and runs with most features working. The team colors issue I mentioned before was just a problem with my VM not supporting OpenGL 1.5. Sound is also working once you install OpenAL. And if you install a few 32 bit libraries (glibc, libX11, libgtk-x11, libgdk_pixbuf-2) on your 64 bit Linux system, you can run the 32 bit binary without any problems :)

There's still stuff to be done like detecting available resolutions, flashing the taskbar for new messages, music, etc.

Sado is begging me to include a Linux build in the release candidate, so I might try that since it seems to be running fairly well now.
- Figure out a way to test for original KaM TPR (what's the point in a Linux build if you still have to install TPR with Wine and install KaM Remake with an EXE installer?)
Could it check and use directly the CD or the gog.com package?
The problem is there are many different CDs (for each language at least) with the data compressed so it's hard to detect that generically. We could check the hash of GOG.com EXE, but then that means you can't buy the game on any other store (unless we support their EXE too) which seems a bit unfair. But something like your idea would be good.
<<

sado1

User avatar

Council Member

Posts: 1430

Joined: 21 May 2012, 19:13

KaM Skill Level: Skilled

Post 25 Jun 2014, 15:50

Re: Linux Porting Thread

I think that installing the game with Wine should also stay as an option for Linux users, because I really cannot think of any better solution for Steam users, given that there is no Linux version of the game so they simply need to either install TPR from Steam on Wine, or eventually force Steam for Linux to download the game data (there was a way to do that I think). So, we either install Remake in Wine then use the included Linux binary, or make an option in the Linux installer to browse for TPR directory.

Speaking of which... is there even a plan for an Remake Linux installer? Not that I think it's really needed (a simple .tar.gz would be enough I guess) but I don't know your plan for TPR detection.

(By the way. I needed to clean up my 32bit chroot and now I can't compile Remake anymore, it's quite possible it's my mistake but I am posting the error anyway: Common/BGRABitmap/bgrafilters.pas(206,76) Error: Can't determine which overloaded function to call)
<<

Lewin

User avatar

KaM Remake Developer

Posts: 3822

Joined: 16 Sep 2007, 22:00

KaM Skill Level: Skilled

ICQ: 269127056

Website: http://lewin.hodgman.id.au

Yahoo Messenger: lewinlewinhodgman

Location: Australia

Post 26 Jun 2014, 02:06

Re: Linux Porting Thread

I think that installing the game with Wine should also stay as an option for Linux users, because I really cannot think of any better solution for Steam users, given that there is no Linux version of the game so they simply need to either install TPR from Steam on Wine, or eventually force Steam for Linux to download the game data (there was a way to do that I think). So, we either install Remake in Wine then use the included Linux binary, or make an option in the Linux installer to browse for TPR directory.
Installing with Wine will continue to work. The Linux build only requires a single binary on top of the Windows build so there's no issue in having one release for Windows and Linux with the same files (obviously there's still a lot of stuff to do before we include Linux in public releases though).

Just out of interest, what are the advantages of not using Wine? Mostly performance?
Speaking of which... is there even a plan for an Remake Linux installer? Not that I think it's really needed (a simple .tar.gz would be enough I guess) but I don't know your plan for TPR detection.
We need to decide how to detect TPR on Linux first. As I said above I don't have a plan for that yet, I'm open to ideas. I can create a Linux installer with InnoSetup on Linux or a .tar.gz but unless we have a way to detect the original game we're allowing Linux people to run KaM Remake without the original game installed which is obviously unaccepted. The problem is to install KaM TPR you need Windows or Wine, so reusing the original game resources is kind of difficult.
(By the way. I needed to clean up my 32bit chroot and now I can't compile Remake anymore, it's quite possible it's my mistake but I am posting the error anyway: Common/BGRABitmap/bgrafilters.pas(206,76) Error: Can't determine which overloaded function to call)
That sounds like you might be using 64 bit Lazarus/FPC... double check that you are running the 32 bit version and compiling for 32 bit.

Return to “Bugs”

Who is online

Users browsing this forum: Ahrefs [Bot] and 4 guests