Development History
KaM Remake - Main
Even though there have been a few attempts at recreating the Knights and Merchants engine from scratch, one of these attempts stood out from the rest and is in active development even today. Below you will find some back story regarding the development of this remake and its humble beginnings.
In March 2008 Krom decided to start work on a "KaM Remake" project based on his KaM Map Editor. A friend known as "Alex" helped Krom during the initial development to create a structure for players, units and houses, because Krom's editor had previously only focused on terrain with no live simulation taking place. Krom started with an extensive knowledge of Delphi and OpenGL and aimed to improve his knowledge and gain other valuable experience through the KaM Remake project. It began as a Remake of The Shattered Kingdom as Krom disliked many of the changes in The Peasants Rebellion. In May Krom made the first public announcement about the Remake on his site with a few screenshots.
“KaM Remake is planned to reach at least KaM TSK level without any changes nor radical new features. Then we'll see. I don't want to plan something out of reach now.”
— Krom's news feed, 21/05/2008
Near the end of 2008 Lewin joined the project. He had experience in Lazarus 
(similar to Delphi but freeware) from his mission editor, which was one of his first programming experiences. Neither Krom or Lewin had worked in a team before or taken on such an extensive challenge. Lewin was far less experienced but Krom patiently helped him and they formed a very solid team together. Krom began working on a custom controls library (buttons, labels, etc.) rendered in OpenGL, while Lewin went through the painstaking task of recreating the KaM interfaces by taking screenshots and lining everything up down to the last pixel. Lewin used his knowledge of the KaM text files (.lib) and mission files (.dat) to make the Remake load them natively from the original KaM resources. Krom already had the sprites and maps loading from his map editor.
During 2009 the team slowly worked on recreating the economy side of the game engine including A* pathfinding, a priority bidding delivery system for serfs and house construction. By July they had there first public demo ready, over a year after the project started. The demo was titled "KaM Remake Economy Demo 2009-07-28 (r363)" and it allowed you to build a village and let it run. There was no fighting and no unit interaction, meaning units simply walked through each other rather than resolving traffic jams and each tile on containing one unit at once. In September they released the 2nd Economy Demo (r428) which fixed bugs from the previous version and added more features. During the last quarter of 2009 the developers focused on unit interaction and eventually created a series of algorithms that produced results much like KaM, and more efficient in some cases.
In February 2010 the 3rd Economy demo (r673) was released, and this time it included unit interaction! Now that the serfs and other units walked around each other correctly it was really starting to feel like a game and the village simulation was very much like Knights and Merchants. Development continued throughout 2010 and by August a new demo was released entitled "Fighting Demo (r970)" and you guessed it, soldiers would now fight! Only melee fighting was implemented in this demo, as there was no code to handle projectiles such as arrows and rocks yet. But at the end of 2010 the 2nd Fighting Demo (r1345) added ranged attacks along with a few other features. Even though the AI was still incomplete, it was now possible to play a proper game against the computer.
Now that singleplayer was mostly complete the team started to consider multiplayer. They already had replays for singleplayer which recorded commands from the user such as placing a building or moving a soldier. The plan had always been to use this same system to allow multiplayer, by delaying commands from the user long enough to send them to the other players. Krom had found an article from the developers of Age of Empires titled "1500 Archers on a 28.8: Network Programming in Age of Empires and Beyond" which described a system like this. A free UDP network library was found and after extensive discussion the development began. By April 2011 basic networking was working! Screenshots were posted showing a multiplayer game running on two separate computers. (see right)
However there were still some issues regarding networking: Because they were using UDP, packets were occasionally lost or arrived out of order. It was also hard to scale the game up to more players because every computer was required to send data to every other computer. Neither Krom nor Lewin had much experience with networking, but Siegfried from the English forum suggested they switch to TCP which would guarantee packets were not corrupted or lost and that packets arrived in the order they were sent. The only downside was slightly higher latency (ping) and slightly more traffic overhead. If they stuck with UDP, they would end up implementing many of these systems by hand which would most likely give a worse result than using TCP, which is well known and trusted. Heeding this sound advice the team switched from UDP to TCP, and also after much discussion switched their topology to use a central server, rather than every client connecting to every other client. The logic behind this was that on a LAN the topology would work either way, but to enable internet play a central server made more sense, as otherwise every player would need to open their firewall to allow the data from the other players back in. It meant that dedicated servers could host games and each player would establish a TCP connection to this server, which would simply act as a relay for the data.
The first multiplayer demo (r1793) was released in June 2011 which allowed simple multiplayer over a LAN. It wasn't really playable because games still became out of sync after a while. An update to the demo (r1902) released early in July fixed a number of bugs and added new functionality to the lobby including ping measurement and a dedicated server, but still suffered from the out of sync issues. Finally a massive update (r2039) released late in July fixed this issue as well as adding many new features including multiplayer saves. Multiplayer was finally working! However, in this demo the servers would get data corruption errors and crash regularly, meaning it was not playable. Less than a week later r2087 fixed this bug and added rooms to the dedicated server, meaning you could play multiple games at once.
xzaz from the forum was hosting a dedicated server and Siegfried suggested the
community organised a time to play KaM every weekend, otherwise it was hard to find someone to play a game with. The developers liked the idea and 9pm CET on Sundays was agreed on as the playdate. On August 21 the first game was played over the internet for almost 2 hours by three players (Lego3, The_French and Shadaoe) before two of the players were randomly disconnected. (a known issue due to the fact that no attempt to reconnect was made when the TCP connect dropped out randomly, as happens over the internet) The screenshot to the right is from this first ever online game. After this success the weekly matches became very popular and a few weeks later over 16 players turned up to play!
“You just made history. Be aware of that - one day you may be proud of it”
— Siegfried, 21/08/2011
It was clear to the KaM Remake developers that multiplayer was a very popular feature, so they focused their efforts on that. It was also apparent that the Remake was becoming playable as a complete gaming experience. A master server was developed to show the list of games running around the world and numerous other features including support for soldiers from The Peasants Rebellion (though not siege equipment) and fully working voices. During this time StarGazer, an alpha tester from the development of The Shattered Kingdom, sent the developers the sprites for a building from the Knights and Merchants: The Shattered Kingdom alpha, which was not included in the final game. The new building was a marketplace! Once the code for buildings had been refactored to allow the addition of new buildings the marketplace was implemented and functional, allowing you to trade one resource to another at a high rates. Given the attention the KaM Remake was attracting the developers also decided it would be nice to create a special website for the KaM Remake. With the help of Bur, the admin of the Russian KaM fansite kamclub.ru they soon had a site running in WordPress at www.kamremake.com.
A few weeks after the site was publicly announced the "1st Merchants Demo r2411" was released on October 16. On the first day it received over 600 downloads which caused the download servers to run very slowly, as the total downloads in the first 24 hours summed to around 86 GB. This level of popularity was completely unexpected by the developers, who were still planning to organise weekly matches every weekend. However, every day since release there has been players online, peaking up to 40 simultaneously! It was now possible to play against other humans any day you wanted, meaning it wasn't necessary to organise weekly matches. This release reached around 3000 downloads in less than two months, by which time the next update was available.
The success of r2411 gave the KaM Remake developers a lot of motivation. Nearly 4 years of hard work had finally paid off. There were a number of rare crashes and minor issues in r2411 which were quickly fixed, and the team also set about implementing an "auto reconnect" system which means a player will be automatically reconnected if they are randomly disconnected due to packet loss or other errors over the internet/network. The "3rd multiplayer demo r2678" was released on December 11 2011 and the changelog was massive, including over 50 bug fixes, 20 new multiplayer maps and sprites to show wares being delivered to the marketplace, thanks to Malin from kamclub.ru. However, this version introduced a few uncommon but fatal crashes, so a minor update was planned to be released within a few weeks.