Tag Archives: KODev

Knight Online Terrain (GTD format for 1299)

I want to talk a little bit about reading from binary files using C/C++ and the file structure for the Knight Online terrain files (i.e. files with the extension “.gtd”). To follow along you need two things:

  1. A C/C++ compiler, here I’ll be using Visual C++ (Community 2015)
  2. A 1299 terrain file, I’m using “karus_start.gtd” but any .gtd will work as long as it’s from the 1299 version of the game – however I’d recommend that you also use “karus_start.gtd” so that our number match up.

The basic concepts I’ll discuss here are valid for reading any of the Knight Online content files. Also, if a particular map spans multiple versions of the game (with each version varying in the file structure) you can use the data you know is required for the 1299 version (discussed in this post) to figure out what was added/removed in other versions. Therefore, even though we’ll be discussing the 1299 file format this information is relevant for cracking all versions of the Knight Online terrain files. Here I’ll be assuming you have little to no C/C++ knowledge. However, I am assuming you know general programming concepts and the difference between memory on the stack vs. memory on the heap.

First let’s read in the .gtd file.

/*
*/

#include "stdio.h"

//-----------------------------------------------------------------------------
int main(int argc, char** argv) {

	char filename[] = "karus_start.gtd";

	FILE* fp = fopen(filename, "rb");
	if(fp == NULL) {
		printf("ERROR: Unable to open file "%s"n", filename);
		return -1;
	}

	fclose(fp);

	return 0;
}

The function int main(int argc, char** argv) is the main entry point for the program. #include “stdio.h” includes the declarations for all standard input and output operations. char filename[] = “karus_start.gtd”; defines a variable name “filename” which is a pointer to an array of “char”s allocated on the stack. FILE* fp = fopen(filename, “rb”); opens the file “karus_start.gtd” in the “read binary” mode. This means we will only be reading from (as opposed to writing to) the file and we wish to treat the contents of the file as if it were all binary data. The rest of the code here checks to make sure we successfully opened the file (printing an error if we were unable to open it) and immediately closing the file and exiting the program.

So far this program don’t do much. We are just opening the file (if it exists) and closing it. What we want to do is start reading in the contents of the file. We can’t read in the contents unless we know a little bit about how the terrain information is structured. So before we start read in all the binary information let’s take a moment to discuss the .gtd file at a higher level.

Having played the game you have probably noticed how the maps are broken up into these little squares.

Each of these rectangles is referred to as a “tile” and each tile is a 4×4 rectangle. The map data for a tile consists of four points in 3D space. In this coordinate system X and Z run along the floor and Y points up into the sky. Therefore, since we know that tiles are 4×4 rectangles, the X and Z coordinates for all the tiles on the map can be set just by knowing how big the map is (this will be important later because only the Y coordinate for these four points is stored in the .gtd file). The idea of a tile isn’t enough though, we also need the concept of a “patch.” A patch consists of a rectangle of 8×8 tiles. Patches carry absolutely no information about how the map should be rendered. Patches carry information for how the map should be updated and are useful for increasing computational efficiency. We eventually plan to have trees blowing in the wind or NPCs performing particular animations, etc. – it would be a waste of CPU power to have trees located on the opposite side of the map blowing in the wind when the player isn’t anywhere near them. Patches are useful for only updating objects which are close to the player.

Interestingly, the 1299 .gtd files start with an 4-byte integer which we do not know the function of (I’d guess it’s likely a version number).

int iIdk;
fread(&iIdk, sizeof(int), 1, fp);

An int in Visual C++ is 4 bytes. int iIdk; allocates the 4 bytes on the stack which we will use to store the first 4 bytes of the .gtd file. fread(&iIdk, sizeof(int), 1, fp); actually reads 4 bytes from the file and copies them into the 4 bytes we have allocated. Next we read in the name of the map.

int iNL;
fread(&iNL, sizeof(int), 1, fp);

char m_szName[0xFF] = "";
if(iNL > 0) fread(m_szName, sizeof(char), iNL, fp);

Here we are reading in another 4-byte integer, although this time it is the length of the map name. This integer tells us how many characters there are in the map’s name. Now we’ll read in the size of the map, set the number of patches based on the map size, and grab all the map data.

int m_ti_MapSize = 0;
fread(&m_ti_MapSize, sizeof(int), 1, fp);
int m_pat_MapSize = (m_ti_MapSize - 1) / 8;

_N3MapData* m_pMapData = new _N3MapData[m_ti_MapSize*m_ti_MapSize];
fread(m_pMapData, sizeof(_N3MapData), m_ti_MapSize*m_ti_MapSize, fp);

Based on the code we have looked at up to this point it should be relatively straight forward to understand what’s going on here. However, we haven’t talked at all about _N3MapData , let’s take a look.

struct _N3MapData {
	float fHeight;
	unsigned int bIsTileFull : 1;
	unsigned int Tex1Dir : 5;
	unsigned int Tex2Dir : 5;
	unsigned int Tex1Idx : 10;
	unsigned int Tex2Idx : 10;

	_N3MapData(void) {
		bIsTileFull = true;
		fHeight = FLT_MIN;
		Tex1Idx = 1023;
		Tex1Dir = 0;
		Tex2Idx = 1023;
		Tex2Dir = 0;
	}
};

Each of the four points making up a tile has one of these map data structs. fHeight is the Y coordinate for a point (remember we can get the X and Z coordinate based on how big the map is). bIsTileFull is less straight forward but I believe it gets used when setting which textures to use for a particular tile (here I’m taking “full” to mean “stuff is covering this tile”). I would be interested to hear possible alternative theories about this. The rest of the variables are used for deciding which texture to blend together (Tex1Idx and Tex2Idx) and how they should be oriented (Tex1Dir and Tex2Dir).

Next we read in the patch information.

float** m_ppPatchMiddleY = new float*[m_pat_MapSize];
float** m_ppPatchRadius = new float*[m_pat_MapSize];

for (int x = 0; x<m_pat_MapSize; x++) {
	m_ppPatchMiddleY[x] = new float[m_pat_MapSize];
	m_ppPatchRadius[x] = new float[m_pat_MapSize];

	for (int z = 0; z<m_pat_MapSize; z++) {
		fread(&(m_ppPatchMiddleY[x][z]), sizeof(float), 1, fpMap);
		fread(&(m_ppPatchRadius[x][z]), sizeof(float), 1, fpMap);
	}
}

This simply gives us how high each 8×8 patch is and the radius of its bounding sphere. Next we’ll read in the grass information.

unsigned char* m_pGrassAttr = new unsigned char[m_ti_MapSize*m_ti_MapSize];
fread(m_pGrassAttr, sizeof(unsigned char), m_ti_MapSize*m_ti_MapSize, fp);

char* m_pGrassFileName = new char[260];
fread(m_pGrassFileName, sizeof(char), 260, fp);

This information is used for rendering the grass which pops up out of the tile. Then we read in information about the textures which we will use for each of the tiles on the map.

int m_NumTileTex;
fread(&m_NumTileTex, sizeof(int), 1, fp);

int NumTileTexSrc;
fread(&NumTileTexSrc, sizeof(int), 1, fp);

char** SrcName = new char*[NumTileTexSrc];
for (int i = 0; i<NumTileTexSrc; i++) {
	SrcName[i] = new char[260];
	fread(SrcName[i], 260, 1, fp);
}

short SrcIdx, TileIdx;
for (int i = 0; i<m_NumTileTex; i++) {
	fread(&SrcIdx, sizeof(short), 1, fp);
	fread(&TileIdx, sizeof(short), 1, fp);
}

m_NumTileTex is the number of textures which get used for this particular map. NumTileTexSrc is the number of files from which all these textures will be loaded from. SrcIdx is the index into the source name array for which this texture is located and TileIdx is the index of this texture within the file. Basically the DTEX folder has a bunch of files in it and each of these files contains a bunch of textures put together one by one. In order to get one of these textures you first must know which file to look into and then how far into that file to look in order  to get your texture.

Here are the last few variables we need.

int NumLightMap;
fread(&NumLightMap, sizeof(int), 1, fp);

int m_iRiverCount;
fread(&m_iRiverCount, sizeof(int), 1, fp);

int m_iPondMeshNum;
fread(&m_iPondMeshNum, sizeof(int), 1, fp);

If you have a few rivers or ponds on the map then each files have to be loaded and looked through.

This is the basic structure of the 1299 .gtd file. If you would like all the code along with a simplistic OpenGL implementation of the rendering go here.

Using the N3PMesh converter

Knight Online uses an internal file format called “N3PMesh” to store a lot of the game’s item meshes along with other various game meshes. A lot of these files are located in the “Item” folder found within the game’s install directory. You can think of every object in the game has having its own 3D mesh. In order for a object to appear in game we need to first know all the vertex information in order to draw the object using a 3D rendering library (in KO’s case it would be the DirectX library). A lot of the details aren’t necessarily important for this article but sometimes players would like to modify the 3D meshes in order to change how the objects look in game. I’m going to show you how to do this using the N3PMeshConverter.

Using the converter it is possible to convert .n3pmesh files into .obj files (allowing you to modify the appearance of the game’s weapons) and it can convert .obj files (along with other various formats) into .n3pmesh files. All this allows for you to export objects from the game, modify those objects, and import the modified objects back into the game.

In order to export an object you first must convert the .dxt item texture to a .bmp/.png image file using the N3TexViewer. Next run the following command using the command prompt (Windows Start Menu -> Search Bar -> Type “cmd” -> Press Enter).

N3PMeshConverter -export obj 1_6011_00_0.n3pmesh item_co_bow.bmp

This command will convert the .n3pmesh for the basic bow into a .obj file. Using the .obj and image file you can easily load the KO mesh into a 3D mesh editor like Blender.

In order to import the .obj file back into KO’s .n3pmesh format (presumably after you have made your modifications) use the following command (again via the command prompt).

N3PMeshConverter -import 1_6011_00_0.obj

Simply paste the new .n3pmesh into the KO content folder and the newly modified weapons mesh will appear in game!

Getting setup with the OpenKO project

I recently made a video on how to get setup with the OpenKO source code and another video explaining the basic structure and history of the codebase. Feel free to take the time and get the code running! If you have any questions feel free to send me an email (info@stephenmeier.net).

This second video explains a little about where the source code originates from and what major changes have taken place since I first started working on everything.

Kodev – Updating the UIQuestMenu

I recently made a video explaining how the quest menu GUI works for the game Knight Online. This is the first part in a two part series where we look at why these GUI components work the way they do and how we can use the GUI editor to help us to implement functionality for later versions. Part two for this series has already been recorded I just need to upload it! I hope to do that in the next week or so. If you have any questions or would like to suggest future topics for videos please email me at “info@stephenmeier.net”. Thanks!

EDIT: I was recently able to upload part 2! In the future I will try not to make the videos so long because they quickly become hard to upload.

 

Current Projects

I have been really busy with work recently but I’m making some time to discuss my current hobby projects. One of these project is more of a solo gig and the other, I’m hoping, will start with me doing most of the work at first but eventually I’d like other developers to join in and help.

This first project is similar in style to Arthur’s Grove (AG). AG was a little game I released not too long ago but this time I’m stepping up my game a little more now that I’m more comfortable with a lot of the game mechanics I used when developing AG. The worlds are completely different and the stories have nothing to do with each other (not that AG had much of a story anyways) but the game itself will be similar in style because I find myself gravitating towards this style of game a lot.

The tech is also a bit different. Here I’m relying more on the Tiled Map Editor to set collision boundaries and add entity objects into the level. The Tiled Map Editor exports to JSON which I then parse using Duktape. I have been using Duktape for a while and anytime I need to do a bit of scripting on parse Javascript/JSON it is my go-to library!

The second project has recently grown in importance to me since, apparently, Snoxd has been down for whatever reason (most likely due to the lack of contributing community members). As much as I would like to see Snoxd succeed, my feeling is that the private KO server community needs strong developers and not a bunch of “binary hackers.” A community that doesn’t share will never grow and a community that doesn’t support and praise good contributions will never provide the positive reinforcement necessary to keep people contributing. Sometime in the future I’ll write up a more complete article on my feeling towards the KO development community and a direction for where I would like to see the community head towards in the future. But nevertheless, I have been working on a completely new implementation of the KO AIServer architecture using SDL2 and MySQL server. Currently I’m able to spawn in a group of NPCs with this custom code and have the AIServer communicate with the stock 1298 Game server which allows the player to login and see the NPCs. Most of the work here comes from re-implementing the multi-threaded nature of the AIserver using SDL2, along with all the network calls using SDL2 Net. This code is completely cross-platform and will compile on Windows/Mac/Linux.

One very important reason for using MySQL server instead of Microsoft’s SQL server (which the official USKO, as well as all private servers, use) is because if you were to compile the KO servers on Linux you would will need access to a Microsoft SQL server on another remote windows machine. Therefore, no matter what, a machine running Windows is necessary. However, if the database is a MySQL database this isn’t an issue as MySQL can be installed on any prominent operating system. Hell, even a custom coded server running Sqlite would provide more versatility than SQL server (but I digress).

So that’s the sort of stuff I’ve been working with recently. If you have any questions, or would like to help contribute to the OpenKO project, let me know. The long-term idea is to replace each of the KO game servers for the stock 1298 release one at a time. Once this is complete and the binaries are virtually interchangeable with the originally released 1298 private server binaries we will start writing a custom 1298 client using SDL2 and OpenGL. This is the tentative plan, and likely won’t recent full attention until I have finished my PhD and have find a job. But the progress made so far is still quite an accomplishment relative to the stagnation which unfortunately permeates the KO development scene. This will eventually change though because I know there are passionate developer out there and I would love to help be apart of that change.