Wolverine’s Blog

Recently I had some time to work on my several ongoing projects. Unfortunatelly I have failed my exams, but I did it by my own will, so I don’t have any regrets because of it. I guess I’m still not ready to devote all my time towards studying something that’s not my primary interest right now. I like physics, but I hate theoretical basics and learning theory just for learning it. I must have practical applications of things I am learning or I’m getting bored quickly. But because of this fact I had more time to work on my projects. There are two business projects that I’m currently continuing. One of them is my CMS which is getting pretty advanced, because the deadline set by the client is coming fast. It’s been in development for almost a year now. I’m finally getting towards finishing framework. For the last two weeks I’ve been working on marker component of the plugin system. The only two things left in it is layout and article analysis for finding markers and then running function associated with each marker. That will be the end of Plugin Manager system. Then I’ll have to rewrite some static layout functions to dynamic plugins using the plugin system and perhaps write a Composing Manager, but it’s not a priority. With Plugin Manager finished I will be able to fix currently broken permission system. This will complete the functional framework upon which I’ll build a workable administration system plugin. This part would require writing functions associated with saving in the Storage Manager (virtual filesystem). I’m very happy because most of the hard work is already behind me and CMS is nearing it’s completion with basic facilities that will be able to handle critical operations. From then on I’d improve functionality by simply writing new plugins.

In the meantime I was approached by a client to create a simple website for him. He was not willing to pay much, so I’ve decided not to go my usual route to approach my friend who is the graphic artist and webdesigner, and ask him to create a layout for me. I decided to do it myself. In the past I did few sites, but I’m not really an artist so my productions are somewhat unprofessional, but they’re not very bad either. It was also good time to learn Inkscape and fully vectorised layout as opposed to earlier raster-edited ones. I have to admit that for the first play with Inkscape the website is not bad. Creating graphics was one thing, but creating XHTML and CSS code for this layout is NOT such an obvious thing. Many thanks to my friend who helped me a bit. I’m still learning CSS, so I’m not very familiar with this technology. Of course the CSS Zen Garden is quite helpful when it goes to understanding advanced styles. Besides that I have to write a PHP script that will allow to place orders. It’s quite simple and doesn’t involve any kind of shopping solution or paying via internet. This project is a little distracting, but I’m in great need for cash, so every coin counts.

You could think that I had a bit of work lately, but honestly said it wasn’t too much. 🙂 Because I’m not attending to University anymore, I have found the time to spend it doing some other things, playing and coding MOO2 Graphics Viewer. And this is the primary reason for this post. I have abandoned working on the GUI for now and went towards writing some functionality functions. Hang on your chairs ladies and gents. The Graphic class currently allows TO SAVE images! That’s right! You read it correctly. I’m currently able to save images achieving almost 100% compatibility with original graphics from Master Of Orion 2. 😀 The only thing that is left is saving internal palette within the image, which is very simple thing to do and this will be the end of fully functional graphic class. However writing this functionality wasn’t simple reverse process from loading functions. To achieve compatibility with original graphic files I had to do some additional reverse engineering on the format, because of the way frames within the image are saved. It’s linked with format reading specific that is used within Master Of Orion 2. I had to write additional python scripts to do some calculations on the offsets to see what’s going on, because this format reading specific wasn’t at all obvious to me at first. I had almost working saving function yesterday, but due to my stupid error I was stuck for a day searching for the problem. It turned out to be really simple thing that I ommited by accident. But now it works for 100%. Stay on your toes because graphically modding Master Of Orion 2 is really close right now. I don’t have any screenshots for you today, because it’s only saving functionality that doesn’t have any editing capabilities. To have a fully functional editor is not my goal.

I’ve been browsing graphic files and checking them for any anomalies and this is what I have found so far. Some of the files doesn’t use any external palettes stored within fonts.lbx and ifonts.lbx archives. However these files contain mouse cursors, besides the palette, and at least one of the files contains fonts. As for the graphic files. I have indicated earlier the council.lbx archive has internal palette of the first file that is used by most of the graphics from this archive. But that’s not all. I was able to see that most of the files doesn’t really use external palettes. For example. Planets.lbx that has planet graphics on colony screen they all have internal palettes, but not 256 colors! It’s quite obvious those palettes are merged with others. Also some external palettes have purposely left some of the colors out, i.e. they have many occurences of the same color. I’m assuming these are the “places” where merged palette goes in. It’s quite intelligent in fact, because for example ships.lbx has few graphics that are 1x1pixel size and doesn’t represent anything. But they’re interesting from the palette perspective. Those files are simply palette holders that are used to recolor the ships!

And this is one of the main purposes I’m not releasing the code now. I was working on implementing simple XML format to catalogue those dependencies that would be portable to other applications that I hope will begin to popup after releasing MOO2 Graphics Viewer code to the public. And I’ll encourage everybody to use it. The next thing I want to code in the viewer would be the Converter class. It will allow converting BMP or GIF files to native MOO2 graphics format. However the converter class would be quite hard to use alone, without knowing which palette is used for what image and that’s why I’m again delaying publishing the code to the public, before I have fully working and usable package that in fact could be used by non-programmers. Also there’s obvious need for palette export/import into a format readable by graphic programs. I’m not aiming at creating fully functional editor mind you. Main purpose of the program is still viewing Master Of Orion 2 graphics, but I want to include packing/depacking LBX archives (already done), saving/loading graphics (done) and finally importing/exporting images to/from BMP or GIF files (to be done). Before that I’m not really ready to publish it. I’ll try to get it done as fast as possible, but I’m not making any promises.

As I have said the last time it wouldn’t be long before I’d show some eyecandy for those of you waiting for it. But before I’ll give you some screenshots let me tell you something about development. First of all I thought this update would be sooner, but several bad things ™ have occured and I wasn’t able to do it any faster. You may have experienced several server outages trying to reach this page in the last few days. As I have said in the comment for the last post, the server is at my home and I only have wireless radio link. There were quite nasty winds here for the past few days and my microwave antenna isn’t very stable, unfortunatelly, when the wind speeds are reaching over 70 km/h with top speeds over 120 km/h. This contributed to high packet loss over my wireless link. Besides this also caused one day of network outage, because the receiver antenna at the ISP has fallen down. However the winds have passed and so did the problems with my internet link. Hopefully this will cause no more problems.

Another thing was with the development itself. I have written several helper classes in Python including… LBX decompression and COMPRESSION routines! Tada! They’re fully working. I have depacked several MOO2 LBX files and decompression is flawless. After this I have used my compression routines to create almost identical LBX archives as the originals from unpacked files. I have said almost because the difference is with the junk bytes. Normal LBX archives have junk bytes from memory dump within the file position offset space. I’m simply filling this remaining space after legitimate offsets with zeroed bytes until 800h offset where the first file resides. This shouldn’t cause any problems for MOO2 when reading such LBX archive, because those junk bytes are ignored by MOO2.

However after writing those routines and finishing the TkInter based simple viewer I’ve been investigating several GUI toolkits. As I have said previously I have found PythonGUI toolkit, but it turned out to be rather bad for creating graphical tool such as my application. So I have checked out Wix, wxWindows, PyGTK, among many others. The criteria for selection was primarily ease of use, drawing speed on Canvas objects (this is the most important, especially when using Python, because the language execution is so slow) and last but not least portability. My goal for this app was to make it portable from Linux based system to Windows and if possible to MacOS X. PythonGUI seemed to match the portability and ease of use criteria, but it turned out that it even doesn’t have decent drawing facilities. So I have disqualified it as graphical user interface (GUI) toolkit. wxWindows is too crappy and too low-level for me to play with it. I don’t have time and patience for learning such a toolkit. And it still needs GTK in Linux and Unix. PyGTK is just a wrapper over non object oriented toolkit, so the development would need significant shift in my perception and I really hate GTK. So it was disqualified also. After many trials, errors and researching I have finally decided to go with PyQT. QT is clean and fast, fully object-oriented, easy to use, it’s very portable (QT4 is natively portable to Windows also, not sure about MacOS X), you can create derivative widgets with simplicity and the development is fast. But most importantly it has PERFECT facilities for complex Canvas drawing that are FAST even in Python. QT is written in C++ and PyQT is almost identical wrapper over C++ functions, so this is great if in future Python turns out to be too slow, I’ll be able to quickly rewrite the app in C++. And there’s also one more thing. QTDesigner for rapid creation of usable GUI layouts. There’s a compiler that takes QTDesigner UI files and converts them to native Python code. PyQT is awesome. However there’s one downside. Commercial applications written with QT need paid licenses. However as I have said before I’m going down the Open Source road and going to use GPLv2 license, which is also the license for QT and PyQT when used for non-commercial applications. So this isn’t a problem in this case.
And now for the awaited eyecandy. 😉 I have to mention that this wouldn’t be possible at all if not for my russian buddy Grig de Griz, who sent me supplemental information regarding the formats. Once again. Thanks man! And here it is:

Ok. And now a little description. The first image shows the TkInter based viewing window. It shows one of the buildings from Colony Screen in MOO2. As you can see it’s properly shown. The violet color is a special one. This file has a “functional color” flag which means this violet colour is used for drawing shadows over the Colony landscape in the game. As you can see under the window there’s part of the code used for loading and interpreting graphics. This is part of the Graphic class which is responsible for file loading. The second screenshot shows another building from BLDG0.LBX archive. As you may see under the window there’s a shell showing usage of the the viewer. Viewer currently parses arguments from the shell and handles a -p switch which is used for palette loading. Unfortunatelly TkInter toolkit is ugly and doesn’t have essential widgets, so I have decided to switch development to QT. The third screen is showing QT based palettebox widget that I have written for my application. This widget (as other classes in my app) could be used out of the box in other applications as well without need to change anything within the class. It currently handles showing, resizing, on mouse hover color highlighting, selecting colors and few really neat things that could be used in future within the to-be-done editor. 🙂 The widget takes standard hexadecimally encoded RGB color list of any size. This way it’s suitable for ANY application which needs palette selection box. The fourth picture also shows the palettebox widget with other palette from MOO2. This PaletteBox is a widget and this means it could be embed within the window frame or any other QT container. You can also create inherited class from it and make more complex widget using this PaletteBox as a base or even embed it within custom container widget to create for example palette editor widget. Hail to the power of QT! 😉

Currently the app has Palette/Graphic/LBX archive management classes, PaletteBox widget class and now I’m developing GraphicView class which will be essential component in the viewer application. I’m going to leave as much facilities within the class for future inclusion of editor routines as possible. The QT’s QPainter and QCanvas classes are really awesome and powerful toys for such things and they are perfectly fitted for this task. After writing GraphicView widget the only thing what will be left is to use QTDesigner and form compiler to create some kind of pretty user interface and polishing of the code. After this I’m going to release first version of the code into public. Program uses Psyco Just-In-Time compiler optimisation if it’s found on the host system for speeding up the app. Also I’m aware of existence of cxx_freeze and py2exe compilers for creating binary distributions of the application. I’ll try to compile it and test the app after I finish it on the machines available to me. However the time is hard on me, so the code release is a priority for me. Unfortunatelly tommorow I have the exam from Linear Algebra and I have to educate myself a little. 😉 Semestral exams session is just starting and it will last for about two weeks. I will have few days of free time, but I’m assuming I will use them for learning. There’s also another downside. Recently the client contacted me to finish my CMS system and set up website using it for him. He accepted the specified deadline so I will have to work hard on it to match the deadline. This means I won’t have time for MOO2 Viewer within a couple of weeks. However as I have said I’ll try to finish the code and release it’s first public version. The progress is steady and most of the essential program functionality has been written. My guess is that I’ll be able to release the code within a week if time allows. Most of the code I have written through todays night and this update is brought to you with help of few teas, cigarettes and totally awesome Digitally Imported radio chillout&psytrance channel. 😉 Stay tuned, I’ll try to keep you informed whenever changes occur.