Preview release v0.5.2

It's almost as if I have a monthly schedule, new preview release is available for download. Semi-exclusive research is a new feature and there have been a number of improvements and bug fixes.

Download Stareater v0.5.2

Semi-exclusive research had it's own post so I'd focus on under the hood improvements. I won't bore you with coding details this time, let's just say that the game is back to 1000 FPS. As usual whenever the game starts lagging it's my fault, I either forgot to remove debug info collection or performed steps in wrong order. This time it was order of the steps, text rendering was initialized during the scene rendering instead of before it, making invalid OpenGL calls. Unfortunately it didn't crash the program nor bugged out graphics (which it should), driver made an effort to control the damage so I didn't notice the issue right away. After playing a bit with GLIntercept, an OpenGL debugging tool, I've found the culprit and was relieved that the cause was my sloppiness instead of something in programming language + OpenGL + driver combination.

I've also made a bit of code clean up and some refactoring. I've decided to drop T4 text template technology because editor support is not on the level I'd like and it cumbersome to work with it. Once you make it work, it works but when you have to make a change it ends up taking whole week. So far I've reworked two "classes" of T4 dependent code but the final and the most complicated class still remains. I have a prototype of an alternative approach in making but it will take me some time to incorporate it. I'm most probably postpone it and focus on something else.

So in a mean time enjoy the preview and feel free to give feedback!

Semi-exclusive research

A hybrid between Master of Orion II exclusive research system and Space Empires V incremental system is done. When research breakthrough happens a player unlocks new topics and chooses their priorities. Higher priority items have lower development cost making lower priority items harder to acquire but not impossible.

In Master of Orion II research works by accumulating research points until breakthrough happens in a selected field. Each field level unlocks between one and three applications (building, ship component or passive bonus) and normally player can pick only one. Other applications can be acquired from other players through trade or espionage if they happen to research them or by having a creative race trait which gives all applications on breakthrough. Some players hate this exclusivity because if you don't play creative race you will miss out some very useful technologies and creative trait is quite expensive which limits amount of traits you can pick with it. Other players love the system because it gives the reason to use diplomacy and can make the game play a bit differently every time. Anecdotally my best MoO II experiences were when there was an uncreative race in the game. Uncreative trait gives random application (no choice) on breakthrough which makes otherwise predictable AI take interesting paths which in turn may make them actually stronger. It can make AI design better ships by preventing them from getting technologies for making their usual jack of all trades (but master of none).

How to make AI is currently hot topic in 4X community, I should address it at some point too but the topic of this post is research system. I wanted to make a system where player can have a cake and eat it. A system with good parts of exclusive mechanics and ability to eventually go back to missed out choices. One idea was to make some way of reresearching missed out application but that would either make research rewards random or would make reward list increasingly longer. The idea I settled on was to have research give all rewards every time and move the problem to development, where long list of possibilities and varying investment costs would be normal. How exactly would development cost vary with priority is still open question which I intend to iron out when I add more technologies. Speaking of which I have made a spreadsheet with tech "tree" and added to the game small part of if it. I'll add more as game mechanics get implemented and I fill more blanks in the spreadsheet.

And finally, this is how breakthrough screen looks like in game. On the left is the info about research field where breakthrough happened, name, level, description and in the middle are unlocked development topics. At the top is a list of topics which can be rearranged with buttons next to it. Order of topics determines their priority, higher on the list, lower the cost. Below it is currently mouse over or selected development topic description and accept button which finalized priority selection. I'll make new build soon so stay tuned.

Preview release v0.5.1

A new preview release is ready! Ship upgrading and refitting is the main news but there were a number of changes and improvements under the hood.

Download Stareater v0.5.1

Refitting and upgrading ships was explained in previous post and stayed pretty much the same but the game loop was reworked again. Last implementation caused weird performance issues, whenever regular Windows GUI should pop up over OpenGL area there would be a noticeable delay before a window started appearing and it would be appearing piece by piece. The delay and the time needed for a window to get full drawn was not proportional to the complexity of OpenGL scene but oddly by the number of components (buttons, labels, check boxes, ...) on the window. It looks like as if something in Windows is synchronizing main and background thread for each window component. For that reason I've moved OpenGL logic back to the main thread and used background thread loop for sending events when to redraw the scene. So VSync, framerate limiting, power hungry and power conservative frame timing methods are still available.

There was also general graphics performance improvement for non-animated objects. Previous method worked sort of like asking OpenGL to perform A+B and new method works like asking it to perform A and then perform B separately. For some reason former method is slow and I've observed double the framerate with latter method. There is more room for improvement since drawing stars takes 30 - 40 ms (~30 FPS) but only sometimes and the scene is simple enough to be drawn close to 1000 FPS. As I was writing this post I looked again at the issues to confirm the numbers and accidentally found a few bugs including one that caused slowdown. Stuff likes happens that way, there is a problem, you try to pin it down without success, you go to explain to someone what is bothering you and in the process get both correct diagnose and solution without other person ever saying a word :). In any case fixes to those bugs will be temporary band aids until I rework graphics engine to OpenGL 3.x compatible functions.

And there has been some code maintenance changes. Data loading logic was among the first things laid down in the last rewrite and it had some presumptions about future code which are not valid any more. One such thing was progress reporting about "how many percents was loaded" but that information was not used anywhere and implementation cluttered the code needlessly. If the need arises, I'll make cleaner solution. There was also unresolved issue of loading localization data. Previously it was done when settings data was requested for the first time which is not the most reliable approach and there was no good way to add error handling to it. Now the languages and other localization stuff are loaded in controlled manner like all other game assets (technologies, ship equipment, AI modules, map generators). Whole asset loading has been improved too so GUI has more freedom for choosing when to start it and receive notification when certain parts or all assets are ready. There has been a change about how loading logic finds it's data and that is an area which can be improved more. Responsibility for managing files has been moved from core to view so on one hand it would be easier to replace WinForms view with Unity3D but on the other hand view has to know too much about internal working for the core.

There is a lot of work left to do but feel free to try latest build and leave a comment!

Refitting ships

First v0.6 feature is done, ships can now be upgraded or outright repurposed. It also includes automatically designing upgrades when new technologies are developed, as it was before rewrite.

The idea is that colonies passively produce certain amount of repair points which are first applied to repairing ships and then to upgrading/retrofitting. That much have been told in previous post. Currently ship repair is not yet implemented since no ship damage is carried over after a battle so all repair points go straight to retrofitting. Refit prices are currently simply full ship construction cost and repair points are generated at a rate of 1/5 per employed population (colonist with a workplace) making it 25 times slower then production of a new ship. I intend to revisit the numbers at a later date and make more nuanced cost calculation which takes into account equipment difference so upgrading will be cheaper then scraping and rebuilding ships.

So how does it look like? On design list screen each design has red "x" button (which I should change to something more descriptive) which opens "Refit to" dialog where player can choose to remove design, refit to other design or keep the design (cancel removing or refitting).

As with other things in the Stareater, nothing permanent (except exceptions) happens before ending a turn. Marked designs are removed at the beginning of a next turn if there are no ships of that design. If there were technological advancements which unlocked higher level of ship components then upgraded designs will automatically be added and older designs will be marked obsolete and set to refit to new design. That automatic upgrade order can be changed by player but only by selecting non-obsolete design to refit to. Building queues also get changed in the process, swapping obsolete designs ship orders with their upgrades.

I think that's enough for new preview build.

New game loop

I had fun time learn how operating systems do (and don't) handle timing, what's guarantied, what's not and how and how achieve results by messing whole OS. On the one hand I'm disappointed there is no guarantee for having timing resolution below 64 Hz (ticks per second) and that there is no cross platform way of getting the resolution. But on the other hand I've observed 1000 Hz resolution on all machines I have access to.

Let's back off a little bit. Game loop is part of game logic which juggles when input events (mouse, keyboard, ...) are collected and processed and when a image frames are drawn. Typically it repeats following steps: check stop condition, process input, update game state, draw frame, sleep until next frame is needed. In previous Stareater version there was no explicit game loop, it was part of WinForm's internal "message pump" which in turn is too an endless loop that waits for a message, delegates it to an appropriate handler and checks if it got "quit" message. For signaling when to draw a next frame there was a timer component which was periodically sending "timer tick" messages. It was serviceable but not a "proper" solution. Timer's timing and my implementation of it's handler were sort of a "do it when you can".

When I started adding more game loop features, namely adjustable FPS limit, a mode without FPS limit and "battery mode" where game loop is bit more mindful about power consumption, it turned out I have to do a bit of research and rework the stuff. When it comes to timing precision there are two options, busy spin (a loop which repeatedly checks the clock) and thread sleep which is more concerned with letting other threads breath then timing. Timer is basically same qualitatively as thread sleep with a difference in how a thread is notified to wake up. So it's not surprising that most game engines, including the one supplied with OpenTK (graphics library used in the project), use busy spin approach. Busy spin is good when you have a lot of animations and movement in general but one has to keep in mind that such method uses all available CPU time which drains more power, heats CPU and makes cooler fans spin while not doing constructive work during the wait period. Since Stareater may or may not have a lot of animation, I made support for use both timing methods.

There was even bigger change to game loop, it was moved to a separate thread. I looked around the internet what is the best way to implement the game loop and there was no clear best practice. One approach (repeated from basic game dev tutorials) was to have ordinary loop which has extra step in cycle to let message pump process accumulated events. Another approach was vague reference to rendering in background thread. I've decided to marry those two concepts and suffer the consequences. The biggest reason I did that is to separate WinForms stuff from OpenGL stuff. This way window can handle it self at it's own pace and frames can be rendered in parallel. No need to send redraw request through who knows how many layers of bureaucracy. Many programmers fear the multithreading but it turned out to not be that bad. Sure I had my share of deadlocks (threads blocking each other) and race conditions (parallel tasks messing with eachothers data) and had to learn some new stuff but it was worth it. I finally had a chance to use thread join operation and discovered wonders of atomic operations and it didn't take long to fix bugs. In fact I'd like to share a few stories:

• First one was an experiment. In theory when a process quits all of it's threads are killed so I tried to make game loop without explicit end condition. It turned out that exiting an application GUI left background thread working alone. Maybe it was the case when running from Visual Studio or Sharpdevelop, I didn't investigate further because I needed stop condition anyway. Game loop had to have some cleanup logic and it has to stop before GUI components get cleaned up.
• What happens when GUI and main thread hosting it quit before game loop thread? Errors and exceptions because OpenGL is being used after the application had it deinitialized (disposed the context). Cure for that bug was use thread join so that main thread waits for background thread to finish.
• More code you have guarded by synchronization mechanisms means deadlocks are more likely. When a combat happens, main thread informs game loop to switch to space battle "scene". Game loop switches scenes and calls scene initialization inside a guarded block and battle scene initialization "invokes" a piece of code on main thread. This code hides drop-down menus which triggers resize of draw area. Main thread tries to inform game loop of resize event but has to wait because loop is in guarded section so no thread can advance. Fix for this one was to make lighter signalization mechanism. Instead of guarding whole blocks of code I've reduced guards to only cover receiving signals (not handling) and simplified signals themselves (used atomic test and set instead of "monitor"). In retrospect asynchronous invocation would fix an issue too but simplifying code has further reaching benefits.

Enough with technical stuff, here is how settings menu looks like now. You can select or enter frame rate frequency limit or choose unlimited mode which will draw frames as fast as possible. Options below regulate whether to use precise but power consuming timing method or potentially imprecise but much less wasteful method. A little technical detail, power mode detection (whether a computer runs on battery or is plugged in) doesn't work in Mono so Linux and Mac won't have a benefit of game automatically switching modes. Players on those platforms will have to use "always" and "never" options which force the usage a particular method.

This is just a first phase of reworking graphics engine. It went better then expected. I know I'm repeating for the hundredth time but multithreading issues are nasty beast and I'm glad there wasn't so many of them. Next phase is moving to newer OpenGL API and improving performance. I've run profiler and noticed there are very bad performance issues. There is a lot of work left to be done.

Plan for 0.6 features

Loads of tech, that's the plan! And some space monsters.

It's been only a month and I already feel like v0.5 rewrite was done over a year ago. Strictly speaking software is never finished but I drew the line back then. Big rewrite was over and there were no glaring bugs. There was some functionality like ship design upgrade and colony/star list which didn't make through migration but they are either moved to future plan or moved back to drawing board.

This version will be about improving the "engine" and adding more gameplay, both featruewise and contentwise. There will be again moderately big code refactoring but I hope nowhere near as much as in previous version. In any case it should be possible to break it down to smaller (doable within a month) chunks. Code improvements made in v0.5 rewrite made that possible. Without further ado here are features planned for version 0.6:

• Diplomacy with some basic capabilities like "declare war" and "cease fire". Will see for more complex interactions but for now I'd focus on laying foundation, figuring out where to put negotiation logic and how to handle it on user interface.
• Hot seat multiplayer. Multiplayer as a whole is not in short term plan but I made a statement some time ago about how certain change brought the hot seat implementation closer to reality. It's time to test the theory.
• Semi-exclusive research like research system in Master of Orion II where you can get only one item per technology field level. Stareater won't be so harsh, you will be able to get all items but you'll have to spend extra development effort. Idea is to instead of picking only one item you'd make your first, second and third choice. First choice item will be the cheapest to prototype, second choice will be expensive and the third will be even more expensive. On top of reworking research system I plan to start adding actual technologies to the game. I've made an draft document with about 120 candidates and more are to be added.
• Ship repair and upgrade will be reintroduced. Ship damage will persist after battle and colonies will passively produce repair points. Once all ships at a star are repaired, surplus repair points will be used for upgrading or retrofitting ships.
• Missiles in combat. First ambition level is to have "instant hit" missiles which will function similar to beam attacks but with slightly different chance to hit and limited ammunition. Next ambition level are torpedoes, missile which move across the map like space ships, can be destroyed and do area damage.
• Space monsters are going to remind you that you are inside of a living organism. Most of them won't attack you directly but will interfere with growing your empire. On the development side their purpose is to give you something to fight before meeting other players or during the peace time and to provide some lore about the Stareater universe.
• New rendering engine is already in development. Previous engine was using old OpenGL API, rendering loop was not very controllable when it came to timing and I'd like to make it easier to link visible objects with user interaction. Part of it is already done by now since messing with OpenGL and multithreading was more interesting to me then writing a post so that's why there is a delay :).
• OpenGL based GUI is something I was looking forward to for a long time. WinForms are so restrictive, even for making ugly but servicable UI. The game uses a lot of lists, each displaying items differently and WinForms have two kind of GUI controls for lists: a simple ListBox where each item is just text and ListView which is basically carbon copy of file list in Windows Explorer. Both are inadequate for my needs so I've been faking lists with other components and the process is so prone to errors and full of workarounds. On top of that there is no way to easily change the look of controls so if I'm going to basically rewrite their drawing logic I might as well do it in OpenGL and solve other problems.
• Refactoring state data is something I've been constantly doing in the last version. State data are stored in data types which beyond just holding the data don't have any more functionality of their own. The problem is there are a lot of such data types and each needs copying, saving and loading logic so there is a lot of repetitious code which looks like it can be generalized except no programming language feature can do that. Efficiently at least. In the last version I've tried C# text templates which are pieces of code for generating code (or any other kind of text file). They work but are hard to debug and update. I've been prototyping another approach, looks promising so I'll try it out.

Library and ship building from galaxy view

As I mentioned last time two new features got in previous release build, library and star system management from galaxy map.

Library is a place where player can see information about all researchable technologies and equipment in the game. Much like civilopedia in Civilization series where player can look up all sorts of information in game instead of resorting to manual or game wiki. At the top left are categories, below it are topics in a category and the rest is information about the topic. There also an input for item level so player can see how technology benefits or ship equipment attributes are progressing. It's not the prettiest user interface but I'm not inclined to improve it at this point. In version 0.6 I'm planning to switch to OpenGL based GUI framework where I'll have much better control over how interface items look like and behave. When I get the framework working I'll gradually convert old Windows Forms interface to new one and library will be revisited eventually. In the mean time I have to rework how text is fetched from data files. Currently each item has a separate reference code for name and description text and with library there is a need for a third one, long description. So instead of actually adding third reference code on each item in game's core, I'm going to remove one and use it differently depending on context. For instance laser cannon will have only "LASER_CANNON" as text entry code and user interface will look under "LASER_CANNON_NAME" for weapon name, "LASER_CANNON_DESC" for short tooltip description and "LASER_CANNON_PAGE" for library text. It will be smart enough to append appropriate prefix on it's own.

Managing star system (the system as a whole, not individual planets) from galaxy is a small feature which greatly improves quality of life. When testing a game I usually need to build a few ships of certain kind and conclude the game in very few turns. This feature greatly accelerates the process and I hope it's usefulness goes beyond being development tool. Once I sort out new GUI framework I might extend the feature to show more star system information on map before it's selected. Much like cities in Civilization series where you can see what is being built, how long it would take and how well developed are the cities by just looking at the map, without needing to zooming in to particular city. And I'll keep adding quality of life features as I notice the opportunity.

Next in pipeline: version 0.6 plans and new rendering loop.