This is, well, super important, and follow this link quickly before it’s too late HURRY—
Nearly 11 years ago, I was checking out some of the free and open-source games included with the openSUSE distribution when I came across a fantasy turn-based strategy game called Wesnoth. I vaguely remember taking a quick look at it and dismissing it for some reason. It wasn’t until an OS update later the same year that I would actually give version 0.9.5 a proper try. Immediately upon doing so, I was enthralled by the game’s sheer quality and its surprisingly accessible gameplay.
During the first quarter of 2006, I started toying around with WML and the map editor, and put together three or four semi-playable campaigns rife with awful prose and flat characters just for fun. At the time, the possibility of publishing my content online for others to play it didn’t even cross my mind — and frankly, it’s for the best that it didn’t. Much later that year, though, I decided to join the forums and actively interact with the community. The next year I started to seriously think about giving back to this community, as a way of saying “thank you” to the countless people — developers, artists, musicians, translators, and players — who invested their time and energy into this project.
It would be a big understatement to say that joining the Wesnoth development team became a turning point for me, personally. But that is far from the only experience that changed my perception of open-source software and people in general. Looking back on it, making a campaign comprising 30-odd scenarios, all to give a rejected Elvish Shyde replacement sprite a practical purpose, was an incredibly ambitious idea and I probably wouldn’t have attempted anything like that a year later, and certainly not now. But it’s a thing I did, and a lot of people seemed to like the result despite the generous borrowing of elements from other campaigns — mostly mainline — so I decided to keep maintaining it, and even started to work on a sequel after a while.
That epic-length campaign was originally codenamed “Armageddon”, but for production purposes I went with a longer, albeit less sophisticated name: “Invasion from the Unknown”. I asked around for better name ideas, but nobody answered that call, so the name stuck. Its sequel was codenamed and later properly titled “After the Storm”.
IftU changed a lot during its early days. Scenarios were moved around, renamed, rewritten, and characters introduced or redesigned as I saw fit. In particular, working on AtS throughout 2008 and 2009 and preparing for a prospective addition of IftU to mainline brought in more profound changes to the story line, partly to accommodate some future plot points better. But I also began to view this campaign in a critical light. Various changes in my life helped exacerbate my negative opinion of IftU, and this eventually spilled over to AtS as well, resulting in approximately two years of creative block.
Finally, at some point mid-2011, I snapped out of that mindset and decided to embrace some of IftU’s less favorable qualities and complete AtS and its sequel — which eventually became AtS episode III — but I also decided to improve and rework what I could as soon as I had the chance. After releasing the completed AtS 0.9.0 in Q1 2013, my work on codename “Reconstruction” began. I was finally free to do as I wished with IftU, and make the campaign that I tried to make in 2007, except without the broken grammar, malapropisms, awkward dialogue structure, close-to-nonexistent characterization, and unmaintainable program code. What could possibly go wrong? Well, for starters, everything.
It’s pretty clear to anyone who’s been following Wesnoth up close that things aren’t what they used to be back in 2007. Interests and demographics have changed, and IftU and AtS have been hit hard by these trends. People these days are more interested in complete products than in actively participating in the development process. There is an overabundance of long campaigns and very little time to play them. Wesnoth’s audience overall is also shrinking and engaging less with content creators, which further compounds the problem and isn’t a huge motivation in general. And even though I wasn’t alone on it, maintaining core Wesnoth — especially during my tenure as Release Manager during 2015 — sucked up pretty much all of my time and energy, leaving me barely able to spend perhaps a couple of days a month on Reconstruction. But after coming across a person who actually valued IftU and AtS for what they are, I managed to focus again and eventually start releasing the Reconstruction release candidates throughout late Q4 2015. All that was supposed to culminate in a 2.0.0 release on December 24th, but a lot of unexpected things happened during December so I had to hold that plan off for a while. And now we’re here.
I’d like to think that my 2007 self (and especially my 2006 self) would have played the Reconstruction version of IftU right after finishing UtBS, and thoroughly enjoyed every single minute of it — even if he didn’t really have the experience needed to discern bad prose from acceptable prose, or code, for that matter. Maybe it’s not the best campaign ever, and maybe it fills a niche that no longer exists, but it’s a thing I made and I think that’s important in some intangible fashion. I created, therefore, I am. I don’t know.
Version 2.0.0 is by no means intended to be the final update to IftU, and I have a few ideas brewing for version 2.1.0 already. It probably won’t happen until AtS 0.10.0 is out, however, and that’s going to take me at least another month due to recent incidents beyond my control that have been keeping me from working on it. Yep, it keeps happening.
Version 2.0.0: -------------- * Scenarios: * S21 - Innuendo: * Minor balancing changes. * 22A - Face of the Enemy: * Lift shroud prior to highlighting the exit gate during the boss fight. * Units: * New unit type descriptions * Shaxthal Assault Drone, Minor Imp, Imp, Blood Imp, Gutwrencher Imp, Armageddon Imp, Skeleton Rider (taken from Liberty), Bone Knight.
I’d like to thank vultraz, nemaara, and pydsigner/pyndragon for their invaluable input, assistance, and moral support throughout Reconstruction’s development (especially the second half); Espreon, AI0867, and Alarantalara for maintaining the previous incarnation of IftU on my behalf while I was fully preoccupied with AtS development; Kitty, Loonycyborg, Mica, Mist, and Mythological, for helping with IftU during the Wesnoth 1.3.x and 1.5.x times; and my beloved companion who stuck beside me all these years and helped me through rough times and put up with me until his departure in February this year. This release is dedicated to all of you, and to the Wesnoth community as a whole.
Good luck, have fun, and trust your heart.
Version 1.99.2 (a.k.a. 2.0 Release Candidate 3) is out now!
The complete changelog for this version follows:
Version 1.99.2 (codename Reconstruction RC 3): ---------------------------------------------- * Scenarios: * Fixed boss AI engine breaking when AtS is not installed. * Changed the savegame prefix for both episodes to "IftU", dropping the episode number since it's effectively meaningless in practice. * S7 - Goliath: * Removed all healing glyphs. * S14 - Bye and Behold: * Added a bare-bones attack animation to the Explosive Arrows item (issue #17).
Version 1.99.1 (a.k.a. 2.0 Release Candidate 2) is out now!
This is a bug fix-only release which primarily addresses some of the bugs reported after RC 1 was published, so I don’t have much to say about it.
The complete changelog for this version follows:
Version 1.99.1 (codename Reconstruction RC 2): ---------------------------------------------- * Scenarios: * Removed redundant "(Bonus)" annotation for some objectives in S9, S11, and S16. * S2 - A Real Confrontation: * Fixed pre-captured elven villages not being counted for the scenario defeat condition. * S3 - Memories from the Depths: * Fixed Mal Keshar not speaking one of his lines. * S20 - The Heart: * Fixed alternate victory by defeating all enemy leaders never triggering without using cheats. * S21 - Innuendo: * Increased turn limit. * S23A - Into the Lair: * Added a somewhat more elaborate initial sound transition. * S23B - Until Death: * Fixed an oversight so that player recruits no longer require upkeep. * Terrains: * Ported gate fix from AtS 0.9.1 meant to solve clipping issues for gates adjacent to (convex) stone wall corners. * Units: * Door units now clear terrain beneath them on 'last breath' rather than 'die' events, in order to ensure shroud is correctly cleared before scenario-specific event handlers are run. * Replaced Chaos Crossbowman and Arbalestier's sword attack with an axe to match the sprites.
There is nothing noteworthy about AtS 0.9.16 whatsoever, but IftU 1.99.0 constitutes the first public codename Reconstruction release after several years of work. That is, not counting this year’s April 1st release (which apparently everyone simply assumed to be a hoax instead of actually downloading it from the trunk add-ons server).
I’m cutting this post short since — as should be painfully evident — I don’t really feel motivated to update this blog nowadays. Still, the announcement over at the Wesnoth forums has some more details about this first IftU 1.99.x/2.0 RC series release that are worth checking out if you are planning on installing or updating to this new version.
Version 0.9.14 is out.
Still nothing new at all.
The complete changelog for this version follows:
Version 0.9.14: --------------- * General: * A work-in-progress version was uploaded to the add-ons server as 0.9.13 by mistake. In order to avoid confusion and ensure automated upgrades work correctly, the final release number is 0.9.14. * Graphics: * New or updated terrain graphics: Dark Hive, Dark Hive Surface, Dark Hive Depths. * New attack icon for Elynia's staff. * New or improved unit animations: Elynia (E1/E2 defense, ranged attack). * Language and i18n: * New translations: Japanese. * Scenarios: * E3S7A - Dark Fire: * Fixed the Potion of Life being able to be picked up multiple times. * Terrains: * Added Wall Moss terrain overlay. * Removed custom campfire terrain (phased out in 0.9.12). * Removed custom trash/remains terrains (phased out in 0.9.12). * Units: * Added a new unit type for Mal Hekuba. * Added a new unit type for the E2S11 boss.
Every once in a while I get asked this question, usually in the context of bug analysis. What does Wesnoth use to render text? While at first glance it seems to be a really simple question, the answer turns out to be highly convoluted.
Let’s start from the basics. There are two text rasterization pipelines or APIs in Wesnoth, each relying on a different set of dependencies:
- Legacy/SDL_ttf-based: as the name suggests, this component employs the SDL_ttf library, itself a wrapper around the ubiquitous FreeType library. FreeType is most commonly used in X11 environments such as the most popular Linux distributions, as well as other Unix-type operating systems with a graphical user interface; a notable exception being Apple OS X.
- ttext: I call it
ttextbecause that’s the name of the C++ class encapsulating the pipeline for all code requiring its services. Internally, it uses Pango to select a font and apply styles and markup-based formatting, and renders the result with Cairo. The result is then decoded and copied into a native SDL surface.
This duplication of functionality is first and foremost a historical artifact. The legacy pipeline does pretty much nothing that the newer
ttext pipeline doesn’t already do.
ttext uses Pango markup for formatting arbitrary spans of characters where the older code uses homegrown markup that operates on a line-by-line fashion.
ttext also has purportedly support for right-to-left languages than its predecessor, and recognizes certain Unicode ranges that SDL_ttf does not. It also provides some degree of integration with the operating environment by leveraging Fontconfig’s functionality through Cairo.
ttext doesn’t have that the legacy API does is caching of rendering results. The legacy API caches up to 50 text surfaces unless running the (default) MP lobby, which raises the limit to 1000.
Now, here is where things get frustratingly complicated.
GUI1 and GUI2
Two different user interface APIs co-exist in Wesnoth due to neither one implementing a superset of the other’s functionality.
GUI1 is the older API that was already in use in Wesnoth 1.4.x back in 2008. It implements static labels, push buttons, toggle buttons (normally found in check box form), menus (used to implement list boxes, combo boxes, and pop-up menus), progress bars, scroll bars, sliders, and single and multi-line text boxes. It also includes a very basic framework for setting up generic dialogs with a limited set of widgets. Most notably, all layout is done using absolute positioning on the screen, and there is no built-in ability to ensure widgets don’t overlap each other or overflow their dialog. List boxes lack horizontal scrolling, tab pages and other form of pagination do not exist, and both widget and dialog layouts are hard-coded into the game’s C++ code with magic numbers all over the place.
Development on the GUI2 framework commenced in 2008, with the intention of replacing GUI1 entirely once completed. One of the first tangible fruits of the endeavor was the new WML
[message] display introduced in Wesnoth 1.6, which was in turn designed for the specific purpose of enabling the use of larger, transparent portraits for speaking units. GUI2 revolves around using a dynamic grid-based layout method for both dialogs and widgets, leaving almost all of the presentational details to WML, and even defining the entirety of a dialog’s layout in this language. One of the original design goals is support for user-defined themes; in theory, this would allow campaigns and other user-made content to completely reshape Wesnoth’s UI to suit the content’s flavor or address shortcomings in the stock theme.
In practice, however, GUI2 is an incomplete experiment that just happened to become the de facto foundation for most user interface components introduced in Wesnoth 1.6.x and later; this is because most of the aforementioned features are just too convenient. However:
- The dynamic grid-based layout is both a curse and a blessing, as there are barely any officially-sanctioned mechanisms to force a specific layout independent of textual contents.
- Furthermore, the ‘dynamic’ part only refers to automatically rearranging the grid when a widget’s contents or state changes. While it is possible to remove or add widgets at runtime, it essentially requires violating the framework’s established protocol and messing with implementation details. This happens to be why GUI2 does not implement any kind of grid swapping mechanism that would enable us to have tabbed dialogs and such.
- Certain functionality from GUI1 has not been ported yet, in some cases due to flaws in GUI2’s current implementation preventing a ‘clean’ way to add such things, and in others because it was considered low priority to do so. In particular, combo boxes, pop-up menus, and multi-line text boxes are still missing.
- GUI2 theme support does not ‘officially’ exist yet, even though most of the underpinnings are there.
If we ignore for a moment the matter of which parts of the game UI are written using GUI1 or GUI2, the answer to the opening question seems relatively simple, if needlessly bifurcated. GUI2 uses
ttext for rendering text — in fact, it was specifically introduced for use by GUI2 before it crept into other components (more on that later). GUI1 uses the SDL_ttf pipeline instead, and only because nobody bothered to switch it to
ttext — which, although doable, is not a trivial task.
Since Wesnoth 1.10.x, GUI2 is used to implement pretty much all non-fullscreen dialogs, as well as the title screen with the main menu. The most notable exceptions include the Load Game dialog, the Add-ons Manager, the Preferences and Hotkey Preferences dialogs; and the in-game Help browser, Advance Unit, Attack Unit, Status Table, and Unit List. Finally, while the (default) multiplayer lobby screen does not use the stock GUI1 dialog code, it makes extensive use of GUI1 widgets.
So what’s the catch, then?
The two chimeras
The in-game user interface — displaying unit stats, the minimap, and the menu bar — is the heart and soul of Wesnoth’s UI. It is themeable, using an obtuse layout format in WML. It is used by the built-in map editor, whose UI is essentially a special theme with some hard-coded behavior. It also combines stock GUI1 widgets — like the menu bar buttons and End Turn button — with custom user interface elements describing the highlighted unit, as well as providing interaction with the game map. Strictly speaking, it belongs neither to the GUI1 nor GUI2 APIs, but rather sits on top of the former and beside the latter.
And here’s where the elegant dichotomy from earlier falls apart.
Everything in this screenshot, with the exception of the Menu, Actions, and End Turn buttons, is rendered using
ttext. Who would’ve thought, right?
The themeable game UI (or “Theme UI” for short) is not alone. Back in Wesnoth 1.7.x, I took it upon myself to write a new implementation for our campaign story screens enabling the use of Pango markup and a couple of additional options for text positioning. The result is that
ttext is used both for the text heading (most often seen on the last story screen before starting a scenario) and the story text proper at the bottom. The advance/skip buttons are stock GUI1 widgets, however, which means they use the legacy font API.
Finally, sitting alone in its corner, we have the loading screen with the Wesnoth logo on a black background and the progress bar below. It’s not a GUI1 dialog, it’s not a GUI2 dialog — indeed, it’s not a dialog at all. Because Wesnoth is a single-threaded application for most intents and purposes, it wouldn’t do much good to have the loading screen run its own event loop like other parts of the GUI.
Instead, the loading screen exposes its own API to other components like the WML preprocessor and parser, and leaves it to them to perform a status update every time something happens. The visuals — including the fancy progress bar — are all done by hand in the loading screen’s implementation, meaning that no GUI1 or GUI2 widgets are at play here. Thus, the loading screen calls the legacy font rendering code directly.
Since Wesnoth initializes both font rendering APIs early on before the loading screen first comes up, there is really no reason for us to prefer one API over the other for the loading screen. Much like GUI1’s, the loading screen’s choice is merely a historical artifact.
Mixing calls to both APIs in the same visual context often leads to jarring results.
ttext is subject to Cairo’s use of the Fontconfig library to obtain font rendering settings from the system, in addition to calling a system-specific rasterizer such as ClearType on Windows. The legacy SDL_ttf-based API calls FreeType on all systems with its own hard-coded settings instead. While this means
ttext can benefit from platform features where SDL_ttf can’t, it also exposes bugs in
ttext, as well as bugs in Pango/Cairo/Fontconfig themselves:
- Bug #21648: ClearType’s subpixel hinting isn’t handled well by
- Bug #20337: Essentially, subpixel hinting isn’t handled well by
ttext, platform-independent edition.
- Bug #23560: Pango/Cairo/Fontconfig refuse to use our fonts on Apple OS X.
Thus, a single screen can contain multiple text elements in the same font and size, and maybe even with identical contents, but looking either slightly or completely different due to
ttext’s bugs resulting in different output compared to SDL_ttf.
Where do we go from here, though? Right now, it’s highly unlikely that GUI2’s development will progress any further in Wesnoth 1.13.x, but this does not mean that the old API used by GUI1 can’t be removed once and for all in favor of
ttext. Whether this can be done basically depends on two things:
- Our ability to fix the aforementioned
ttext-specific bugs, or at least work around them somehow.
- Determining whether the legacy API’s only unique feature — the render cache — is actually useful, and porting it to work transparently with
ttextinstead. It should be noted that GUI2 does suffer from inefficiencies with large amounts of text, but due to the additional layers of complexity it adds on top of font rendering, it’s hard to tell how much it would benefit from this feature.
Of course, #2 above basically rests on me at the moment.
Version 0.9.10 is out.
Due to circumstances, it has been quite a while, and honestly I lost track of what this release was supposed to have besides a thing that requires another thing from another campaign that has not been completed or released yet. So let’s talk about what AtS 0.9.10 actually has.
Firstly, the minimum Wesnoth version requirement now is 1.11.11.
That’s right. Previous versions (including 1.10.x) are no longer supported. Ever since I moved to 1.11.x following the release of AtS 0.9.0, maintaining support for previous versions (including buggy development releases) required a series of unwieldy kludges that made the code uglier and harder to maintain and were, for the most part, untested beyond the classic “does it compile?” test. With all those bits gone, it will be easier to improve and optimize some aspects of the campaign, as well as work on the thing that requires the other thing I alluded to above.
For now, the first one such aspect I have worked on is converting several units to the 1.11.x animation WML syntax. Although some other people seem to prefer the new syntax over everything, I have chosen a more pragmatic approach for this campaign, so the set of units that I’ve converted in this release is rather limited. Hopefully more will follow soon, but I’m certain that there are a few for which the change hurts code readability. Plus, since most of my units are headbutters, the code size gains are marginal in the average case.
Finally, somebody reported to me of an issue with the player’s recall list and gold being discarded during certain key scenario transitions. It turns out this resulted from a change in Wesnoth 1.11.13 purportedly intended as a bug fix for MP campaigns. I was aware of the change and its implications at the time 1.11.13 was released, but I wrongly assumed AtS would not be impacted because I failed to take a tiny detail into account. Exactly three months later, I realized the sheer gravity of my mistake — but fortunately, it seems nobody else played AtS on 1.11.13+ in the meantime. (Thanks to RainerT for the report. This would have gone unnoticed for who knows how many more
years months otherwise.)
As you can see, there is not a lot to talk about in this release other than the version requirement change. Since it’ll be a while before Wesnoth 1.12.0 is released, and AtS remains largely the same as it was the last time I posted in this topic, I believe stable version purists won’t be missing out on anything for now — at least not until the thing is done.
Also due to circumstances, this release is largely untested, so I would not be surprised if I accidentally broke a thing or two since 0.9.9.
The complete changelog for this version follows:
Version 0.9.10: --------------- * General: * Raised minimum game version requirement to 1.11.11. All existing compatibility code for previous versions has been removed. * Graphics: * New or updated unit graphics: Sprite, Fire Faerie, Forest Spirit, Dryad, Demon Shapeshifter. * Scenarios: * Added an option to certain scenarios to ensure Wesnoth does not discard the player's gold and recall list under certain circumstances due to a behavior change in version 1.11.13 and later. Affected scenarios: * E1S9.3 - The Triad, part 3 * E1S11 - Return to Wesmere, part 2 * E2S0 - Transience * E2S11 - A Final Confrontation * E3S0 - Opening (Within) * E3S6 - Divergence * E3S8B - Destiny, part 1 * E3S11 - After the Storm * Units: * Balancing changes: * Changed Leech's alignment from 'lawful' to 'neutral'. * Decreased Leech's HP from 62 to 42. * Decreased Leech's melee damage from 11-2 to 9-2. * Decreased Leech's unit level from 3 to 1. * Converted to the simplified 1.12 animation syntax: * Dusk Faerie, Night Nymph, Nightshade Fire * Sylvan Warden * Sprite, Fire Faerie, Dryad, Forest Spirit * Elvish Wayfarer * Faerie Avatar * Demoness Hellbent Tide * Verlissh Control Spire * User interface: * Cutscene themes now use the 1.11.10 [theme] id attribute on 1.11.10 and later.
WML is the primary source code language for all add-on and mainline content for Wesnoth. Creators like me write large amounts of WML to instill life into their user-made projects. Our wiki has a rich reference section for the various dialects of WML understood by the game in different contexts. Tools like wmllint exist to make sanity-checking and porting tasks easier for
the few people who can tolerate its obtuseness us. There is a whole forum section in Wesnoth.org dedicated to assisting people with their WML endeavors.
With so many resources available, you’d think WML is the most fun thing to work with, right?
1. The problem
One of the most annoying aspects of add-on development is that whenever something goes horribly wrong with the WML document’s well-formedness, you may get a barrage of file paths thrown at you in a superdense text-wall format, particularly so if the error site has been subject to file or macro substitutions, which is the case with pretty much every add-on that has more files than its top-level
_main.cfg. Trying to find the relevant portions of the substitution trace to debug the issue can be frustrating at best.
When more than one add-on fails to load, though, the results can be even messier.
The report is displayed on the screen by calling a generic GUI function that creates a new instance of a generic message dialog with the specified contents, which are also printed in stderr. Since the only formatting option available for the dialog is to rearrange the text with a combination of whitespace and newlines, the same thing winds up in
stderr in the same format:
20140216 10:50:12 error config: error reading usermade add-on '/home/shadowm/.wesnoth-1.11/data/add-ons/After_the_Storm/_main.cfg' 20140216 10:50:12 error general: The following add-on had errors and could not be loaded: /home/shadowm/.wesnoth-1.11/data/add-ons/After_the_Storm/_main.cfg ERROR DETAILS: Found invalid closing tag [/unit] for tag [side] (opened at ~add-ons/After_the_Storm/episode3/scenarios/13_Epilogue.cfg:98 included from ~add-ons/After_the_Storm/base-loader.cfg:20 included from ~add-ons/After_the_Storm/_main.cfg:174 included from ~add-ons/After_the_Storm/base-loader.cfg:108 included from ~add-ons/After_the_Storm/_main.cfg:174 included from ~add-ons/After_the_Storm/episode3/scenarios/13_Epilogue.cfg:101 included from ~add-ons/After_the_Storm/base-loader.cfg:20 included from ~add-ons/After_the_Storm/_main.cfg:174 included from ~add-ons/After_the_Storm/base-loader.cfg:108 included from ~add-ons/After_the_Storm/_main.cfg:174), value ']' at ~add-ons/After_the_Storm/episode3/scenarios/13_Epilogue.cfg:94 included from ~add-ons/After_the_Storm/base-loader.cfg:20 included from ~add-ons/After_the_Storm/_main.cfg:174 included from ~add-ons/After_the_Storm/base-loader.cfg:108 included from ~add-ons/After_the_Storm/_main.cfg:174 included from ~add-ons/After_the_Storm/episode3/scenarios/13_Epilogue.cfg:112 included from ~add-ons/After_the_Storm/base-loader.cfg:20 included from ~add-ons/After_the_Storm/_main.cfg:174 included from ~add-ons/After_the_Storm/base-loader.cfg:108 included from ~add-ons/After_the_Storm/_main.cfg:174
This issue affects mainline and UMC developers alike, and yet, it has gone unsolved for a whole decade of development, even while other usability issues have been and continue to be addressed. Perhaps the very fact that the problem has existed for so long has silently perpetuated a misguided notion of impossibility — because, surely, if fixing the relevant code was that easy, somebody would have already done it? Or maybe, just maybe, everyone is afraid of altering code that lies deep within the core of Wesnoth’s WML processing pipeline; because it is an indisputable truth that the WML preprocessor and parser aren’t the most accessible modules for inexperienced contributors, and changing functionality that constitutes the circulatory system of the game engine is obviously a risky thing to do.
But wait! Wouldn’t it be possible to just sort of... make the error report presentation nicer without meddling too much with the implementation details?
2. The solution
While the basic proposal sounds simple and the results of my two-day-long coding effort made it into Wesnoth 1.11.10 (a.k.a. 1.12 beta 1), it turns out that there are many more factors to take into account, and in the end I did have to mess with parser and preprocessor implementation details to a greater degree than I feel comfortable with.
Against what one would expect, there is only one exception object class in existence for parser errors, and it is extremely generic. Callers have no way to tell what just happened when WML parsing has failed, other than asking the exception object for an error message... which is both stored and delivered in plain string form. The error type, the error site, and the WML preprocessor substitutions that led to it; these are all provided to the caller in an atomic string that may contain portions translated to the user’s language beforehand. This means that we (the parser’s callers) can’t reliably scan the error message to choose the format in which we are going to present the error to the user. All we can do is grab that big-ass message string and throw it into a dialog.
The dialog used to display the report in versions up to and including 1.11.9 is part of the GUI2 framework, but the particular dialog class in use is a plain generic message box. Naturally, it is not optimized for presenting WML error messages to the user — or displaying overlong strings in general, really. It is for this reason that the first part of this little project of mine consisted of designing a new dialog class (
gui2::twml_error) for presenting this information to the player in a more useful fashion.
3. The dialog
Because the Wesnoth config manager will try to load all add-ons at once, the first thing that the dialog needs to do is to display a list of add-ons that caused a parser or preprocessor error when attempting to load them. The 1.11.9 approach is to hastily concatenate a newline-delimited list of paths to the files Wesnoth attempted to load, which are nearly always the
_main.cfg files. This can be quite misleading in most cases, and newbies might try to look for the faulty code in the listed
_main.cfg since it is the very first thing displayed in the error message.
gui2::twml_error instead displays the user-friendly name (1) of the add-on that was stored in a special file (
_info.cfg) when it was first downloaded from the server — failing that, it makes its best guess by replacing underscores in the directory name with blanks. If multiple add-ons failed to load, they are shown in a nice bullet list.
Where there used to be more plain dialog text, there is now a pretty scrollable box with the report contents (2). The report itself contains the list of errors found when loading the affected add-ons, but it is no longer all thrown at your face at once — the error log for each add-on is followed by an additional empty line, so telling them apart should be far easier than before. Finally, right above the report there is a button (3) to copy it to clipboard so players don’t need to take a screenshot of the whole thing to give to the add-on maintainer(s).
To be perfectly honest, the only reason that copy-to-clipboard button even exists is that I find error message screenshots irritatingly cumbersome to work with at both ends. It doesn’t help that I’m usually on slow Internet connections where downloading a screenshot of a Wesnoth instance surrounded by some fancy window decoration and taskbar and (occasionally) desktop wallpaper can be excruciatingly painful. Furthermore, as a forum administrator who has to keep an eye on how people use attachments, this preposterous waste of disk space can be quite annoying.
But replacing the error report dialog and the presentation of reports as a whole is not enough to improve the WML coder’s experience.
gui2::twml_error would still display the text wall-like errors if it were asked to do so by the config manager. Even though I hinted at the essentials above, the muddy implementation details screened by
gui2::twml_error would still fill up a whole blog post on their own. Thus, I am going to leave that story for another time.
Version 0.9.9 is out, just in time for the holidays!
Since I have tested and developed this campaign primarily on Wesnoth 1.11.x since the 0.9.0 release, this version promotes support for Wesnoth 1.11.8 to official status. Because most of the campaign makes use of 1.11.x-specific features (both from me and other mainline developers) when available, it is quite possible that I will entirely drop support for Wesnoth 1.10.x in a future release even before the first Wesnoth 1.12 beta arrives. I still intend to make sure certain additions and changes for episodes II and III land before the last AtS version supporting 1.10.x, if time permits.
This 0.9.9 release primarily deals with prose corrections and improvements, and various other ‘cosmetic’ changes. There are also various fixes for some instances of dysfunctional AI recruitment on Wesnoth 1.11.7 and 1.11.8 resulting from the
recruitment_save_gold aspect being enabled by default in those versions (but not 1.11.9 and later).
There isn’t much in terms of new graphics since the prose and code changes (plus some mainline stuff) have kept me far too busy to do much more than some doodles. On the other hand, this release contains various animation fixes and improvements to the Aragwaithi units ported from Era of Chaos. There are also a handful of balancing changes affecting both Aragwaith and non-Aragwaith units.
Also featuring in this release are a number of WML optimizations intended to reduce campaign load times — especially on 1.10.x, which has a slightly slower tokenizer implementation than 1.11.x. Since the affected bits of code have been completely rewritten in Lua, it is possible that I accidentally introduced new bugs in the process that I may have missed during my playthrough, extensive as it was.
There is also a new secret feature that is not mentioned in the changelog. What is it, you ask? Well, if I told you, that would ruin the surprise. Think of it as a Christmas present!
Finally, from this release onwards, After the Storm is no longer part of the Wesnoth-UMC-Dev Project, and will be hosted on GitHub instead. Ever since development of the campaign started, Wesnoth-UMC-Dev provided SVN repository hosting for both After the Storm and Invasion from the Unknown, but over time that has proved to be an inefficient solution due to technical and organizational concerns. Although the conversion process was not easy in the least, I believe that this move will make things easier for me in the long-term, since I had been using git-svn to work on IftU and AtS since late 2008 anyway.
Release tarballs will continue to be hosted by Wesnoth-UMC-Dev for the time being, until I decide to phase them out entirely in favor of GitHub’s Releases page. If anyone is using them because they cannot normally download AtS or AtS_Music from the wesnoth.org add-ons server, I’d appreciate it if you let me know so I can make a more informed decision in the future (or point you to add-ons.wesnoth.org, that works too). For the time being, the tarball compression format has changed from Bzip2 (
.tar.bz2) to xz (
Special thanks to vultraz and 8680 for their proofreading assistance, without which this release would be about 1% less awesome. Also thanks to the current and past Wesnoth-UMC-Dev admins, AI/AI0867 and Espreon, for their continued support all these years — IftU and AtS simply wouldn’t be the same without Wesnoth-UMC-Dev.
The complete changelog for this version follows:
Version 0.9.9: -------------- * General: * Removed Wesnoth development versions warning from the campaign menu entries as support for 1.11.8 and later is now mature. * New complete algorithm for calculating the relative direction between two hex grid locations, handling all six intrinsic facing directions instead of only SW and SE. * Updated Aragwaith faction from Era of Chaos 1.3.1+dev up to commit 9dedeba7cddc2a027745c9994a917fdcb78ed341. * Stripped optional whitespace from terrain map and mask files, decreasing uncompressed directory size by about 62%. * Graphics: * New or updated unit graphics: Blood Core, multiple Aragwaith units, Demon Grunt. * Assigned a more dignified generic portrait to Cron (1.11.x only). * Music and sound effects: * Mitigated [fade_out_music] causing a portion of the previous track to be heard at full volume at the end of the fade-out sequence. It still won't help in all cases. * Scenarios: * Fixed additional bugs with hero ellipses on Wesnoth 1.11.6 and later affecting Anya on every scenario and Durvan on scenario E3S7B and later. * Use STARTING_VILLAGES_ALL instead of STARTING_VILLAGES with large numbers to assign all villages to sides. * Skip inclusion of death events for characters that are not present during the first few scenarios of E1. * E1S3 - Civil War in the North: * Fixed the first defined on-map unit (usually Galas) becoming permanently invisible. * Minor prose tweaks. * E1S4 - Terror at Dusk: * Balancing changes to make the scenario easier on Wesnoth 1.11.7 and later, possibly connected to the new AI recruitment gold saving aspect introduced in Wesnoth 1.11.7 and enabled by default. * Minor prose tweaks. * E1S5 - Bay of Tirigaz: * Made it so Mal Keshar speaks for bat units when investigating shipwrecks. * Minor map tweaks. * Minor prose tweaks. * E1S6.1 - Quenoth Isle: * Fixed Elynia's ellipse reverting to a generic unit ellipse on 1.11.x during the faerie fire cutscene. * Not-so-minor prose tweaks. * E1S6.2 - Elves of a Different Land: * Not-so-minor prose tweaks. * Extended map for large screens. * E1S7 - The Search for the Past: * Improved ending cutscene transition. * Minor AI adjustments to make the undead minions recruit correctly on Wesnoth 1.11.7 and later. * Not-so-minor prose tweaks. * E1S7x - Resolutions: * Minor map tweaks. * Not-so-minor prose tweaks. * E1S8 - Fear: * Not-so-minor prose tweaks. * E1S9.1, E1S9.2, E1S9.3 - The Triad: * Not-so-minor prose tweaks. * Various cutscene improvements and changes. * E1S10 - Tears: * Minor prose tweaks. * E1S11.1 - Return to Wesmere, part 1: * Not-so-minor prose tweaks. * E1S11.2 - Return to Wesmere, part 2: * Fixed story text not appearing because of a missing macro inclusion (long-standing bug that's existed ever since the scenario was first released). * Minor map tweaks. * Minor prose tweaks. * E1S12 - The Queen: * Balancing changes. * Excluded time area for the E1S11.2 starting area on Wesnoth 1.11.7 and earlier (including 1.10.x) due to a bug with time area ids not being saved, resulting in a time area with local lighting that interfers with a cutscene sequence after reloading from a non-start-of-scenario save. * Fixed long-standing offset-by-one bug with a terrain mask applied near the end. * Minor map tweaks. * Not-so-minor prose tweaks. * Various cutscene improvements and changes. * E1S13 - Death and Rebirth: * Minor cutscene improvements and changes. * E2S1 - By the Moonlight: * Minor AI adjustments for Wesnoth 1.11.7 and later. * Not-so-minor prose tweaks. * Now the scenario lives up to its name. * E2S2 - The Heart Forest: * Fixed fog not being cleared correctly when Allyna first appears. * Made it so Allyna introduces herself once three of the five bandits have been killed rather than all of them. * Minor AI adjustments for Wesnoth 1.11.7 and later. * Minor prose tweaks. * E2S3.1 - Unrest in Raelthyn: * Minor prose tweaks. * E2S3.2 - Revelations: * Minor prose tweaks. * E2S4 - Shifting Allegiances: * Minor prose tweaks. * E2S5 - The Eastern Front: * Minor prose tweaks. * E2S6 - The Voyage Home: * Not-so-minor prose tweaks. * E2S7 - The Voyage Home: * Minor prose tweaks. * E2S8 - And then there was Chaos: * Minor AI adjustments for Wesnoth 1.11.7 and later. * Not-so-minor prose tweaks. * E2S9 - New Hive: * Minor prose tweaks. * E2S10 - The Betrayal: * Minor prose tweaks. * E2S11 - A Final Confrontation: * Maybe-minor prose tweaks. * Minor cutscene tweaks and improvements. * E2S12 - Fate: * Minor cutscene tweaks and improvements. * E3S0 - Opening (Within): * Minor cutscene tweaks and improvements. * E3S1 - Beyond her Smile (A Light in the Darkness): * Minor map tweaks. * Various cutscene improvements and changes. * E3S2.1 - Return to Raelthyn: * Minor map tweaks. * Increased initial gold supply for the second human player side. * E3S2.2 - Reckoning: * Minor prose tweaks. * E3S3 - Amidst the Ruins of Glamdrol: * Minor prose tweaks. * E3S4.1 - Outpost of Hell: * Minor prose tweaks. * E3S4.2 - Gateway: * Minor prose tweaks. * E3S5 - Pass of Sorrows: * Minor map tweaks. * Minor prose tweaks. * Minor ending cutscene improvements. * E3S6 - Divergence: * Minor prose tweaks. * E3S7A - Dark Fire: * Prevent crashing Wesnoth 1.11.8 due to a missing initial time of day (part 1 only). * Minor AI adjustments for Wesnoth 1.11.7 and later. * Minor map tweaks. * Minor prose tweaks. * E3S7B - Dark Sea: * Minor AI adjustments for Wesnoth 1.11.7 and later. * Minor prose tweaks. * E3S8A - Interim: * Prevent crashing Wesnoth 1.11.8 due to a missing initial time of day. * Minor cutscene improvements nobody could possibly notice. * Minor prose tweaks. * E3S8B - Destiny, part 1: * Minor aesthetic changes nobody could possibly notice. * Fixed parts of the map being unintentionally uncovered upon entering Hemérilyel's chamber. * Made Hemérilyel more aggressive towards the player on Wesnoth 1.11.2 and later. * E3S8C - Breakdown: * Minor AI adjustments for Wesnoth 1.11.7 and later. * Minor prose tweaks. * E3S8D - Destiny, part 2: * Minor prose tweaks. * E3S9 - Dark Depths: * Minor prose tweaks. * E3S10 - Blood: * Minor prose tweaks. * E3S11 - After the Storm: * Minor prose tweaks. * E3S12 - Destiny, part 3: * Minor prose tweaks. * E3S13 - Epilogue: * Minor prose tweaks. * Units: * Balancing changes: * Imps are now immune to the plague weapon special. * The Protection ability affects own units of any lower level again instead of only level 0 and 1. * Affected units: Demoness Hellbent Tide, Aragwaith Shield Guard, Aragwaith Ancient Banner. * Physical endurance no longer resets statuses (poisoned, slowed, etc.). * Decreased Lumeril Glyph Mistress' arcane damage resistance from -10% to -20%. * Decreased Fallen Faerie's cold ranged attack strength from 11-3 to 10-3. * New or improved unit animations: Verlissh Matrix Core, Shaxthal Custodian Drone, Shaxthal Queen, Verlissh Matrix Flow System, Verlissh Control Spire, multiple Aragwaith units, Dusk Faerie line. * Made it so the Falcon unit type and the lightfly movetype are only defined if the mainline Khalifate faction is not present, by testing the existence of core/units/khalifate/Falcon.cfg. * Fixed a minor inaccuracy at the beginning of the Terror ability description. * The spawn controller code (used e.g. in Shaxthal hives) has been completely rewritten in Lua. No behavior changes expected.
Merry Christmas/Happy Holidays, everyone!
After my successful extraction of the kate-wml-syntax portion of Wesnoth-UMC-Dev, the next goal obviously had to be just a little more complicated than that. Between Invasion from the Unknown and After the Storm, AtS has a relatively simple VCS and meta-history spanning only one main branch and one short-lived branch, as well as a number of tags based only on the main branch.
In reality, a tool called reposurgeon ought to be the most optimal for performing this task correctly and elegantly, but I have my reasons to avoid it. Nevertheless, if you need to convert your own SVN repositories to Git, it is probably what you should be checking out instead of these blog posts. I’m here to do things my own way and learn while at it; the
git svn portion of this procedure is very boring and nothing out of the ordinary, but
git filter-branch is a handy power tool that comes bundled with Git and which may prove useful to me for non-conversion tasks as well, and in fact, it already has done so. It’s worth noting that being a power tool, you do not want to put a clueless simpleton in charge of applying it on your repository; then again, if its documentation is anything to go by, reposurgeon is not much better in that regard. All this is probably for the best.
(I’m also taking this opportunity to express my absolute befuddlement at Git’s liberal
git push -f implementation, presumably a consequence of Git being originally designed for the Linux kernel’s mail-based workflow. While
git push -f can be useful under very specific circumstances, I have heard and seen people use it as a Git panacea without realizing the consequences.)
I already described the
git svn step in broad terms in Part II of this series, so I am not going to go into details here. The fun part continues to be the
git filter-branch (repository rewrite) operation, although this time we also need to deal with SVN tag branches.
As I had the perform the After the Storm repository rewrite multiple times on a remote server before I realized my desktop box’s superior disk I/O and tmpfs configuration and lack of console lag resulting from a slow SSH link would be a considerable advantage, I wrote a few scripts to help me throughout the whole procedure. I made several highly risky and specific concessions aimed solely at dealing with my particular use case, though, so I can’t possibly publish those scripts without either feeling ashamed of my subpar shell scripting skills or inadvertently allowing one of the aforementioned specimens to wreak havoc on somebody’s data, perhaps even their own! Either outcome would be regrettable.
Thus, I am providing a pseudocode outline of the rewrite procedure instead:
- For every tag branch BT:
- Let its tip commit be HT.
- Create a corresponding annotated tag T pointing to HT, with T’s authorship information matching HT’s (author and committer identification and timestamp).
Due to how SVN tagging and branching works from
git svn’s point of view, HT is an empty commit object that incurs in no tree changes. This is an important thing to keep in mind for the next step.
- Because the original HT commit message is highly SVN-specific and not particularly useful (
Tagging add-on 'After the Storm', release <VERSION> from trunk, using r<REV>), I opted for an automatic minimalistic tag message for T of the form
After the Storm version <VERSION>, reminiscent of the
AtS: version <VERSION>commits preceding the tag branching point.
- Because the original HT commit message is highly SVN-specific and not particularly useful (
- Delete BT. HT is now only referenced in the repository by T.
- For every commit C:
- Rewrite the
git-svn-idmetadata trail in the commit message so it reads e.g.
[Wesnoth-UMC-Dev SVN r12345]and strip extra empty lines between paragraphs.
- Delete the
/musicdirectory recursively if it appears in the commit’s tree.
- Rewrite any tag objects pointing to C while keeping any other attributes intact.
git filter-branchdoes not do this by default for some reason, so you may end up with tags pointing to dangling objects otherwise.
- If C is empty (no tree changes) after the previous steps, delete it.
From point 1, this results in every HT commit being erased, and T being rewritten to point to HT’s immediate ancestor, which is in an ideal SVN repository part of the origin branch and not the short tag branch.
- Rewrite the
git fsckto check that the repository isn’t broken at the end.
Part 1 was done through a “simple” sequence that extracts the needed information from every tag branch. At the risk of making myself eligible for the questionable classification from the start of this post, the code I used for this was:
tagproject="After the Storm" git for-each-ref --format='%(refname)' refs/heads/tags | cut -d / -f 4 | ( while read ref; do set -- `git log -n 1 --format="format:%ct %at" refs/heads/tags/$ref` export GIT_COMMITTER_DATE="$1 +0000" export GIT_AUTHOR_DATE="$2 +0000" set -- `git log -n 1 --format="format:%ae %an" refs/heads/tags/$ref` export GIT_AUTHOR_EMAIL="$1"; shift export GIT_AUTHOR_NAME="$*" set -- `git log -n 1 --format="format:%ce %cn" refs/heads/tags/$ref` export GIT_COMMITTER_EMAIL="$1"; shift export GIT_COMMITTER_NAME="$*" echo " * $ref (REF: `git rev-parse refs/heads/tags/$ref` CTS: $GIT_COMMITTER_DATE ATS: $GIT_AUTHOR_DATE CA: $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> CC: $GIT_AUTHOR_NAME <$GIT_AUTHOR_EMAIL>)" git tag -a "$ref" -m "$tagproject version $ref" "refs/heads/tags/$ref"; unset GIT_COMMITTER_DATE; unset GIT_COMMITTER_EMAIL; unset GIT_COMMITTER_NAME unset GIT_AUTHOR_DATE; unset GIT_AUTHOR_EMAIL; unset GIT_AUTHOR_NAME git branch -D "tags/$ref" > /dev/null done )
The most glaring issues with this code are that it invokes
git log multiple times for each commit (which could be solved if I were writing this pipeline for general usage, which I am not), and it does not take into account author/commit timezones; but it does not do so because
git svn does not either! Subversion normalizes dates to UTC, so
git svn cannot possibly know the original timezones used. Ideally, one would use
git filter-branch to rewrite these commits on part 2 (or even before, in fact) to convert their timestamps to local ones with a timezone offset attached, but determining the offsets to use involves knowing the authors/committers’ locations around the globe and any local daylight saving time regulations that may come into play depending on the local date and time. For a repository that has more than one committer or author, this is just pointless pedantry bordering on pathological perfectionism. I have a whole history with the latter and After the Storm which I would rather not repeat with something as mundane and short-lived as a one-time Subversion-to-Git conversion. Seriously.
Avoiding the kind of perfectionism that ruins lives is also the reason why I opted to leave commit messages intact save for the metadata trail instead of taking the opportunity to fix commits with long/inexistent summary lines, sloppy grammar/spelling/punctuation, commit messages consisting solely of a bullet list, and Kri’tan.
Rewriting commits and updating tags
Compared to the tag preparation part, the actual
git filter-branch invocation is rather simple:
# Temporary dir on /tmp (a tmpfs mount) to trade disk access for RAM gittmp=`mktemp -uqd --tmpdir gittmp.XXXXXXXXXX` # Strip double blank lines and convert git-svn-id metadata trail # to a more readable format (e.g. [Wesnoth-UMC-Dev SVN r99999]). msgfilter="cat -s | sed -re 's/^git-svn-id.*@([0-9]+) .*\$/[Wesnoth-UMC-Dev SVN r\\1]/m'" # Remove external music/ dir indexfilter="git rm --cached --ignore-unmatch -q -r ./music/" # Needed to rewrite tags to point to the rewritten commits tagfilter="cat" git filter-branch -d $gittmp --prune-empty --msg-filter "$msgfilter" --index-filter "$indexfilter" --tag-name-filter "$tagfilter" -- --all"
To be honest, disk I/O on my desktop machine is good enough (with a 7200 rpm HDD on a SATA III link) that asking
git filter-branch to use a directory on a tmpfs mount doesn’t make a noticeable difference, but your mileage may definitely vary. AtS’ repository is not too large nor does it contain particularly a particularly large number of files. Moving the operation from a remote VPS to my desktop, on the other hand, did make the overall process about two times faster.
--tag-name-filter "cat" part was actually recommended by both
git filter-branch and its documentation, since otherwise tags are left untouched and stop making sense since they point to commits that have been rewritten and thus no longer have the original SHA1 hash or ancestry. The rest of the invocation is just pedestrian business involving
git rm and standard Unix tools; it merits no explanation whatsoever.
Well, the recursive
/music directory removal actually does require some justification. This Wesnoth add-on originally included a couple of large music track files (in Ogg Vorbis format) within it, thus they ended up as part of its SVN tree. Over time, however, this approach proved suboptimal both for me (uploading new versions of the add-on from a 3.5G mobile broadband link) and players, who had to download over 14 MiB or so every time instead of 7 MiB, the approximate base size of the add-on without music. Thus, at some point I deleted
/music from the SVN tree in favor of a separate rarely-updated add-on, but its history remained there. Since converting to Git involves starting from a clean slate in a way, I decided I might as well delete a portion of history that’s no longer relevant or used and may take up a significant portion of the packed repository data... or it may not; it’s not the point, either way.
The end result is now available on GitHub, and it’ll become official once AtS version 0.9.9 is released.
It is worth mentioning that past a certain point, power tools like
git filter-branch become an unfathomable corrupting force; if you spend too much time tweaking an invocation sequence and examining the results, this force seeps into your very soul and makes you do absolutely ridiculous things. It is no exaggeration when I say that such an outcome was but narrowly averted for my conversion of the AtS repository.
Anyway, the last SVN commit and the first native commit to the repository now look like this:
commit bbf8df680322ce585debb2b3068b446b87616371 Author: Ignacio R. Morelle <shadowm@xxxxxxxx> Date: Mon Nov 25 06:27:34 2013 -0300 Add a Markdown README.md containing general info, replaces BUGS commit 040ca5e81ef2a8851e331cfc5a4f59ae03d6bca2 Author: Ignacio R. Morelle <shadowm@xxxxxxxx> Date: Sat Nov 23 02:18:29 2013 +0000 AtS E1S4: mismatched tense fix [Wesnoth-UMC-Dev SVN r19373]
I opted for not stripping the
AtS: prefixes in past commit summaries (mostly those since 2011) used to identify Wesnoth-UMC-Dev projects. The reason is that I don’t really want to spend more time figuring out what words after the colon would then need to be excluded from capitalization aside from wesnoth-optipng. Laziness.
The whole of
/music was excised from the repository history, resulting in an overall size decrease from 25 MiB to 17 MiB. Given the number of PNG file additions and recompressions (“wesnoth-optipng pass” commits) taking place in the past, as well as the existence of some shorter Ogg Vorbis files under
/sounds, I do not think it’s feasible to shrink it any further without dropping additional relevant history. Again, the
/music removal was purely for practical purposes.
I guess you could say that balancing out my perfectionism with my unrelenting pragmatism and laziness was key to accomplishing this task within a finite amount of time.
Next target: Invasion from the Unknown. Since it contains oddities like changing tag branches and an unusual layout for its first commit (revision 1), it’s probably not going to be as easy as AtS.