Extended error reporting in GLib

We have landed a feature in GLib called an extended error domain, and it is available in GLib 2.67.2.

Extended error domains allow the developer to attach extra information to GError without either cluttering the error message string or creating a new error type or adding another out parameter to the function for error details. You can see how to do it by reading an example in GError documentation (search for "extend the GError type").

The API is not yet set in stone. It will freeze on 2021-02-13, so please try it out and send us feedback. Thanks.

GNOME Bugzilla Upgrade

GNOME Bugzilla is about to be upgraded to 4.4. Finally. It was really long overdue. And I'm not alone with this opinion.

To be able to reach this point I needed to port more important customizations into extensions. But before embarking on this task, I wrote a patch splitter tool.

Patch splitter

The diff between stock Bugzilla 3.4 and GNOME one (gotten with some magic bzr invocation taken from this site) was just too large to grasp. I tried rebasing GNOME Bugzilla on top of 3.4, but I gave up after less than 2 minutes. It was a folly.

So instead, I moved the code to git, produced a git patch, annotated it and wrote a tool that split it into myriad of smaller, self-containish patches. The effect is this branch.

I did that to know exactly what changes in code each customization brought. This effort wasn't probably worth it, but at least it was a bit of fun. The tool sits here. Go there at your own risk.

GnomeAttachmentStatus

Next was GnomeAttachmentStatus extension. I already wrote a post about it, so I won't be repeating myself here. I'll only add that I'm not happy with its code. Now I wonder if some of the hacks there are necessary. Probably yes, who knows?

TemplateOverrides

Technically it wasn't next extension I put into customizations, but it was the second one I wrote from scratch. I needed a central place where template overrides could be stored. The necessity appeared when I was porting some other extensions. Two of them were overriding the same template. So, best idea is to take that file from those two and put it into the third, right? Right.

Also, the problem with overriding templates is that if original one changes then the override might need updating. And that doesn't happen by itself. Nor does this extension do this for us. But instead it makes the checksetup to fail if Bugzilla admin does not acknowledge the change. Or forgets to do so. Or misses to do so.

ExtensionDependencies

That is the third and the last extension I wrote from scratch so far. Its name is rather self-explanatory. It mostly serves as a kind of enforcing documentation. That means that checksetup will fail if any of listed extensions is either not loaded or inactive. Of course, the dependencies are provided by extensions itself. This one only provides a mechanism.

Hm, it's quite least useful of all of them out there I think.

Other

Browse, DescribeUser, Developers, GNOME, PatchReport, ProductInterests, StockAnswers and WeeklyBugSummary extensions were already ported some years ago by Olav Vitters (as far as I know). I just had to import them into single git repository and make some minor changes. Like porting to GnomeAttachmentStatus. Moving their template overrides to TemplateOverride. Adding list of runtime dependencies for ExtensionDependencies to check. Fixing some small bugs.

Boooring.

TraceParser

TraceParser extension was slightly more work - it is targeted for 3.6. Fortunately 3.6 introduced a new extension format which hasn't changed too much since then. The only minor difficulty was that some TraceParser's table was referencing comment_id column in longdescs table. And that column has changed its width between 3.6 and 4.0 from INT3 to INT4. Or, translating from Bugzillese to mysqlese, from mediumint to integer. In the end most of the time spent on this extension was waiting. Waiting patiently for checksetup to finish its churning only to find out that either it was killed because something. Or checksetup was running on wrong test instance (without updated TraceParser extension). Third run finally confirmed that migration works.

Splinter

Splinter - even moar porting. I could use the port made by Mozilla, but I decided against it. It didn't work for me and I didn't feel like digging in it and finding out why. So I did my own thing, but there is nothing to boast about. Owen Taylor (author of splinter) might be disgruntled with this port after seeing that I dumped the testing infrastructure into misc directory to bit rot and put all the javascript code into single monster file.

Of course, this all sounds backwards, but my changes in logic were nonexistent, so everything should work (famous last words).

So, please don't tell him this port, ok? Thanks.

The end

Well, that's it. There are still a lot of bugs to be fixed, I'm sure. But Bugzilla is already usable. Ever-awesome André Klapper already jumped in and fixed a bunch of them. The customizations are in bugzilla-gnome-org-customizations git repo. Many thanks to André and to Andrea Veri for overseeing and helping me with this work. And to Olav for doing most of this work for me two (or more) years ago.

Oh, and actually I am going to repeat myself - the git-bz tool will need some small update, because attachment status field name has changed.

GNOME Bugzilla attachment status

I have been asked to help in migrating GNOME's Bugzilla instance to a latest stable version. Currently it is a version 3.4 with lots of modifications (or extensions, if you will). Quite an oldie - it is 2.5 year old. And it (obviously) does not get any bugfixes or security updates anymore.

If I remember correctly, I was asked because I admitted to Chris that I know a wee bit of Perl and I did not dislike it. So when Kat (a member of GNOME Foundation Board) asked him if he knows anybody who could help with it, he directed her to me.

(Did I mention that I, Chris and Kat used to work in the same company, at Openismus, for a short while? "lovely how our company links still work" said André, GNOME Bugmaster and also a colleague from the company. Heh heh.)

I am no Bugzilla expert, but after reading a diff between stock 3.4 and GNOME Bugzilla I had an impression that 3.4 is not really extensible in clean way - lots and lots of modifications are sprinkled all over the place. In default templates, in Bugzilla core, in CGI scripts.

So I was asked to help in porting the attachment status to 4.4. From user's perspective this extension only adds three things:

• Another column to attachments table in bug view (example) showing status of patch. Or nothing, if attachment is not a patch. 
• A combobox in attachment editing page (example). Click "edit details" at the top of the page to see it.
• An option to fields combobox in advanced search page (example, see "custom search" section).

Development was painfully slow, mostly because it was done in my oh so copious free time. But, after 5 months or so, it is finally there, ready for testing. Just see the example links above. The code is on GNOME git now. Note that this extension is probably not compatible with an old one, so third-party tools might not work with it. I also managed to keep all the code inside the extension, that is - no changes in core. It is unfortunately not very pretty.

Any future work on this extension? Probably making third-party tools (namely git-bz) work with it. And maybe adding an attachment specific section to advanced search if GNOME bug wranglers want it.

Gmmproc progress

I have recently pushed over 30 commits to glibmm gmmproc-refactor branch. This concludes gmmproc's first "milestone" - to generate glibmm sources, build a library, build the tests and examples and finally succesfully run the tests. But by saying "glibmm" I mean only GLib wrappers - Gio wrappers are next on the list.

First milestone?

Yeah, right, as if there are any. That "first milestone" may look modest, but I actually think that it is like 75% of work done. GIR parsing, type conversions system, parsing templates, replacing m4 code with perl and then tying them and squeezing them with foot into some semblance of a program - all of those had to be written to have something working.

I haven't yet made any benchmarks of gmmproc rewrite, but right now it seems to be faster than the old one. Still old one can be faster if we run parallel build (make -jX) - there is one of main differences between currently used gmmproc and its rewrite. The former processes and writes one set of files (like from widget.hg and widget.ccg to widget.h, widget.cc and widget_p.h) on every run, while the former processes all files in single run. That forces me to actually do parallel processing inside my program instead of reusing make's powers. Fortunately, I tried to write most of my code in a way that parallelizing it would be easy if done in OpenMP style.

Talking about speed - there is one area I would like to speed up in general - GIR parsing. It looks like the slowest part of whole gmmproc, but I didn't yet tried to profile it. I was thinking about using typelib instead, but from what I heard, there is no detailed C type information there (the "c:type" attributes from gir). This is still to be confirmed as it is not on top of my nowhere written list. Otherwise, using typelib would be, I imagine, faster. Provided that C-to-perl interface to GIRepository is written first - whooo, yak shaving!

Some plans

Still, I feel no hurry in rewriting as I rather see it to be used in next version of glibmm/gtkmm (I assume that those will be glibmm-4 and gtkmm-4). Thus I am not writing a strict drop-in replacement - I already have made (or I plan to make) some changes in generated API (like exception-to-GError conversion in vfuncs wrappers) and in template API (I would like to get rid of _CUSTOM_CTOR_CAST and similar in favour of _CLASS_FOO options).

There is still much to be done beside threading the application:
1. generating Doxygen documentation based on docs in GIR,
2. code documentation,
3. gmmproc macros documentation,
4. tests (there are some, but still not enough),
5. reports about unwrapped classes and functions,
6. and probably more.

I am happy to get this far and to actually finally see something more or less working.

Small disclaimer

The code is ugly mess. It needs to be reorganized, reshuffled in several places. WrapParser class is huge, even with moving some functions into shared modules. 'use' clauses have to be reviewed. 'constant' module for section traits was probably a crappy idea. Sometimes class/function naming is bogus (identity conversions, imbued types, tokens store?). Code style is very inconsistent (I started reading Modern Perl somewhere in the middle).
Oh, and commit messages are useless - they are probably going to be squashed at some point into single enormous commit for merging into master (and I am going to get only +1 commit of contribution on glibmm on Ohloh, damn!).

SyncEvolution

These are bit dated news (well, only two weeks or so), but the fork/exec rewrite of DBus server was merged into master branch. The main aim of rewrite was to allow several concurrent synchronisations to be done (provided that the sync sessions don't conflict with each other). It wasn't possible before, because libsynthesis (the main library used for sync) does not have asynchronous API. There were two solutions: either use threads or spawn children processes doing syncs. The latter solution was chosen, because Patrick Ohly did not want threads to avoid complications with some possibly thread-unaware libraries. (Of course waiting for libsynthesis to get asynchronous API wasn't an option.)

The rewrite involved splitting the code in one file of syncevo-dbus-server into several files and then detangling all tightly coupled classes - I guess that the result is quite nice compared to what it used to be. In the meantime new C++ DBus layer based on GLib GDBus was added. Most of this work was done by Chris Kühl - I was mostly helping in fixing issues in new DBus layer and then working full-time on it and the code using it. As a final step I ported command line test from C++ to Python, so our work could be proved to work.

This was a really long task, very often I felt that I was losing grasp on the code I was working on and probably sometimes I had actually lost it. Now I feel happy to see it finally merged into master.

gmmproc refactor

What?

gmmproc is a script used by most of C++ bindings of GNOME stack (called also mm-modules). It is responsible for generating actual C++ headers and sources from templates. That is so, because wrappers for most C API are straightforward, so writing them manually would be just a tedious job. Instead of that we just provide C API information, specify what names should C++ wrappers have and how to get a C instance from C++ one and vice versa and all happens automagically. Even documentation from C sources is taken and translated to use C++ names.

1 or 2 years or so ago.

Some time ago I was thinking about refactoring gmmproc. I was not the only one, because there were several notions that a rewrite could be welcome. I even made a topic on gtkmm-list about it and created a page at live.gnome.org. Main points would be using GIR as a source of API information and documentation, and getting rid of m4 as it makes it hard to add new features. I had some grand plans like creating two backends - one for defs (old API information) and GIR (new API information), creating conversion files on-the-fly, some configuration file a'la Doxygen, blah blah blah. Eventually I started some coding in Python, drawed some diagrams in my sketchbook, switched to C++ and then Perl and suddenly development slowed to molasses, because I already got bored. Apparently writing API structures is really uninteresting. Especially when it goes really slow and no end of it is in sight.

Now and ongoing.

I got back to it recently with a firm decision that I will finish it. I got over API structures by generating them basing on GIR definition I had to deduce from some python code in gobject-introspection (yeah, I wrote my own definition, because there seems to be none of such thing upstream - documentation is pitifully out of date). Also, most of the parser is also generated, but most important things are hand-written. Then I switched to refactoring a WrapParser and Output, which are 10 years old Perl code, added some needed infrastructure replacing m4, converted _CLASS_GOBJECT from m4/defs to Perl/GIR, added a scanner finding C <-> C++ type equivalents. My first aim is to have any compilable code being generated. For that still much work left to be done - mostly understanding and translating rest of m4 code to Perl (booooring!) and writing C <-> C++ conversion code (the one saying how to get `GtkWidget*' from `const Gtk::Widget&'; slightly less boring, but still).

Future plans are... no no no, no future plans for now. I did that before and got baffled by amount of work that needs to be done. Slowly all needed features will be introduced.

All of the code sits in gmmproc-refactor branch of glibmm. Additions here are mostly a bunch of kilo-line commits happenning once for two-three weeks, because I work on it offline at home in my free time. In the end all of it is going to be probably squashed into single commit before merging it to master.

I am usually pushing changes to repository when my single patch grows to at least 500kb. Now it has only 200 kb, so you have to wait. :)

SOAP server example

Recently at Openismus I was asked to write a SOAP server, which would run on an OpenWRT router set up by Dave. The server uses libsoup, so writing it was really straightforward, but fun. Of course there were some glitches, like trying to use SoupAddress to set up where server should listen - it ended up in segfaults somewhere inside libsoup. I decided to not use it in the end, but Jens gave me a hint about a function of SoupAddres resolving it first. Segfaults are clearly a sign of bug somewhere in libsoup. Primarily SOAP server had to be someting bigger but for now ended up as an example. There are some things lacking for sure. Like actual compliance to SOAP standard (probably), documentation or subclassing the SoupServer into SoapServer and providing some nice interface hiding all ugly details. But at least it has autotools based build system, client application and some basic tests. So it maybe doesn't look that bad. The code (LGPL 3+) is on Openismus Playground.