Previous Entry Share Next Entry
Release 2.1
jducoeur wrote in querki_project
After months of waiting (heck, *years* of waiting), I've just cut Querki 2.1. From a technical perspective, this is the biggest release in Querki's history, so please bear with me -- there's a lot to talk about.


The most important change is that Querki now has full, built-in History -- from today on, all changes you make to Spaces are tracked, and will (eventually) be undoable. (Yes, in programmer terms this essentially means that everything in Querki is now fully version-controlled.)

In practice, there is one major new feature available to the Owner of a Space: View History. If you go to Actions -> Advanced... -> View History, you get the History View of your Space: a reverse-chronological list of all changes to the Space. This list is pretty crude right now, and will quickly get very long, since it is listing every piddling little change; in due course, we'll improve the UI to summarize these changes and let you drill down into them.

The *really* cool new feature is under that: View Space at Time. When you're looking at the History View, you can click on any change and it will show a "View Space" button. Click on that, and you are now looking at your Space as it looked immediately after that change. You can wander around the Things, Explore, Search, and so on, pretty much as it was at that moment. (You can *not* Edit anything -- it doesn't make sense to edit the past.)

This plugs what I've thought for a couple of years was Querki's biggest hole: I found myself slightly *nervous* about editing my Spaces, and moreso about somebody else editing them, because they might break something. Now, it's possible to go back and see what things looked like before any given change, and in the not too distant future it will become possible to undo any given change, or in extreme cases even revert the whole Space to the way it was at a previous time. In programmer terms, this means you can "code with courage", without needing to be too timid about losing data -- it is now nearly *impossible* to lose data.

There are several limitations on History at the moment:
  • History starts today: for your existing Spaces, you will see the Space as having been "created" today. Basically, this is the point when we're starting to record everything.

  • Only the Owner of the Space is allowed to View History. Frankly, from a security POV the History feature is kind of dangerous, so I want to tread carefully here. In the medium term all Managers will be able to View History, but I'm not sure how much further we can safely go. It may become a separate security permission.

  • When viewing the Space at a particular time, we currently don't show Conversations. There's nothing intrinsically hard about that, but there's some extra work involved in showing the Conversations sync'ed to the same time.

  • User Values are *not* recorded in the History yet -- they're one of Querki's most advanced features (even I have only used them a few times), and it's going to be a bunch of extra work to put them into the History. It'll happen, but it will be a while yet. But this doesn't affect most Spaces for anything other than Ratings and Reviews.
Note that this is a *huge* change to the underpinnings of Querki, and involved a rewrite of several of the most central parts. I've tested it pretty heavily, but this is the sort of time when one discovers unforeseen bugs. So please: take a look at your Spaces, and if anything looks newly broken, tell me *immediately*. Let's make sure everybody's Spaces are running smoothly.

For the programmers:

In case anybody's interested in *how* Querki is doing this, here's a brief burble on the changes.

Up until now, Querki was entirely based on an underlying MySQL database, and worked in a fairly ordinary (if idiosyncratic) way. Each Space had its own set of tables in MySQL, and one of those tables represented all of the Things in the Space. When you made a change, it rewrote the row for that Thing. When you loaded the Space, it swept that entire table into memory (literally a "SELECT *") to build the internal representation. So basically, we had one row per Thing, and no sense of "history".

The underlying architecture is now *completely* different. We are now running on top of Akka Persistence, which is running on top of a new Cassandra cluster. In this world, a Space *is* its history: instead of writing the new state of each Thing to disk, we write the change itself to an ongoing history log. When you load your Space, we literally replay that log to rebuild the Space, exactly as it was before. (We also take periodic "snapshots" of the Space's state, to optimize the loading for particularly large Spaces with a lot of history.)

This approach is very natural in the Akka Persistence / Cassandra world, and it has some sweet advantages. Since reloading the Space is exactly the same as its original evolution, we wind up with a single code path, where before we had very different ones for normal operation vs. reloading. Since we've got that history just sitting there, we can introspect on it and show you what happened. And we can show the Space at a particular time simply by replaying the History up to that moment.

This has actually *always* been Querki's intended design -- I just didn't have the resources to build it myself in the beginning. But now that Cassandra is fully mature, and Akka Persistence is mature enough to be safe to use, they're a natural fit.

Other Bugfixes and Changes

While I've been at it, I've been tweaking and fixing a bunch of other stuff. In rough order from most to least significant:

It is no longer possible to create illegal Property Names: (thanks to Alexx for keeping on me to deal with this) for the longest time, it's been altogether too easy to enter Names for Properties that weren't legal. Often, this would cause the Property-creation process to silently fail; in a few cases, it would actually create the Property with a Name that then caused errors. This UI is now much more vigorous about enforcing proper Names. As part of this, fixed the Link Name code so that you can't set a Link Name that will later cause errors.

Querki Web traffic is now gzip'ped: this shouldn't be visible to you in any way except that, by and large, response times are likely to be significantly faster, especially on mobile. (I thought I'd done this ages ago, but it turns out I hadn't gotten it right.)

Logging in while looking at a Space now reloads the page: this was an odd regression that I don't entirely understand, but I made some tweaks to make this work properly again.

The Name and Link Name Properties for Spaces are now always separate: for most Things, there is a flag you can set (which is usually turned on), that tells Querki to derive the Link Name for the Thing from its Name -- that is, the URL will depend on its Name. That's problematic for Spaces -- it's technically a bit challenging, and it means you can easily accidentally change the URLs for *everything* in the Space, which isn't a great idea. So for Spaces, Name and Link Name are now always separate: you should say explicitly if you want to change them.

You can now use mailto: URLs in QText links: these aren't as useful as you might wish, because doing a fully-fleshed-out mailto: is too much for the link syntax to handle, but you can at least pop an email window with the address set. I will likely be adding a function to display complete mailto: links shortly. (I need it for one of my use cases.)

Rating Stars now work correctly for _average: the _average function runs on Ratings and Reviews; as you'd expect, it shows the average value of the Ratings on this Thing. Problem is, the number-of-stars display couldn't cope with fractions, so if the average wasn't exactly an Integer, it wasn't displaying. This now works correctly, showing fractional stars when it should.

Adding an element to a newly-created List in a fresh window now works: this was a subtle edge-condition bug that could show up if you refreshed your browser window while you were in the Model Designer / Advanced Editor.


Log in