Previous Entry Share Next Entry
Release 1.2
jducoeur wrote in querki_project
After a rather long delay (over a month), I've just cut the latest release of Querki. While we've been wrestling with delays in migrating to AWS, I've been working on a few things; here are the highlights.

The feature most likely to be noticeable is that most of Querki now uses the client. Previously, the new client was used whenever you were looking at a page within a Space, but all of the site-wide pages -- your profile, your index of Spaces, and so on -- were still using the old, hard-to-maintain web pages. Now, pretty much everything except signup and login go through the client.

The most obvious immediate effect of this is that you can now get to your notifications from your list of Spaces, right after logging in, without having to go into a Space first. In the medium term, this will also make it much easier to add desireable features like sorting your list of Spaces.

The really big motivator for this change, though, was the addition of Exporting and Importing Querki Spaces. This takes two forms.

First, from the Advanced... page of any Space, you can now Export the Space to XML. This produces a pretty faithful representation of the Space, with just a few exceptions:
  • Comments do not yet get exported.

  • User Values do not get exported.

  • Complex Nested Types aren't guaranteed yet. (I believe they export correctly, but don't import right if they are more than one level deep.)
The main data of the Space all gets exported, though, into a nice portable (and relatively readable) format.

Of course, this wouldn't be interesting if we weren't also able to Import an XML Space into Querki. On the Create a Space page, there is now an Import button; you simply give the name of the Space, choose the XML file to upload, and watch the nice progress bar keep you posted on it. (It isn't lightning-fast -- it's deliberately throttled so as not to overwhelm everything, and can involve a *lot* of database operations -- but I worked to provide accurate feedback of how far along it is.)

The main point here is to enable people to run their own Querki instances, and move Spaces around. A few folks have mentioned that they would like to run their own personal Querki sites, or at least want to be sure they can do so, and I'm broadly in favor of that. Querki's license prohibits *commercial* use of the code without a license, but *personal* (or non-profit) sites are fine. And I know that, as a user, I would be more comfortable using Querki if I knew that I could, if necessary, extract my Spaces and run them myself -- data portability is a Good Thing.

Sometime in the not-too-distant future, I plan to implementing Add XML to a Space. The plan is that you'll be able to ask Querki to show you an XML schema for your Space; you can then write an XML file from some external source matching that schema, to add new Things to the Space. This will be the first mechanism for importing external data into an existing Querki Space.

Closely related in code, although it's a completely different use case, and more useful to most people: you can now import a MySQL Database into Querki. This is a big deal for both me and Aaron -- we each have personal MySQL DBs that we'd like to be able to move to Querki, so as to be easier to use. Indeed, my Comic Book Database is now an official Querki stress-test: it only includes titles, but is already over 3000 records, so I'll be using that to test the scalability of both the code and UI; expect some enhancements to come soon as a result.

Mind, this is *highly* experimental at this point. It only works with MySQL dumpfiles; I don't guarantee to work with any random SQL. It only deals with a subset of the possible SQL syntax -- it handles the most common stuff, but there are undoubtedly a lot of less-common edge cases that it needs to handle yet. If you try uploading a MySQL file and it chokes, please drop me a line: I'm happy to add enhancements for this.

That said, it's pretty easy to use. Again, from Create a Space, press the Import button, choose to upload a MySQL file, and watch the pretty progress bar as it slowly builds the Space. It tries to do a bunch of smart heuristics, in particular:
  • The common data types for columns are translated into corresponding Querki Properties.

  • Standard auto-increment ID columns are elided, and columns referring to them through constraints are replaced by Link Properties in Querki.

  • Auto-updating Timestamp columns are elided, since that feature is built into Querki. (As the mod_time function.)

  • Names in the usual "fee_fie_foe" format in MySQL are automatically converted to Querki's "Fee Fie Foe" idiom.

  • If a table has only one text column, it figures that that should probably be used as the name of the Thing in Querki. (This means that links to enumeration tables tend to Just Work.)
As this feature gets used more, we'll refine these heuristics. We'll also eventually add a UI, to let you fine-tune the details on the way in. But it already works fairly well for straightforward databases.

To support the MySQL Import, Querki now has two new Types: Large Number and Floating Point. These correspond to "long int" and "double" in typical JVM usage: both are 64 bits.

And now, back to beating my head against AWS...


Log in

No account? Create an account