Querki Development Journal

Today's release was a fine example of the ball-of-twine effect, where you start out trying to do one thing, and it leads you to another, then another, then another. I originally set out to solve one problem (email unsubscription), and wound up making changes that I think are far more important and useful to most users. So I'm going to start with the important changes first.

Changes to Invitations and Signup

Guest Mode: the change that is probably going to matter most is a serious rewrite to how you accept invitations. Traditionally, if somebody invited you into a Space, you would click on that, and then immediately be presented by a "log in or sign up for Querki" screen. Functionally this made sense, and it took years for me to question this design, but in practice it's kind of off-putting. If I've just been invited so that I can fill in a form or submit an RSVP or something like that, it's terribly heavyweight, and I suspect we've driven more than a few users away because of it.

So this is now turned on its head. When you click on an invitation link, you are immediately taken into the Space, logged in and able to do whatever the invitation allows you to. You're still somewhat anonymous, but you are logged in as a Guest, and the system knows how to associate that link with what it's allowed to do. This allows you to build eVite-like systems in Querki, only far more powerful: you invite people, and they can immediately come participate.

In fact, this feature was in the original prototype of Querki -- Querki's very first use case was managing my wedding, so it was critical, allowing folks to not just RSVP but say things like whether they had children in tow. (And the Space allowed also let Kate and me do things like organize the planned photos in advance.) But that original implementation was a horrible hack that never worked well, so I ripped it out. It's finally been reimplemented, in a way that I think makes sense in the long run.

Of course, having to keep the invite email around is a pain in the tuchus if you are going to come back to this Space over and over again. So when you enter the Space this way, it starts you off with an alert that you can click on, which lets you log in or sign up right there -- if you do so, it will associate the invitation with you, and you are now a proper Member of the Space. But it's now strictly optional: the invitee gets to decide whether to use a full Querki account or just click on the link when they want to access the Space.


Shared Links: and while we were gaining that feature, I decided to go all the way and add a new feature that I've been wanting roughly forever -- shareable links.

Most of you are familiar with these from Google Docs and suchlike: you can now ask the Space for a URL, and the access rights are associated with that URL. Unlike conventional invitations, which are intended for a single recipient, Shareable Links are specifically designed so that you can paste them into email, web pages, Facebook or whatever. You tell Querki what the recipients of the Link are allowed to do, and anybody who clicks on the Link immediately has those permissions. (Under the hood, it creates a Custom Role with those permissions, and all the invitees automatically get that Role.)

Shareable Links are very similar to the conventional invitations -- when you click on it, you enter the Space in Guest Mode unless you are already logged into Querki, and you can log in or sign up from the alert that shows on the first page. But they are great for any time that you want to provide access to a number of people at once. I don't recommend using them *completely* willy-nilly, and you should think about what permissions you want to give to the recipients, but it's a great way to say, for instance, that everyone on a given mailing list can now access this Space.

Email Unsubscribe

This was actually the original thread that set the ball of twine unwinding. Querki's email system has always been pretty primitive -- not only did the emails look kind of illegal, they weren't technically legal because of anti-spam laws. This is why Querki has refrained from ever sending any sort of "What's going on in Querki" emails to the existing users: there was no Unsubscribe link, and it's officially spam if you don't provide that. So I've kept email usage close to zero, since I don't want to break the law.

This is *finally* fixed -- from here on, all invitations sent by email except for ones you specifically request to yourself (such as reset-my-password confirmations) should have Unsubscribe links at the bottom. Those Unsubscribe links are smart about the email that you got them from, and provide you with unsubscribe options that are appropriate to that email -- for example, Space Invitations now allow you to opt out of receiving any further invitations from this inviter, or from all Querki invitations period.

(The reason that this led to all the Invitation-handling improvements is that I had to introduce a notion of "identity" based on email address, not just on Querki logins. Once I had that, it became possible to interact with a Space solely based on that identity.)

While we were at this, I restructured the invitation mechanism to behave as you would expect in a modern platform: if you are already a Querki member, and you get invited to a Space, you receive an in-Querki notification as well as the email. We'll slowly be moving more and more in this direction, and eventually allowing you to choose which notifications you want to receive in-Querki, and which by email.

Oh, and along the way I put some work into spit-and-polish so that invitation emails are no longer hideous. I won't claim to be a UI designer -- they could definitely still use some work -- but at least they are no longer actively ugly. (Note that the fix for this does depend on halfway-modern mailers, that support stylesheets in email. Now that GMail finally supports this, I believe that it should work in current versions of all major mailers, but some old versions or more-obscure mailers may not cope. Sorry -- fixing it for all mailers is far more work than I can bite off right now.)


All of the above features are very new and a little experimental, so it wouldn't astonish me if there are some bugs. Please drop me a note ASAP if you encounter something misbehaving.

There were also a fairly large pile of bugs fixed, but they were all edge cases that I suspect nobody besides me has encountered in practice.

Given the presence of Unsubscribe (so folks can easily opt out), I can finally start to send emails to all the users. I don't want this to become spammy -- my current target is a once-a-month update. Hopefully the first one of those will go out next month, so I can remind folks that Querki exists and is improving apace...

Today's release doesn't change anything terribly obvious, but it has one major effect:

You can now sign up for Querki from the Login dialog: previously, the only way to sign up was from the Querki homepage -- if you were reading a Space, and wanted to join, you had to get out of that Space, go to the root page, sign up, and then come back again.  That was inconvenient to begin with, and it's borderline intolerable if you're trying to join by using an App.

So we've seriously reworked the signup workflow.  In particular:

• If you're looking at an App when not logged in, and click the "Create a Space from app name" button, it will pop the login dialog so that you can log in; it will create the Space once you've done so.


• Moreover, the login dialog now has a Sign Up button, so that you can create a new account right then and there.  As soon as you've done so, it will create the new Space.


• The Terms of Service page has now been pulled into the Client, in order to support this.  It looks largely the same, but it's been entirely rewritten.


• The activation-email procedure has gotten a facelift.  Previously, if you signed up for Querki you couldn't do anything until you clicked the link in your activation email.  (So that we can validate your email address.)  That turns out to work poorly when you're signing up as part of setting up a Space from an App, so it's been loosened.  But some key functions are still restricted until you activate, and several more places now have messages to that effect (as well as "resend activation email" buttons).

Put together, this should make it much easier for folks to start using Querki from a link to an App.

And now, it's time for me to go start seriously building some Apps.  I have a few to do first, but suggestions greatly welcomed...

Today's release is a bit of a grab-bag, but has one significant new feature that I've been wanting for about two years now: the Edit Space Info page.

The Edit Space Info Page

Previously, when you pressed the Edit button on the root page of your Space, you went to the Advanced Editor for the Space. That's complete and flexible and all -- but it's also Advanced, as the name says, and hard to use at best.

So now, that button takes you to the Edit Space Info page, which is a simple and focused page that lets you edit the stuff you *usually* care about for a Space. Specifically, it lets you edit:

• The Name of the Space


• The brief Summary of the Space


• Any additional Details describing this Space


• Whether this Space is Public (everybody can read it), Members-only, or something Custom

For now, that's it. I suspect we'll add more to this page as we go, but I want to keep it small and focused on the really common needs. (There is also a link to the Advanced Editor, for times when you really want to get into it.)

The Summary and Details here are kind of new: they're standard Properties that have been around forever, but they've rarely been applied to Spaces. Now, you can easily add and edit them on the Edit Space Info page, and if you have them they will be shown on the root page of the Space by default. This provides you with an easy way of saying "this is what this page is about" without having to write a custom Default View for the Space.

This Summary and Details are also now used when Extracting an App from this Space -- if they are present, they'll be used as the default text to use for the new App.

The Public/Members-only distinction is a *wild* over-simplification of Querki's very fine-grained security model. But I suspect that 90% of the time, it's all you actually care about, so we're going to make it very easy.

Most importantly, the Edit Space Info page is now interposed into the Create Space workflow: as soon as you create a new Space, you go to Edit Space Info. While you don't *have* to do anything here, this encourages you to think about these fields. (Especially whether the Space is Public or Members-only.)

Other Significant Enhancements and Fixes

There are a bunch of bugfixes and tweaks in this release, but most of them will probably fly under the radar. The interesting ones are:

Querki now has Undelete: this is a feature I've been wanting *forever*; now that we have History, it was fairly easy. If you go to Advanced... -> View History, and click on a Delete record, there is now an Undelete button. Click on this, and the Thing will reappear as if it had never gone away. Yay!

As a consequence of having this under the covers, we've also make it easier to Delete things. Previously, you always had to go through a confirmation dialog with a dire "Are you sure?!?" warning, since there was no way to recover. Now, in most cases Delete just works. But *after* deleting, it will show a message with an Undelete button, so you can recover immediately in case you did it by accident.

Photo Upload works from Mac Safari again: (At least, I *think* it does. I'll be more confident after more experiments.) Sometime in the past, *something* changed in Safari, such that photo upload stopped working. As far as I can tell, this related to the BlueImp upload library's attempt to resize the photo in order to save bandwidth. Turning that off seems to fix the problem. Fingers crossed.

Meta-note: given the recent exodus of a lot of Querki's early adopters from LiveJournal, these release notes will probably move somewhere else in the not-too-distant future. I might move them to DreamWidth, but unfortunately, I gather that communities don't have the same DW-LJ crossposting capability as personal journals, so that's not an obvious win. Alternatively, I might finally implement RSS for Querki -- a feature that has been planned for at least 3 years -- and begin to maintain this log *in* Querki proper, with feeds in other systems. Opinions welcomed.

It's been fully two months since 2.1.3 -- that's partly because of the holidays (I was away for the bulk of December), and partly because I've been heads-down focused on the big new feature. It's not perfect, but I think it's good enough to start trying it out on real Spaces, so here we go...

New Feature: Apps

Apps are the last of the Big Central Features that have been in Querki's design from the very beginning. Querki has always made it (relatively) easy for you to build a custom Space to manage your data for a particular problem; Apps are how you let your community (and maybe others) create their own versions of that Space.

I'm not going to go into full documentation here: the feature is new enough that I don't recommend everyone go lifting Apps out of their Spaces until I've done it for a bunch of mine. But here are the broad strokes.

What is an App?

An "App" in Querki isn't quite the same thing as on a smartphone, but the concept is close enough that we decided to use that term here. You generally don't write an App from scratch; instead, you build a Space that works, and then you extract an App from that Space. This "extraction" shouldn't visibly change your Space much, but it actually makes some pretty radical changes -- it creates a new "App" Space, takes the Models and Pages from your Space, pulls them out into the App, and then converts them into "shadows" in your Space. Basically, the Things in the App are the "real" versions; the ones in your Space now inherit from those. The App is a new Space, owned by you, containing the *structure* of your Space; meanwhile, your *content* remains in your Space.

That being the case, you can then share the App with other people: they can easily create their own Spaces from the App. Their Spaces will have the same structure as yours, but they can populate them with their own data.

Of course, Querki is all about customization, building Spaces to suit *your* needs. This is what makes Querki Apps *really* different from the usual concept: everyone who uses an App is free to customize their own version as they like. They can add their own Properties, Models, Pages and so on. And while it's not yet implemented, they will eventually be able to take their own changes and suggest them as enhancements to the shared App.

(The programmers who work in the open source world are now going, "wait -- you've reinvented GitHub". Yeah, kind of, albeit with a lot more structure and a lot less incomprehensible jargon. I've intentionally adopted a lot of git's better ideas in the design of Querki Apps.)

Creating an App

Like I said, I don't recommend trying this *quite* yet. But for the brave, here's how you create an App.

To begin with, all Spaces now have an Info Page. You'll find a little "i" icon on the root page of your Spaces. This takes you to the Info Page for this Space. (There isn't yet a good way to actually *write* this Info Page, but that's coming shortly.)

On the Info Page, you'll find an "Apps" section; under that is a button labeled "Extract an App from this Space". Clicking on that takes you to the Extract an App page.

The Extract an App page asks for:

• The name of the App


• A one-line summary of the App


• A more detailed description of the App

All of these are required.

In addition, it displays a list of the Models and Pages, as a tree-structured checklist: if you open one of the Model nodes, you'll see the Instances of that Model, usually unchecked. This allows you to customize exactly what will be extracted to the App. In theory, you shouldn't usually need to customize this -- I'm trying to set the default up so that it's usually what you want to do. But you can check and uncheck items from this list, to build the App you want.

Once you've filled all that out, click "Extract App from this Space", and it will create the new App.

That's it! The whole point is that, once you've built the Space, pulling out an App should be quick and painless -- literally a one-minute operation.

(There's also a new App Gallery; your App currently gets automatically enrolled there, but I'm rethinking that decision. It's likely to become something you do more deliberately.)

Instantiating the App

Once you've created an App, how do other people use it? Well, after creating the App, you're taken back to the Info Page of your original Space, which now lists the App. (Eventually, it will be possible for a single Space to inherit from many Apps, so you can have "mixin" functionality.) If you click on the App, it will take you to the Info Page of the App, which will show the Summary and Details that you entered when you created it, along with a button to "Create a Space". Press that, and it will ask you for the name of the Space you want to create. Fill that in, hit the "Create" button, and *poof*, you have a new Space based on this App.

You can share the Info Page's URL with friends as you see fit -- this is really the point, and I encourage sharing. You can also create multiple Spaces yourself, based on the same App; that doesn't make sense for all Apps, but sometimes it is really useful. (Imagine, for example, specialized Inventory Spaces for different collections.)


I'm going to be playing with this stuff extensively in the next few weeks; you'll probably see me announcing Apps for people to use on various appropriate forums. I may well ask some of you if we can extract Apps from stuff that you have built. It's exciting times -- this is the feature that is going to allow people to use Querki with absolutely no building required on their own part, just leveraging the work of other members of their communities. It's been one of the goals for Querki from the start; I'm looking forward to seeing what we can collectively do with it.

Revert Space

One of the major concerns about Apps has always been that they do *dramatic* surgery on your Space. In theory, the changes should be invisible, but until we've done it a lot we won't know if there are bugs. And one of Querki's increasingly-central tenets is that you should never lose data.

So along with Apps, I've released another key feature: Revert Space. This was one of the main reasons why History was the feature right before Apps -- Revert allows you to recover if something goes horribly wrong.

For now, you get to the History of your Space by clicking on Actions -> Advanced... -> View History. That's been around since 2.1. What's new is that, if you click on an event, you'll find a "Revert to Here" button. This requires confirmation, because it's dramatic and powerful: if you confirm, it will undo all changes since that event. So if your Space has become broken (due to a bug, or vandalism, or whatever), you can roll it back to before the damage.

Note that Revert doesn't actually *remove* the changes -- instead, it adds a new event, setting the state of the Space back to the specified point. You can later change it to some other point, rolling forward to where you started or somewhere else. I don't recommend going crazy with this, but it's very flexible.

(The open-source programmers in the audience are nodding and going, "still sounds like git". Yep, basically.)

Note that Revert rolls back the *entire* Space at this point. We'll eventually add finer-grained versions, that allow you to reset the state of a single Thing, or even a single Property value. But one thing at a time.

Other Significant Changes

Apps consumed most of this release, but there are a couple of other changes:

Sharing to Facebook now displays the Summary: we're gradually figuring out how Sharing *should* work; historically, Querki pages shared on Facebook resulted in a pretty terrible preview. Now, if you specify a Summary on the Thing, that will be used as the text -- that's often what you want.

Lots and Lots of Unit Tests: it's not obvious to the user, but goes a long ways to making *me* more comfortable with the system. Suffice it to say, I've rewritten some of the core code, that wasn't previously unit-testable (because it was wildly asynchronous) to be friendlier to the test environment. (This uses some of Scala's crazy power, to build things in such a way that code can be synchronous or not, depending on how it's called. It's very cool: buttonhole me about it sometime if you're into neat programming techniques.) The result is that, for example, there is now a proper unit test for Extracting an App, which exercises the real workflow so we can catch bugs quickly.

Started extracting the Gadgets library: mostly interesting to the programmers, I've begun to lift Querki's homebrew UI framework into a proper library, so that others can potentially use it. (We got talking at the Scala eXchange conference last month, and it looks like a fair number of people are looking for a library like this.) This is mainly relevant here in that I've been rewriting some of the details of the UI as I go. Those changes *should* be invisible, but bugs are definitely possible. As always, if you see something that isn't working, please yell.

Today's release contains a whole bunch of tweaks to stuff that has been annoying users (including me) for some time. Going through it...

Reworking the Create Property box

I've been noticing for some time that the workflow for creating Properties, while theoretically clean and elegant, has a lot of little hiccups. Today's changes were primarily focused on smoothing those roadbumps.

Changed "Exactly One" to "Required": one of Querki's idiosyncrasies is that it doesn't make the usual assumption that values either exist or contain the magic value "null", as most databases and programming languages do. Instead, when you create a Property, you always specify its contents. I originally thought of that as how *many* elements it contains, which led to one of the choices being "Exactly One", but that's a weird non-adjective, and rubbed along poorly with "Optional". So I've changed that term to "Required", so that it contrasts better. So your choices are now "Required", "Optional", "List" and "Set". (Yes, List and Set are different. But I'm less and less convinced that this difference really matters to the user, so they might yet get squished together.)

The default Collection now depends on the Type: since the inception of Querki, new Properties defaulted to containing Exactly One value (what is now Required). But over the years of using it, I've concluded that that is almost never correct. In practice, the ideal Collection usually is related to the Type.

So the Collection now gets set to a default that depends on the Type you choose. Tags default to Set, Photos default to List and user-defined Model Types default to List; everything else currently defaults to Optional. These choices are not mandatory -- you can choose something different, and there are good use cases why you sometimes would want to -- but these defaults seem to be appropriate based on my experience of creating a lot of Spaces by now.

(Note that I am planning to tweak the semantics of Required in the not-too-distant future.)

Rearranged the Create Property box: to support the above, the choice of Collection now comes below the choice of Type. Previously you chose the Collection first, *then* the Type, but the other way around turns out to make more sense in practice.

Create Property box no longer resizes constantly: when you choose the Type you want, we show the documentation for that Type. Which is great, but it was resulting in the box size constantly shifting, and the buttons jumping around, which is really annoying. So the documentation now shows in a scrollable, fixed-size window instead.

"Pointer" types now require you to choose a Model upfront: this is actually the big one. Querki has two "pointer" Types -- Thing, which always names an existing Thing, and Tag, which is a name that *might* point to a Thing that you haven't created yet. There is a Restrict to Model meta-Property, that lets you say which Model this Property uses; for example, if you have a Meeting Model, with a Participants Property, that might be a Tag that is restricted to the Person Model.

In practice, you should *usually* restrict these Types to a specific Model: it means that, when you start typing, it can do a better job of prompting you with existing values that you've used before. But historically, the only way to get there was to manually create that other Model *first*, then create your Property, then edit your Property to point to the Model. It's been an ongoing pain point.

So this is now seriously revamped. When you are creating a Property, as soon as you choose "Tag" or "Thing" Type, it immediately presents you with a sub-panel that *requires* you to choose this upfront. You can choose to leave it free-form, or point to an existing Model, but most usefully you can tell it to create a new Model, and specify its name. It will immediately create a skeleton Model for you, and use that for the new Property. You can flesh out that Model later if you want, but don't have to -- it's not unusual to have a Model that exists solely to hang a particular kind of Tags off of.

This isn't necessarily the last word in this workflow, but I believe it'll be a big improvement, shaving 30 seconds off of Property creation in many cases. (Yes, that's a lot of time -- part of the point of Querki is that you can set up a Space in a few minutes.)

Model Type only shown in Programmer Mode: finally, when creating a Property, you can create it from an existing Model -- this is how you create complex data structures in Querki. This is *crazy* advanced, though: even I have only used this feature half a dozen times at the user level. (We use it internally in various places.) So we no longer display the option to do this in Builder mode: this is very much a Programmer feature, and it's only shown if you choose to be in Programmer mode.

Other Changes

Improvements to Facebook sharing: in the long run, being able to share Things on Facebook (and other such platforms) is a pretty high priority -- Querki is all about collaboration, and we want to work well with the social networks. But Facebook's rendering of links is, let's say, kind of persnickety. It's easy to deal with if you are writing a platform where everything is structured the same way, like a blog or news platform, but it's tricky for something as general as Querki, and there are a lot of details to get right. I've put in a couple of tweaks on this.

First of all, we now suppress HTML tags in the code we share with Facebook, because it turns out that they show up *as* tags rather than as the text you want. This fix makes the displayed text a *lot* more reasonable.

Second, if you specify one, Facebook linking will now show the contents of the Details Property on this Thing. (This change is actually subject to change: the more I think about it, the more I think it should be showing Summary instead. I'll likely tweak that shortly.)

Finally, I'm still trying to get the icon right. Suffice it to say, FB's rules for displaying a logo aren't especially obvious, so don't be surprised if this continues to evolve.

So far, I think I've taken it from "it sucks" to "it doesn't suck *too* badly". There's going to be more experimentation, to get us to a point where it looks reasonably good. (Suggestions welcomed.)

Design a New Model now works conventionally: Alexx pointed out to me that, when creating stuff (which he mostly does on mobile devices), he likes to long-press on links, and open them in a new tab, but that didn't work for Design a Model, either in the menu or the button. On a desktop browser, you similarly couldn't right-click and get the expected menu. Both of these were because those buttons weren't implemented as normal links, but as special magic that happened when you clicked on them. (This was so we could pop the "choose a Model" dialog before navigating.)

Suffice it to say, that mechanism has been completely rewritten, so that the menu item and button are now normal links. You won't see much difference, except that the URL changes at a different time in the process, and you can now do the long-press/right-click as expected.

Fixed Computed Name: this is one of the more advanced meta-Properties, and should be used when you don't want to give explicit Names to each Instance, but instead compute the Name based on its Properties. Somewhere along the line, links to Things with Computed Names stopped working, due to some parsing issues. This is now fixed, and they should work as expected.

Fixed Notifications: (I think.) I recently noticed that Notifications had gotten flaky. I've fixed one bug that could be causing that, but I'm not 100% certain that it was the only problem. If you find that you're not getting Notifications that you should be, please tell me.

Fairly significant code changes under the hood today (which should make the underlying machinery of Spaces more stable going forward), but just a few bugfixes and enhancements.

Enhancements

Added the _emailLink Function: this function allows you to compose complex mailto: links -- basically, you can code up a button that, when pressed, will open a pre-filled-out email in your mailer of choice.

Note that Querki does *not* send emails on your behalf -- we might do that someday, but we have to be *very* careful about it in order to not get abused by spammers. But we can at least make it easy to have a Space full of contacts, and make it easier for you to send out letters to them.

Bug Fixes

Problems creating Spaces or Properties on Android devices: since the last release, it's been impossible to create a Space in Chrome for Mobile (which is what most Android devices use). If you tried to do so, nothing would happen when you tried typing the Space's name. This was a side-effect of the enhancement we made to prevent you from entering illegal characters into these names.

Suffice it to say, I blame Google for this one: Chrome for Mobile turns out to work differently from every other browser when it comes to keyboard input -- when I looked it up, I found scads of complaints about it, but they have declined to fix it. So I've essentially disabled that improvement for Mobile Chrome. We might eventually figure out a way to make this feature work on that browser, but for now it's back to the status quo ante.

Creating Spaces was often showing an error: this one bit me while giving a demo on Monday. Due to an internal bug relating to how messages get passed around between the servers, if you created a new Space, 2 out of 3 times nothing would apparently happen for a while, and then you'd get an internal-error message. The Space would be created successfully, but the message *acknowledging* that creation was getting lost. This is now fixed.

_linkButtons now open a new window when appropriate: Querki has moved to a standard where external links -- buttons or links that take you *out* of Querki -- open a new window. We've found that that's usually what you want, and it's the way most services work nowadays. So the _linkButton() function has been adjusted to match that.

A variety of tweaks and bugfixes this time around, mostly occasioned by observations from alexx_kay (you can get your name in lights, too, by reporting bugs!). In rough order from most to least important:


The big(ish) one: Typos in QL Expressions now raise errors. This requires some explanation, both to illustrate how things previously worked and how they now work.

The thing is, Querki was originally conceived as a wiki that you could run queries on. (The name is short for "Queryable Wiki".) The QL language was designed to be a more-powerful version of what you put in brackets in a typical wiki, so if you said [[My Page]], that would give you a link to that Page. And like a wiki, if you said that and "My Page" didn't exist yet, it would still produce a clickable link, so you could create it. Querki aped that functionality pretty exactly: if "My Page" didn't exist, it would be treated as a Tag -- you could click on it, and the Edit button on its page would actually create it.

The problem there is that, since *any* name could be treated as a valid Tag, there was no way in QL to detect typos. For example, in the error that led to me thinking about this, he typed __isEmpty, with two underscores instead of one, in an expression. A simple typo, but not an easy one to notice on the screen, and the results in the expression were bizarre and confusing -- there was no good way to realize what was wrong.

Add to that the fact that the originally-intended use case, of being able to easily write the name on the page first and then define it later, is now fairly unusual in Querki. The only place it was ever common was on the main page of the Space, to list the main sub-pages. But now that new Spaces list all of their Pages by default, you rarely need to do this any more.

So: I'm officially tweaking the semantics of QL. If a name appears in a QL expression, and it doesn't exist as *either* the name of an actual Thing or a Tag that is in use, we will now treat it as an error, and display it as such. This means that this sort of common typo now shows up as a nice clear error.

Just in case anybody *really* wants to use a not-yet-defined name in a QL expression, there is a workaround, the backtick syntax. You can say [[`My Page`]], in backticks, and it will *always* be treated as a Tag. This actually isn't new -- it's been around for a year or so, as the way to use a display name in QL that includes illegal characters -- but from here on I'm going to call this the Tag Literal syntax.


Related to that: QL errors are now better-contained. Heretofore, the entire page has basically been treated as one big QL expression; if you had an error in a QL expression *on* the page, all that would display would be that error, and the rest of the page wouldn't show at all. This made it harder to understand where the error was, since you lost the context. And frankly, it was ugly and annoying.

So things have now been tweaked to make more sense -- all of the QText will be rendered normally, and the QL error will be displayed in-place. I think this will make errors a little bit easier to understand and less ugly, and it means that you *can* write the referencing link first, and define it later; it'll just show as an error on the page for now.


Missing function parameters are now treated as an empty value. Alexx' observation was that he expected to be able to say

_if(something,,""false"")

and have that work as expected -- it would display "false" only if something was false, and wouldn't show anything if it was true. In fact, it was giving an Unexpected Error, and the page was failing to load, which is Querki for "something's broken here". (If you ever see Unexpected Error, please tell me: by definition, that's a bug.)

I really had never planned for this syntax, but it seems reasonable to want, and it provides a convenient way to say "this is empty". So this syntax is now officially sanctioned, and fully tested in the case of _if(). Please tell me if you come across places where it doesn't work as expected.


_textLength can now take a parameter. The error here was that Alexx was trying to do something like

_textLength(""Hello"")

and was getting mysterious results like 32. I confess, I spent quite a while scratching my head over this before realizing that _textLength doesn't *take* parameters -- the only legal way to use it is something like

""Hello"" -> _textLength

The desired approach was clearly reasonable, though, so I've made it so that the function can now take a parameter, and will use it if provided; it will use the received value if there is no parameter. I've also added a story, to be dealt with later, to raise an error if you provide a parameter to a function that doesn't accept one, which should make situations like this less mysterious.


While loading a Space, show the Space's name. A detail suggested by Aaron, to provide slightly better user feedback.


Up/down arrows now work more consistently. In a few places (notably, the Name field when creating a Property), if you pressed the up or down arrow keys, they would do nothing; normally, in a single-line field, they should go to the beginning or end of the line. This was a regression due to the recent change to prevent illegal Property names, and should now work correctly.


Finally, there were fixes to a couple of minor documentation bugs.

After months of waiting (heck, *years* of waiting), I've just cut Querki 2.1. From a technical perspective, this is the biggest release in Querki's history, so please bear with me -- there's a lot to talk about.

History

The most important change is that Querki now has full, built-in History -- from today on, all changes you make to Spaces are tracked, and will (eventually) be undoable. (Yes, in programmer terms this essentially means that everything in Querki is now fully version-controlled.)

In practice, there is one major new feature available to the Owner of a Space: View History. If you go to Actions -> Advanced... -> View History, you get the History View of your Space: a reverse-chronological list of all changes to the Space. This list is pretty crude right now, and will quickly get very long, since it is listing every piddling little change; in due course, we'll improve the UI to summarize these changes and let you drill down into them.

The *really* cool new feature is under that: View Space at Time. When you're looking at the History View, you can click on any change and it will show a "View Space" button. Click on that, and you are now looking at your Space as it looked immediately after that change. You can wander around the Things, Explore, Search, and so on, pretty much as it was at that moment. (You can *not* Edit anything -- it doesn't make sense to edit the past.)

This plugs what I've thought for a couple of years was Querki's biggest hole: I found myself slightly *nervous* about editing my Spaces, and moreso about somebody else editing them, because they might break something. Now, it's possible to go back and see what things looked like before any given change, and in the not too distant future it will become possible to undo any given change, or in extreme cases even revert the whole Space to the way it was at a previous time. In programmer terms, this means you can "code with courage", without needing to be too timid about losing data -- it is now nearly *impossible* to lose data.

There are several limitations on History at the moment:

• History starts today: for your existing Spaces, you will see the Space as having been "created" today. Basically, this is the point when we're starting to record everything.


• Only the Owner of the Space is allowed to View History. Frankly, from a security POV the History feature is kind of dangerous, so I want to tread carefully here. In the medium term all Managers will be able to View History, but I'm not sure how much further we can safely go. It may become a separate security permission.


• When viewing the Space at a particular time, we currently don't show Conversations. There's nothing intrinsically hard about that, but there's some extra work involved in showing the Conversations sync'ed to the same time.


• User Values are *not* recorded in the History yet -- they're one of Querki's most advanced features (even I have only used them a few times), and it's going to be a bunch of extra work to put them into the History. It'll happen, but it will be a while yet. But this doesn't affect most Spaces for anything other than Ratings and Reviews.

Note that this is a *huge* change to the underpinnings of Querki, and involved a rewrite of several of the most central parts. I've tested it pretty heavily, but this is the sort of time when one discovers unforeseen bugs. So please: take a look at your Spaces, and if anything looks newly broken, tell me *immediately*. Let's make sure everybody's Spaces are running smoothly.

For the programmers:
( Collapse )

Other Bugfixes and Changes

While I've been at it, I've been tweaking and fixing a bunch of other stuff. In rough order from most to least significant:

It is no longer possible to create illegal Property Names: (thanks to Alexx for keeping on me to deal with this) for the longest time, it's been altogether too easy to enter Names for Properties that weren't legal. Often, this would cause the Property-creation process to silently fail; in a few cases, it would actually create the Property with a Name that then caused errors. This UI is now much more vigorous about enforcing proper Names. As part of this, fixed the Link Name code so that you can't set a Link Name that will later cause errors.

Querki Web traffic is now gzip'ped: this shouldn't be visible to you in any way except that, by and large, response times are likely to be significantly faster, especially on mobile. (I thought I'd done this ages ago, but it turns out I hadn't gotten it right.)

Logging in while looking at a Space now reloads the page: this was an odd regression that I don't entirely understand, but I made some tweaks to make this work properly again.

The Name and Link Name Properties for Spaces are now always separate: for most Things, there is a flag you can set (which is usually turned on), that tells Querki to derive the Link Name for the Thing from its Name -- that is, the URL will depend on its Name. That's problematic for Spaces -- it's technically a bit challenging, and it means you can easily accidentally change the URLs for *everything* in the Space, which isn't a great idea. So for Spaces, Name and Link Name are now always separate: you should say explicitly if you want to change them.

You can now use mailto: URLs in QText links: these aren't as useful as you might wish, because doing a fully-fleshed-out mailto: is too much for the link syntax to handle, but you can at least pop an email window with the address set. I will likely be adding a function to display complete mailto: links shortly. (I need it for one of my use cases.)

Rating Stars now work correctly for _average: the _average function runs on Ratings and Reviews; as you'd expect, it shows the average value of the Ratings on this Thing. Problem is, the number-of-stars display couldn't cope with fractions, so if the average wasn't exactly an Integer, it wasn't displaying. This now works correctly, showing fractional stars when it should.

Adding an element to a newly-created List in a fresh window now works: this was a subtle edge-condition bug that could show up if you refreshed your browser window while you were in the Model Designer / Advanced Editor.

(This still shows as 2.0.4 on the site because I messed up slightly, but it's currently running.)

Two subtle improvements this time around:


Large pages now work again: as I alluded to last release, we've had a bug for a while (since we moved the servers to Amazon) that was preventing large pages (more than 128k internally) from displaying. This was a purely internal problem -- when the rendered page was being sent from the back-end node doing the rendering to the front-end web server, if it was larger than that size it was blowing a maximum-message-size limit. This is now fixed: when a page is more than 10k, we set up an internal pipe to stream the contents reliably from the back end to the front. The fix is pretty generalized: while the bug was only showing for page rendering, this fix should apply to all API calls (which is nearly everything) from here on out.

Mind, any page that big is going to be kind of slow, so I strongly recommend using techniques like _QLLink to break it down into sections that get displayed when you click on them. (Really, for everyone's good I *request* that you do so.) But there's no hard limit any more.


You can now open User Values to the Public: I got an interesting question the other day: "Can I set up Preferences for my Space?" That is, is there a straightforward way for individuals to set preferences for how the Space will display for them?

The answer is "yes": Querki has a very general concept of User Values. If you have a Property, and you set the "Is a User Value" meta-Property on it, it becomes a "User Value". This means that, when a user edits that Property, the value they set is specific to *that user*. When they look at something that uses that Property, their value is what gets used. (There are also features for listing all User Values for a given Thing, displaying histograms and averages of those values, and so on: it's the basis for the built-in Rating and Review Properties.) So creating Preferences is actually pretty easy: you define a Page with some Properties, and make all of those Properties into User Values.

But there was a hitch: this only worked for *members* of that particular Space. That was a fairly arbitrary decision on my part, a couple of years ago, because I didn't see a use case for allowing non-members to have User Values. But for a public Space that might have non-Members looking at it, being able to do this Preferences thing totally makes sense.

So I've tweaked it accordingly. Who is allowed to do this is controlled by the "Can Have User Values" permission, which you can find in the Security page. (This requires Programmer mode, since it's pretty advanced stuff.) This permission defaults to "Members", but now allows "Public" -- if you set it to Public, then any Querki User can set User Values on this Thing.

Note that this is still limited to signed-in Querki Users, and probably always will be -- even when we add moderation and begin to allow Anonymous users to contribute comments and Things (if you turn that on), it doesn't make much sense for Anonymous to set a preference. But now that we're in public beta, and anybody can quickly sign up for Querki, I don't think this is an unreasonable limitation.

Getting back to regular releases, just a few tweaks today:


Added the new _textLength function: exactly as the name implies, this receives anything, and produces the length of that value when rendered as Text. Normally I'd expect you to use this with Text values, but you can, for example, pass in the number 9387 and get a result of 4.


_groupBy should now work with the results of _search: more generally, _groupBy has been rewritten so that you should be able to pass *anything* into it. There's been a subtle functional change: _groupElements is no longer a Property of the _groupBy results, it's now a function that you run *on* those results. But I don't expect this change to make any difference in practice.


Tweaked the way that links get rendered: this is probably the most dangerous change -- suffice it to say, to make things more efficient, I've subtly changed how pointers to Things get rendered. This *shouldn't* make any major difference in practice, although it does mean that links to Things whose Names contain QText may get rendered a bit differently. Please poke around in your Spaces, and tell me if this has broken anything.


Also, while I was at it, I've fixed the sub-Pages in the Issue Tracking Space. The "List of Issues, by..." pages have all been broken for a while, because very large pages aren't currently working right. I've fixed these particular pages by making them use _QLLink, and I generally recommend that technique for any pages that are showing lots and lots of stuff, but the underlying bug is still there, and I'm working on it. (It's the proximate cause of the change to link rendering -- that change doesn't fix the problem, but potentially makes it a bit more fixable.)