Previous Entry Share Next Entry
Release 0.6.9 -- _groupBy
jducoeur wrote in querki_project
This is going to be a fairly quiet week on the Querki front. I'm not getting a ton done (between one thing and another, I'm only really working two days this week), and what I am doing is mainly on Conversations (which are coming along, but it requires a remarkable amount of code to get to the first useful milestone). But I did fix a few bugs and add one significant feature last week, so let's make those available.

The big new feature is the _groupBy() Function. On the one hand, this is a bit of a power-user Function; OTOH, it is ferociously useful, sometimes indispensable. I finally dealt with it because it was the only way to make the Issues by Submitter page work the way I've always wanted it to. That has been rankling me for months, and now that we have Model Types it is now possible to do it right.

The syntax of _groupBy() looks like this:
[[LIST OF THINGS -> _groupBy(EXP) -> GROUPS]]
That is, it receives a list of Things (this should eventually become more general, but it's going to take some interesting additional programming to do so). It applies EXP (which will most often just be a Property name, but can be any consistent expression) to each Thing, to get a key. Then it groups the Things by those Keys, and produces a List of Groups. A Group is technically a Model with two Properties: the _groupKey is the generated key from each one (eg, the Property value that these have in common), and the _groupMembers are the Things that had that key. The key may be of any type; it should, in theory, work correctly for all of them.

So for example, the Issues by Submitter display (slightly simplified) is now:
[[Issue._instances ->
  _groupBy(Reported By) ->
  ""### [[_groupKey]] [[_groupMembers -> Summary]]""]]
That is, we take all the Issues, group them by who reported each one, and then display a section for each grouping. Clean, simple, and does pretty much what we want.

IMPORTANT: this is very much a first draft, so use this with a bit of caution for now. I expect it will need some tweaking when we enhance it to cope with any received List. (The only reason it can't do so yet is the question of what Type the resulting _groupMembers Property has -- we're basically getting into generics here, and I don't yet have a mechanism for that.) Now that I look at it again, I may well change _groupMembers to _groupElements sometime soon. The documentation for _groupBy is currently broken -- that is fixed, but not yet released.

In the medium term, I may rethink the return values here. Conceptually, _groupBy returns a Map, and I'm pondering whether it is worth reifying a formal Map Collection for that. That may happen down the road, but it would be a very big deal.

So feel free to play with _groupBy, but don't go hog-wild with it quite yet -- keep track of where you are using it, since breaking changes may yet occur. I expect those to be purely mechanical changes if so, but they may require a little tending. That said, I expect this to grow into one of Querki's more important features: it was the most important element of SQL that didn't yet have a corresponding feature in Querki, and I'm happy to have it working.

I also added a less crucial but quite useful little Function: _resolveTags(). This has the syntax:
[[TAGS -> _resolveTags -> LINKS]]
That is, it receives any number of Tags, figures out which of them are the names of actual Things, and returns a List of Links to those Things. It should be useful in a number of cases where you have a Tag Set Property with a Link Model, and you want to make the Model more interesting.

Fixed the documentation for _join: this was just a dumb bug, which illustrated that I need to be careful about internal name collisions -- the URL for the documentation for the _join command was the same as the URL to join the Documentation Space.

_tagsForProperty now works correctly with new Tag Sets: it turned out that the _tagsForProperty Function (which returns all of the Tags defined on a given Property, and is used on the front page of a typical Space) was returning subtly broken results, such that clicking on one of those Tags didn't get quite the correct URL. That now works right.

Back to working on Conversations. Expect the first release of those next week -- I'm looking forward to it!

  • 1
Mmm. _groupBy will be very useful. I'm always having to sort data (in Excel, MySQL, whatever) by some criteria, occasionally then filtering for uniqueness. I think any system that is managing data like this should have the capability to group/sort baked in.

It definitely feels like things are getting towards the late stages of development. All the big rocks are in, and most of the small, and now it is putting in the sand. How do you feel the progress is going?

Slower than I like, actually, although about as expected. Truth is, several of the *really* big and important rocks aren't even started yet. The really big projects coming in the next year or so include:

* Conversations -- being started now, to make Querki truly social and interactive.
* Notifications -- being able to send and receive messages, both for the system and for users, will probably be started as part of Conversations, but there is a very long ways to go with that.
* Personal Properties -- coming shortly after Conversations, we introduce the notion of Properties that exist on a per-User basis. (Initially, so that we can support the notion of individual Ratings and Reviews of Things.)
* History -- under the hood, I want Querki to be Github-like: you should be able to not only see all changes and roll them back, but actually see what your Space looked like at a particular time.
* Apps -- in a few months, we start moving towards being able to share "code". In practice, I expect most Querki users to mainly work with Apps, instead of building their own Spaces from scratch.
* Mobile -- while the web-based Querki UI is decent for mobile, there are a lot of features (eg, photograph download, offline working) that will require true Mobile Apps.
* Moderation -- a central precept in Querki, not just for comments but for everything in the Space, so that non-Members can contribute.
* Social Network Integration -- so that you can share stuff with your friends. This is actually *quite* central to the business plan: Querki is fundamentally viral in design.
* and the crude business of making money -- Advertising and Membership.

And after that we probably get into things like Crowdsourcing (which I believe is going to ultimately be very important, but crucially different from how it works in other systems), App Monetization (if the legal and practical sides work, I have hopes of essentially sharing a portion of Querki's income with App authors, so that you can instantly and automatically monetize your Apps), and all sorts of other stuff.

So basically, you're only seeing the tip of the iceberg at this point -- the most *central* stuff is done, but a great application consists of far more than just the core. The initial features are approaching Beta, but they're all really focused on power users -- if you're not quite technically hip, you're not going to want to use it. Now we're beginning to flesh out the surrounding features that will make the system usable, as well as the ones that will make it more than just a programmer's toybox...

Ah yes, I meant in terms of the underlying functionality. Much of these look to be implementations on top of the underlying base.

But still, good to see a glimpse of the larger roadmap.

Even on the underlying stuff, there's a lot left to do. For example, we don't yet have the concept of name bindings (our version of "variables") in QL yet, much less the ability to post-process input before you store it. I'm pleasantly surprised we've been able to get this far without them -- it does suggest I've gotten the priorities right -- but I'm sure that we will require them before long, and those are pretty big. (I'm honestly not even sure yet what the syntax for post-processing is going to look like: it's a tricky problem.)

That said, it's true that the internals are settling down. I'm gratified at how few breaking changes I've had to make in the past six months. (I believe only one, really: the switch from being Name-centric to Display Name-centric, and the consequent switch from the old to new Tag Set Types. Even that wasn't quite "breaking", but did result in deprecating a lot of Properties in favor of the new world.)

  • 1

Log in

No account? Create an account