Design draft: Annotation prop input enhancements

[fcollman]

This is an design attempt to reconcile the functionality of #650 (which was an update to #136 ) with the new structured annotation schema introduced by #809.

This PR is a functional design PR, where'd I think it would be good to focus on the UI design and debate the conceptual implementation, independent of the actual implementation before we worry too much about the code.

The use case for this design is that a user wants to rapidly iterate through a set of annotations/locations and annotate that they belong to a certain class. Clicking into the annotation property box to label each class is un-ergonomic.

To accomplish this, this PR proposes adding tool keybinds to the boolean and enum fields of the schema tab.

So that users can quickly cause the selected annotation to change the enum value to the bound hotkey or toggle the boolean value.

The PR (like #650) adds hotkey binds of alt-up and alt-down to allow users to navigate the list.

Relatedly, once a user has tagged a bunch of annotations with this metadata, they often want to review a subset of them. For example, they may tagged a subset with "uncertain", or simply want to show a colleague a subset of examples that they found that were put into some relatively rare class.

This is related to a more general need to be able to see the properties in the list view, and be able to sort the list by those properties, similar to how the segment properties currently works.

To accomplish this the PR lets users add numerical properties to the annotation list view. It also provides up and down arrow buttons to let users sort the list. This doesn't actually change the order of annotations in the state, so that unsorting all columns leaves the list in its original order. The up/down arrow actions move the selection in the currently displayed list and not the order of annotations in the state.

I think this follows the spirit of the UI where the description field shows up in the annotation list below the annotation, so that users can find the annotations that they left particular notes on. With the structured schema now incorporated, users need a way to find those annotations.

The workflow setup in #650 would be accomplished by adding a boolean property for each tag that you wanted to enter, and assigning each a keybind. This is more general in that you could either do that, or implement a "forced-choice" workflow with an enum, such that selecting one category implicitly deselects the other categories, depending on what is appropriate for your use-case.

I thought about, but didn't add, a more general query box like in segment properties that would allow users to search/filter through annotations by combining filters on columns, but that is something we could debate.

I have not considered deeply how to reconcile this with other non-local annotation sources which might be dynamically loading annotations into view. We currently don't show any list of annotations in that case, or way to visualize their properties. Large annotations lists will be dynamically adding/removing data from the view and the list will almost never be authoritative or complete.

[syamace-metacell]

@fcollman, from a design perspective (taking into context #1048 as well)

Adding keybinds to annotations

• The keybinds works well for rapid annotation and it aligns with existing patterns on Neuroglancer.

Adding annotation properties in the annotation list:

• Adding annotation properties in the list makes sense, as well especially with the ability to sort/filter.
• Even though we’re introducing new patterns with the ability to show/hide certain properties in the list, this would be useful and needed if we were to display annotation properties in the list.

This PR's design approach is simple and nice, with some additions (stated below), it would target the use cases while keeping the introduction of new patterns to the minimal. This version make it seem like it's just a small addition on top of exisitng patterns.

These are additions I suggest to make it more aligned with the rest of the UI:

1. Query box (like seg. & skeleton tabs)
2. On hover of each annotation row, filter icons appears for each of the value (keeping the similar pattern on seg. tab for 'filter by label')

Sort minor UI feedback

• Sort UI needs to be aligned with what's on seg. tab with the use of 1 arrow, instead of 2.

[chrisj]

minimal example

shift+A toggles proofread status
shift+R/G/B toggles color

[fcollman]

1. Query box (like seg. & skeleton tabs)
   What would the query box implement? the same thing as i have in Design draft: Alternative annotation list interface #1048 ? a string representation of a query/sort filter?

2. On hover of each annotation row, filter icons appears for each of the value (keeping the similar pattern on seg. tab for 'filter by label')
   what would you do for numerical values?

I see from your comment on #1048 that you might be in a similar conflicted place to where i was.. where i felt like not having numerical filters was too much of a concern, and then i wanted to align the UI as close as possible to the segment properties.... but yeah in the end it's a big change and makes a fairly complicated UI.

In terms of sequencing, it seems like maybe this would be a

1. keybinds on enums/booleans is uncontroversial and everyone so far likes the design that is in these two PRs. This can go in as it's own piece i think (along with hotkeys to move up/down the annotation list).

2. We need to have the ability to list and sort annotation properties in the table view, with some mechanism to add/remove what properties are listed there. We need to align on the json representation of what properties are shown there and what the sorting mechanism is. It seems like the display and sort part of the design people like (with the exception that the sorting buttons have to change to a single arrow tri-state mode like i have in Design draft: Alternative annotation list interface #1048 )

If we pre-align around the idea that there is a query box and a string, and that is what goes into the json state and is parsed into setting the UI, and conversely changing things in the UI edits that string, then we could punt on what the UI is for setting all aspects of that query string. For example, the UI doesn't need to allow people to set numerical filters immediately, or necessarily set filters on enum properties immediately, or we could decide to change it.

We could put the query box into this PR, so that people would have the functionality and we would have the code infrastructure to inter-operate with it and the annotation list.

3. we could add enhancements to allow UI setting of the query string in a way that is visually aligned between segment properties and the skeleton tab, including in/out filters on categorical variables and numerical filters.

4. we can add more default annotation properties to annotations (length, volume, duration, delta_{dim}, extent_{dim}) that are derived dynamically from the annotation data rather then written explicitly by the user.

[syamace-metacell]

1. Query box (like seg. & skeleton tabs)
   What would the query box implement? the same thing as i have in Design draft: Alternative annotation list interface #1048 ? a string representation of a query/sort filter?

Yes exactly. Whats on #1048 is good, it introduces some new type of queries like the boolean and strings of the numerical filters but don't think it'll hurt to add them.

2. On hover of each annotation row, filter icons appears for each of the value (keeping the similar pattern on seg. tab for 'filter by label')
   what would you do for numerical values?

Right now, the same for simplicity, filter icon will only filter by that 1 numerical value. For numerical range, it would have to be done through the query box.

[seankmartin]

Thanks for making this! Overall I agree the most with:

If we pre-align around the idea that there is a query box and a string, and that is what goes into the json state and is parsed into setting the UI, and conversely changing things in the UI edits that string, then we could punt on what the UI is for setting all aspects of that query string. For example, the UI doesn't need to allow people to set numerical filters immediately, or necessarily set filters on enum properties immediately, or we could decide to change it.

I would need to think on it a bit more for exacts. But it feels a little bit similar to a leading drive in the schema UI creation. Neuroglancer annotation properties are the source of truth. The UI can do whatever it likes so to speak, but it can only ever make changes through that property API. Here, if we can manage to align on the core state representation of these annotation properties usage improvements then we can be a bit looser with the UI because it could always update in the future without us worrying about supporting old state representations and hopefully without needing to change the state representation.