« Back to home

Managing ordered child components in Ember and Glimmer

Posted on

In the quest for native quality HTML5 apps with Ember, I started building smoke-and-mirrors, the core concept of which was to build a set of highly performant reusable components for use in applications that needed to optimize rendering.

The primary feature is the occlusion-collection, a highly optimized alternative to listView or ember-cloaking. Interestingly, it's implementation details are fairly similar to ionic's new collection component, which in past iterations was more similar to listView. Ionic apparently reached some of the same conclusions on performance and native scrolling.

Pre-glimmer, the occlusion-collection got it's performance gains in four ways.

  • via the proxying behavior of MagicArray, which prevents the views in the collection from being torn down and thus allows ember to re-render only bindings, not entire views.
  • via occlusion, which improves rendering by removing or making invisible DOM elements that are off screen.
  • via a series of internal optimizations to the mechanics, including cacheing edge calculations and heights.
  • via preserving true DOM height by replacing the removed content with a single empty element with the correct height.

The benefits of this approach paid off quickly.

  • It was relatively trivial to add hooks for nearBottom and nearTop for allowing performant infinite scrolling.
  • Since the underlying array was an instance of ArrayProxy, we could use arrayContentDidChange to detect array prepends and allow seamless insertion above the fold and true bi-directional infinite scrolling without jumping or flickering (so long as requestAnimationFrame is supported).
  • We could easily (not fully implemented yet) allow you to start/preserve your scroll position at any point in the list (a benefit of supporting bi-directional infinite scrolling)
  • We could (not implemented at all yet) add aria support by utilizing the shells left behind for occluded content.

Most of this was only possible because we were using a ContainerView and intelligently controlling the rendering process. One of the main complaints I had with the system though, was I wanted {{#each}} semantics. Today, we have that. Below is the source code for the bi-directional infinite scroll demo for the about to be released 0.2.0-beta.3

<div class="table-wrapper">  
  {{#occlusion-collection
    content=model.images
    defaultHeight=350
    alwaysUseDefaultHeight=true
    containerSelector=".table-wrapper"
    bottomReached="loadBelow"
    topReached="loadAbove"
    keyForId="small"
    as |image|
    }}
      <div class="image-slide">
          {{async-image src=image.small}}
      </div>
  {{/occlusion-collection}}
</div>  

Unfortunately, getting each semantics meant ditching ContainerView for nested Components and losing granular control of the rendering process. Long term, with Glimmer landing, this wasn't something I minded so much. Short term, though, it turned out that the component approach wouldn't work cleanly with glimmer.

Specifically, I was relying on this._childViews to retrieve an ordered list of the child sub-components that that were rendered. (These components aren't the yielded part the user puts within {{#each}} but wrappers around the yield used to control rendering state, calculate height, etc.). The actual code for this was a little rough, some virtual views were present and the code looked like this:

getChildViews: function() {  
  var eachList = Ember.A(this._childViews[0]);
  var childViews = [];
  eachList.forEach(function(virtualView){
    childViews.push(virtualView._childViews[0]);
  });
  return childViews;
}

This worked great until I updated ember to 1.13.0-beta.1 to see how this all worked with glimmer.

And everything went insane.

And many tears and much gnashing of teeth was had.

And I wanted to rage quit and walk away.

... but I didn't.

One of the optimizations above turned out to be really bad with Glimmer, the MagicArray. This is alright, because that optimization was to coerce Ember into maintaining views and DOM instead of destroying them. Glimmer removes the need for DOM (and, much to my happy surprise, removes the need for a LOT of virtual views and wrapper views).

The MagicArray proxies not only the underlying array, but also each object within the array. Both of these proxies would result in change notifications leading to a new glimmer diff. This is apparently not expected behavior, and I'll be drawing up a jsbin to replicate the issue so the core team can patch it before 1.13 stable is released.

The main demo of smoke-and-mirrors is everyone's favorite dbmon, which consists of an each nested in an each. The each nesting meant that the change notification issue with proxies affected the demo collection with a 2X penalty, leading initially to 6X worse performance than just glimmer itself for a component that pre-glimmer out-glimmer'd glimmer. With the new component structure, I could change one array observer to a computed, this improved the problem to a 4X penalty imposed entirely by the no longer necessary proxy behavior.

So out with the proxy, right?

Well, out with the proxy would also mean out with the smooth bi-directional infinite scrolling. I didn't want to sacrifice that. I started mulling options and @rwjblue weighed in to suggest the didReceiveAttrs hook could be used to do the array prepend detection.

Awesome!

Now I just need an array of childViews to actually manage...

You see, the other change Glimmer introduced is that _childViews and childViews are now gone as properties of components. If I wanted to handle and manage the visibility / render state of individual items, I was going to need to find a way to register those sub-components.

A solution was quickly proffered: https://github.com/emberjs/ember.js/issues/11244

Again, the answer was to use the new Glimmer hooks to do the registering. But here's where that got problematic, on the surface, this approach works.

BUT

Any child component with visibility:hidden won't trigger willRender.

BUT

There's no hook that will register all generated sub-components AND guarantee that registration will happen in the proper order. Not with the way Glimmer works.

BUT

And I swear, this is the last one, but not only do I need to keep an updated registry of components, I need to do so efficiently, because this registry is being utilized at high frequency during scroll events.

So how do you efficiently register and unregister sub-components in an ordered fashion?

  • I considered various sorting algorithms (insertion sort for nearly sorted lists is great). I implemented this, but any sorting approach with large lists turned out to be ugly.
  • I played around with various hook combinations
  • I settled on using a hash

Instead of mimicking the _childViews array API, I utilize information I already force the developer to give me, keyForId. This is the same key being passed to the {{each}} in glimmer for glimmer's optimization work, and was used similarly by magicArray and in other places prior to glimmer.

  _children: null, // initialized to {}
  childForItem: function(item) {
    var val = get(item, this.get('keyForId'));
    return this.get('_children')[val];
  },
  register: function(child, key) {
    this.get('_children')[key] = child;
  },
  unregister: function(key) {
    this.get('_children')[key] = null; // don't delete, it leads to too much GC
  }

Instead of iterating the childViews array, I iterate the content array and call childForItem when I need access to it's view.

The mechanics of this are efficient, but when to call register and unregister?

The answer to that was again mostly the didReceiveAttrs hook on the sub-components.

  didReceiveAttrs: function(attrs) {
    var key = this.get('keyForId');
    var oldKeyVal = attrs.oldAttrs ? attrs.oldAttrs.content.value[key] : false;
    var newKeyVal = attrs.newAttrs.content.value[key];
    if (oldKeyVal && oldKeyVal !== newKeyVal) {
      this.collection.unregister(oldKeyVal);
      this.collection.register(this, newKeyVal);
    }
  },
  willDestroy: function() {
    var key = this.get('keyForId');
    var val = this.get('content.' + key);
    this.collection.unregister(val);
  },
  init: function() {
    var val = this.get('content.' + this.get('keyForId'));
    this.collection.register(this, val);
    this._super();
  }

This approach has worked fairly well, but glimmer support is STILL lacking. What's left?

Ugggh. This guy, present in both beta.1 and beta.2

What's frustrating about this guy is it definitely appears to be a race condition, and once it happens, the items affected will no longer render. At this point, I'm not doing anything to affect item itself in any of the code.

You provide content to the collection, which iterates over it with each supplying it to the yield you gave, while wrapping each item with a component to control whether that yield is visible or not.

item itself is not being changed, and my current worry is that this cryptic error from what appears to be extreme stream-sensitivity could become Ember's new achilles tendon.

In my case, the error turned out to be that the topReached action would synchronously add new data to the array (something the demo can do but most real world scenarios can't). The solution was just to async the sending of the action to prevent anyone else hitting the bug.

// this MUST be async or glimmer will freak   this._taskrunner.schedule('afterRender', this, this.sendAction, name, context);

ALMOST there.

Now, everything seems to be humming but some items never reveal? Turns out that rock solid child registration mechanism isn't so rock solid after all, since the keys we were using to identify the images weren't actually unique. #trolled