Emerj.js: efficient HTML UI in 60 lines
Download Emerj.js here on GitHub.
Web-based software has moved fast in five years, from mostly-static apps where the server sends an HTML page with a light seasoning of JavaScript (think the oldest versions of Hotmail), to fully client-side apps that render the entire HTML in the browser on the fly, and just pull raw data from a server via a web API (think modern Facebook).
The challenges of coding web UI have shifted considerably. In the old days, you’d pull some data from the database and populate this into an HTML template on the server side, and send this as an entire web page to the browser. You could make some amazing apps this way. And when your audience failed to be impressed, you would add some JavaScript to the page to do cool things like make a sidebar slide in when you clicked the menu icon.
When AJAX hit the streets, things got cooler. The user could push a button to add something to their shopping cart, and it would just zip into the cart without even reloading the page. You’d write some JavaScript to bump the number of items on the top-right, and the shopper would be so impressed they’d add a few more just to watch the number change before their eyes. Your code would look something like this:
<body>
<div class='num-in-cart'>0</div>
<ul class='cart'></ul>
<button name=add value='product-42'>+</button>
</body>
--
var num_in_cart = 0;
$('button[name=add]').on('click', function() {
var button = this;
$.ajax('/cart/add', function() {
num_in_cart++;
$('.num-in-cart').text(num_in_cart);
$('.cart').append('<li>'+button.value+'</li>');
})
})
That was great, and websites started to feel more like software and less like pages, and we started calling them “apps”.
The trouble is, this way of doing dynamic UIs doesn’t scale. Note how most of the update function is DOM manipulation and ad-hoc HTML construction. And this is an overly simplistic example—I’ve left out loads of important edge cases and usability niceties. You end up writing your HTML twice (once server-side and once to render updates), and that innocent-looking $('.num-in-cart').text() will explode into a thousand strings of spaghetti if you have anything remotely non-trivial about your data.
An elegant solution to this is to use a client-side template language, like Handlebars or Nunjucks (my favourite). Your code might look more like this:
<script type=text/template name='cart.html'>
<div class='num-in-cart'>{{ cart|length }}</div>
<ul>{% for item in cart %}<li>{{ item }}</li>{% endfor %}</ul>
<button name=add value='product-42'>+</button>
</script>
--
var data = {cart: []};
$('button[name=add]').on('click', function() {
$.ajax('/cart/add', function() {
data.cart.push(button.value);
dispatchEvent(new Event('data:updated'));
})
})
addEventListener('data:updated', function() {
var html = nunjucks.render('script[name="cart.html"]', data);
document.body.innerHTML = html;
})
dispatchEvent(new Event('data:updated'));
It’s a bit more code for this tiny example, but the code complexity scales exponentially better: you can have an arbitrarily complex a data structure, and as fancy a DOM as you like, and you only need to write your HTML once, cleanly, in a language much more suited to creating complex HTML than JavaScript is.
But wait.
This works ok. But we have a problem: every single element is replaced with an entirely new one, whether it’s changed or not. This is majorly problematic for two reasons:
- It performs badly. DOM rendering is among the slowest things a web browser does, so a full re-rerender on every update is bad. But it’s still waay faster than a page load, so this is a net win over server-side rendering.
- More importantly, you completely lose any element state. If the user had scrolled partway down a page, or typed some input, selected an option, or whatever, that disappears completely every time you re-render. Super annoying, and utterly non-functional.
You can avoid the second problem to an arbitrary extent by breaking your templates into smaller sub-templates and only updating portions of the page. But this also scales pretty badly, quickly becoming as crazy as the problem you’re trying to avoid.
The basic solutions
Rendering data seamlessly into your UI without a whole bunch of ad-hoc code, is (I believe) the basic problem that has driven the proliferation of frameworks and UI engines in the last few years, like Angular, React, and Ractive.js. It’s the trickiest part of writing a vanilla HTML/JavaScript UI. And some clever solutions have been uncovered.
Roughly speaking, they fall into these categories:
- Don’t bother. The web was never meant to work like this.
- DOM manipulation. So 2000s, with all problems as above.
- Use templating, and obliterate your document state every render.
- Use real HTML with extra semantic attributes to help bind your data directly to the live DOM, like plates.js or pure.js. All the above problems just take care of themselves. At first glance, this seems like the One True Way to do dynamic JavaScript UIs, because you’re using the tools of the web. But all avenues lead (very) quickly to suffering, as you can see from a quick scroll beyond the simplest examples in the plates README. Either your data model, your code, or your HTML will hurt.
- A component-aware system that selectively updates only changed elements, something like Angular and friends.
- A component-aware system that doesn’t bother keeping track of which components need updating. Just update them all, compare the resulting structure with the current live document, and only update the differences. This is what React and Ractive do, if I may oversimplify somewhat.
- Component-aware? Who needs it. Keep reading.
Reacting to the problem
Facebook’s React and the React-inspired Ractive.js are my picks for the best of the bunch. For one thing, they just focus on rendering UI. They’re libraries, not kitchen-sink frameworks, and I believe that’s a Good Thing.
The React folks hit on a great solution to the problem of replacing your entire DOM—instead of overwriting it, React renders to a “virtual DOM” and diffs it against the current virtual DOM, updating only those parts that differ. They describe how this works in their reconciliation algorithm.
To write the UI code in the first place, React provides a custom language called JSX, which I don’t particularly care for (it reminds me a lot of the bad old days before Unobtrusive JavaScript when we mixed PHP and JavaScript into our HTML with mad abandon). No, I don’t want to hear about how I can use native JavaScript instead of JSX if I want.
I like Ractive.js a bit better, but it still leaves me a little uninspired. I also maintain a lot of web UI code written in Jinja/Nunjucks, and it’s infeasible to just convert it all to Handlebars or JSX. I’d rather not be forced to make a choice of template language if I don’t have to.
The emerjent solution
And there’s the thing: I don’t have to. The concept can be made language-agnostic, by accepting anything that spits out a usable document tree. React itself probably could never be this, because they’re invested in their component architecture, but after realising it could be done, it wasn’t hard to implement. Every browser has an HTML parser built in, and this great virtual document model called … you guessed it … the DOM. If you create an element and don’t insert it into your document, then it’s virtual. Not real.
Let’s quickly get one myth out of the way: the DOM is not slow (you are). This is a common misperception, and one I believe was partly behind React’s choice to build their own. What’s behind the myth is that DOM rendering is slow. (and even that’s not slow, for what it does, if you’re careful).
But if your DOM root is not connected to the live document, then it’s super fast. Yes, you could build something faster with lightweight vanilla JavaScript objects, but, depending on your needs, likely not enough faster to be worth the effort.
So that’s pretty much how Emerj.js works. You render some HTML or build an out-of-document DOM however you like (Emerj doesn’t even need to know you exist at this stage), and then call Emerj’s single function to update your DOM. Here’s what the function does:
- Converts your HTML to a “virtual” DOM, if you haven’t already. This is easy:
var vdom = document.createElement('div'); vdom.innerHTML = html; - Loops through the virtual DOM’s immediate children, and compares each one to the live DOM’s immediate child at the same position (or with the same ID).
- For each node, if it differs, updates it with fairly simple logic, mostly borrowed from React: if the tag name has changed, consider it a completely new element and replace it; if it’s a text node, update it if it differs; remove any missing attributes; add any new attributes; update any altered attributes
- Recurses into any children from step 2.
- Removes or adds any missing or extra children.
Your DOMs are now identical, and you’ve only modified the bare minimum.
Hey, you’re cheating!
“You’re cheating. React gives you a way to create HTML, composable components, etc, and Emerj just hands the problem off to a template language. That’s at least 50kb right there, and you haven’t really solved anything.”
Yeah. I’d like to quibble that React’s way of creating HTML isn’t all that exciting, and string concatenation doesn’t look too much worse than JSX. But I’m speaking from the sidelines: I’m not a React convert.
However, I’m not really trying to pitch Emerj as solving all the same problems that React solves. It doesn’t. React has loads more features out-of-the-box, is way more industry-tested, and provides a broader set of UI concepts and philosophies to build a UI around.
I’m also grateful to React for being a sensible solution to a single real problem in a world where everything else is a kitchen-sink framework.
Furthermore, Emerj wouldn’t exist without React. React is what triggered the idea to start with.
All I’m saying is, for me, and I hope others, Emerj addresses the basic problem that would drive me to React in the first place: updating an HTML UI from data on-the-fly, efficiently, without zapping document state, and without a whole bunch of nasty ad-hoc DOM manipulation. As for the rest of React, I’ve either found other solutions I’m happy with, or I’ve just never run into the problem that particular piece solves.
Performance
I’ve tested Emerj in the major browsers, and it performs really well for the job it does.
Specifically, the template in demo.html can be rendered more than 50 times per second in all browsers, on an ordinary modern laptop CPU, with all data fields updated each frame.
For another comparison, I implemented a (very) basic ToDo app in both React (todo-react.html) and Emerj (todo-emerj.html), and compared the times for adding items to the list in each:
| React | Emerj+Nunjucks | Emerj+Nunjucks Using requestAnimationFrame to avoid stacking up unnecessary re-renders | |
|---|---|---|---|
| 1500 items added (full re-render each iteration): | 76s | 86s | 45s |
| 500 items: | 12.2s | 12.5s | 6.5s |
| 100 items: | 1.08s | 1.1s | 0.54s |
| 50 items: | 0.5s | 0.36s | 0.27s |
| 10 items: | 0.11 | 0.07s | 0.04s |
| 10 items added to a 10000-item list: | 6.7s | 8.9s | 3.3s |
(Please contact me if you see gaping holes in these comparisons, or get dramatically different results.)
In many scenarios, that’s quick enough to do animation with, though I don’t really recommend using Emerj for animation. CSS transitions are simpler and better, and direct DOM manipulation may be a better idea than trying to trick Emerj into animating an element by animating your data (a dubious idea at best).
As you can see, Emerj+Nunjucks (without requestAnimationFrame) is faster than React for small DOMs, but slows down somewhat for large DOMs. In actual fact, most of the time is spent parsing the resulting HTML into a DOM using .innerHTML. If you use a different, parse-free method of constructing a DOM tree, you could dramatically improve the speed on large complex documents.
Note the big gain from using requestAnimationFrame — clearly a good idea. The effect of using requestAnimationFrame is simply that the render code is not called more than once per frame. So of course this is faster: it’s doing less. Which is the only way to make code faster, anyway. I’m not sure whether React uses requestAnimationFrame internally (I would hope so, these days), so this column may not be apples-to-apples.
Again, the real advantage is not so much performance, but that state & identity of existing elements is preserved – text typed into an <input>, an open <select> dropdown, scroll position, ad-hoc attached events, canvas paint, etc, are preserved as long as an element remains, and that Emerj provides a simple way to make this happen. Never make your code more complicated to solve a performance problem you don’t have!
Shortfalls and improvements
There are a few minor pitfalls with this model, some of which React also has, but none of which are show-stoppers.
First, third-party or non-emerjent code that manipulates the on-page DOM will interfere with Emerj — any changes made will get overwritten in the next render. The ideal solution is to use Emerj for everything, but that’s not always feasible or even right. I plan to introduce two solutions for this, but need to spend time testing them in real life:
- an “emerj:ignore” attribute on the element, causing Emerj to skip updating the element, and
- an option to compare the virtual DOM with the previous virtual DOM, and only updating the live DOM where the two virtual DOMs differ.
Second, Emerj makes no attempt to solve the inverse problem: updating your data model when on-page widgets are changed (eg, the user types into an input field). Ractive does this. React does not. Kitchen-sink frameworks like Angular do. I believe it’s a separate, though admittedly closely-related, problem, that should be solved separately. And, if you use delegated events (if you’re not, you should be), the vanilla JavaScript way of doing it is not unpleasant.
Third, if you use Emerj with a template language (my preference), as opposed to some DOM-based component architecture (I don’t know any; do you?), the very minimum that needs to happen in a render is to parse the HTML into a DOM structure (and then loop through that DOM structure, but React must do this too). Hopefully .innerHTML is highly-optimised compiled C code, given it’s what web browsers do for a living, so it’s not terrible, but it’s certainly not free, either. React doesn’t have this problem, since it deals in objects, not text. Note this is not a limitation of Emerj as much as it is of whatever method you use to produce your DOM.
Fourth, there’s not much to help you with really complex components that have zillions of sub-elements or for some other reason are particularly slow to construct. React provides shouldComponentUpdate() for this purpose – if you know that a component doesn’t need to rerender, then save your cycles. However, Emerj has no way of doing this, because it doesn’t know anything about your components or DOM until after you’ve rendered. Emerj’s take on this is that it’s your problem. But there are relatively simple solutions. If you’re using a template library like Nunjucks, a simple cache-on-state tag might do the trick.
Fifth, my next reusable component will be an <ol> with English text for bullets. Seriously, React provides a pretty good attempt at a composable component architecture, something that, if you can figure out what those words mean, any good UI library should do. If you have a fancy date widget, but need one with a year selector, just subclass it. The sky’s the limit! Emerj doesn’t provide anything like this. That said, it also doesn’t need to: if you have a semi-decent way of producing HTML, part of that is bound to involve reusability in one form or another. My favourite, Nunjucks, has macros, which make for excellent reusable components. If you wanted, you could also do some pretty powerful things using native DOM instead rendering an HTML string.
Worthy mentions
Some good reads along these lines:
- JavaScript Fatigue by Eric Clemmons
- The React reconciliation algorithm
- The DOM isn’t slow, you are by Kory Nunn
- Nanodom and morphdom, a couple projects in the same vein as Emerj
- Preact, a greener React.
If you like the concept, then grab yourself a copy of Emerj here and start coding!
22 November 2017 by Bryan 2 comments
2 comments (oldest first)
That’s cool. I’d be interested to see your code if you ever run across it again.
I haven’t run into issues with classes or types or tables, though. I could imagine the implicit tbody potentially causing problems if you generate a DOM structure yourself without one, but in my case this doesn’t matter because I use the browser to parse my HTML into a DOM, and it’ll add a tbody.
Part of the purpose of Emerj is to actually avoid overwriting the current .value of inputs — the idea is to preserve current element state that isn’t represented explicitly in the DOM, so you don’t delete something the user has just typed in (or <select>ed or whatever).
Nanomorph has code to update live input values from the value= attribute, but that’s actually something I’ve deliberately avoided for the above reason.
How do you deal with elements that act weird? Last time I implemented similar, I ran into elements or attributes that were “read-only” in some browsers (some attrs e.g.
<input type=, some form elements,<title>, something to do with tables – can’t remember what else) but not others.For those, the solution ended up being to recreate a new element with the desired properties, and swap them out in the DOM.
Have you seen anything like this?