Streaming an Ajax response with Vue.js and Server-sent events (SSE)
The problem: We want to display a large number of search results (from our DAM system) on a Web page. Gathering the results on the server and transferring them to the browser takes a while. To improve the user experience and show the first results as soon as possible, we want to “stream” the results. Each item needs to be rendered as soon as the browser receives it. A simple Ajax call waits until the server has returned everything, so we’ll have to do something a little more advanced.
The solution outlined here combines two technologies I currently enjoy experimenting with: the Vue.js JavaScript framework (a competitor of React and Angular) and Server-sent events (SSE), a lightweight, W3C-standardized alternative to WebSockets.
In our old UI, we used Oboe.js for streaming, but I like the new approach much better because it requires little code, thanks to the magic of Vue.js and EventSource (the Web browser’s built-in SSE support, not available in IE and Microsoft Edge) – and because it’s very lightweight, requiring nothing but a simple script include (no npm, no build toolchain).
The end result looks like this, you can try it live here:
Let’s see how this works:
Server-sent events use a simple text protocol. Here’s what my server sends; pretty self-explanatory (note the text/event-stream content type):
$ curl -v http://example.com/stream.php
> GET /stream.php HTTP/1.1
>
< HTTP/1.1 200 OK
< Content-Type: text/event-stream; charset=UTF-8
<
event:header
data:{"total_items":20,"msg":"Hello from server"}
event:item
data:{"cnt":1,"text":"[1] Hello from server"}
event:item
data:{"cnt":2,"text":"[2] Hello from server"}
[…]
event:item
data:{"cnt":20,"text":"[20] Hello from server"}
event:close
data:[]
This format is transparently handled by the EventSource object. We just need it to feed data into a Vue.js model. Vue.js updates the HTML automatically when the model changes. (Check out the Vue.js documentation.)
It’s important to know that EventSource is designed for permanent connections so it keeps requesting the same URL in an endless loop when the server closes the connection. The evtSource.close() call prevents that, stopping after the server has signalled that all results have been delivered.
It’s all in a single HTML file, stream.html:
<html>
<head>
<meta charset="UTF-8"/>
</head>
<body>
<!-- This div is rendered by Vue.js: -->
<div id="app">
<form v-on:submit.prevent="run">
<input v-model="msg" type="text" placeholder="Enter message"/>
<button v-on:click="run" type="button">{{ buttonLabel }}</button>
</form>
<p>{{ items.length }} of {{ total_items }} times “{{ actual_msg }}”:</p>
<ul>
<my-item v-for="item in items" :key="item.cnt" v-bind:item="item">
</my-item>
</ul>
</div>
<!-- Include Vue.js: -->
<script src="https://unpkg.com/vue/dist/vue.min.js"></script>
<!-- Our own JavaScript code: -->
<script type="application/javascript">
Vue.component('my-item', {
props: ['item'],
template: '<li>{{ item.text }}</li>'
});
var evtSource = false;
var app = new Vue({
el: '#app',
data: {
msg: 'Hello world',
actual_msg: '',
total_items: -1,
items: [],
loading: false
},
computed: {
buttonLabel: function() {
return (this.loading ? 'Loading…' : 'Go');
}
},
methods: {
run: function() {
this.reset();
var streamUrl = 'stream.php'
+ '?msg=' + encodeURIComponent(this.msg);
evtSource = new EventSource(streamUrl);
this.loading = true;
var that = this;
evtSource.addEventListener('header', function (e) {
var header = JSON.parse(e.data);
that.total_items = header.total_items;
that.actual_msg = header.msg;
}, false);
evtSource.addEventListener('item', function (e) {
var item = JSON.parse(e.data);
that.items.push(item);
}, false);
evtSource.addEventListener('close', function (e) {
evtSource.close();
that.loading = false;
}, false);
},
reset: function() {
if (evtSource !== false) {
evtSource.close();
}
this.loading = false;
this.items = [];
this.total_items = -1;
}
}
})
</script>
</body>
</html>
For completeness, the PHP code stream.php producing the above output, which is not that interesting:
<?php
// Parameters
$repeat = 20;
$msg = filter_input
(
INPUT_GET,
'msg',
FILTER_SANITIZE_FULL_SPECIAL_CHARS,
['options' => ['default' => '']]
);
if ($msg === '') {
$msg = 'Hello from server';
}
// Make sure output buffering is turned off
@ob_end_flush();
// Header
header('Content-Type: text/event-stream; charset=UTF-8');
echo dataToStreamEvent
(
'header',
['total_items' => $repeat, 'msg' => $msg]
);
flush();
// Message times repeat
for ($cnt = 1; $cnt <= $repeat; $cnt++) {
usleep(50000);
echo dataToStreamEvent
(
'item',
['cnt' => $cnt, 'text' => sprintf('[%s] %s', $cnt, $msg)]
);
flush();
}
// Tell client to close the connection
echo dataToStreamEvent('close', []);
flush();
/**
* @param string $type
* @param array $data
* @param string $id
* @return string text/event-stream formatted string
*/
function dataToStreamEvent($type, array $data, $id = '')
{
$result = '';
if ($type !== '') {
$result .= sprintf("event:%s\n", strtr($type, ["\r" => ' ', "\n" => ' ']));
}
if ($id !== '') {
$result .= sprintf("id:%s\n", strtr($id, ["\r" => ' ', "\n" => ' ']));
}
$result .= "data:" . strtr(json_encode($data), ["\r\n" => "\n", "\r" => "\n", "\n" => "\ndata:"]);
$result .= "\n\n";
return $result;
}
What do you think of this approach? Let me know on Twitter or by e-mail.
(To keep the examples short, I intentionally omitted error handling and documentation. Don’t use it in production like this!)
Update (2017-06-03): See also Chris Blackwell describing his solution for Server Sent Events using Laravel and Vue.