Delta is a real-time WebGL visualization of nytimes.com page view data. It runs in Chrome and is controlled with a Playstation 3 controller. Delta affords a high-level view of the real-time dynamics occuring on the website, allowing the user to see each page view as a individual transition between two states (from one page to another).
Each page view is represented as an individual dot. These dots appear when an event is created, typically with a human clicking on a link on nytimes.com or when they arrive at a page at nytimes.com. In the geographical view these events are placed at the latitude and longitude as to where they occurred. Events that have been recently created appear on the surface of the Earth, and as time moves forward, move into outer space. The time window for which events are displayed is 3 minutes.
In the site structure embedding each arc segment on the disc defines a path on the website (e.g. *.nytimes.com or nytimes.com/*). Individual pages are placed uniquely in each of these arc segments. Red dots indicate a transition from one nytimes.com page to another nytimes.com page while green dots indicate an offsite visitor (since the origin of these views cannot be embedded in the nytimes site structure they are illustrated as coming from above). Blue lines are routes between pages.
Each page view event is streamed from our event queue and processed with Streamtools. The time it takes for a single page view event to be created, processed, and to appear on Delta is usually between 120-180 seconds and is limited by the speed of upstream processing. Streamtools serves an anonymized and compressed version of the page view data to Delta over a WebSocket.
In the browser, Delta batches incoming page view data as chunks of non-Cartesian
vertex geometry where the coordinates are defined as attributes of that specific
event. For example, instead of
x, y, z, each vertex contains event creation
time, event latitude, event longitude, unique page id, and unique server path
id. Given these coordinates, a shader is used to embed the vertex in a various
set of spaces, either a geographical projection, or one that privileges the
structure of the website. This allows the freedom of tweening and transforming
between different embeddings as well as animating each individual event at very
little cost, affording hundreds of thousands of uniquely displaced vertices.
The animation of each vertex is dependant on the visualization time, which is determined by a rolling average of time at which each event was created. This time wanders a bit, trying to ease into the best suitable window. As the visualization time increases, the embeddings tween the vertex in the space between two points (the origin page and destination pages on the New York Times website). Each vertex effectively describes a line within each embedding.
- Delta on nytlabs.com
- The New York Times: The Research and Development Lab Talks ‘Delta’
- My colleague Mike Dewar’s “Seeing From Above” talk