DSCI 554 lecture 11

3D data visualization tools

Dr. Luciano Nocera

Outline

Overview
Canvas
WebGL
three.js
Processing, Processing.js, P5.js
Vtk.js
Mapbox GL & Deck.gl

Visualization tools

Adapted from [Heer 2014]

Outline

Overview
Canvas
WebGL
three.js
Processing, Processing.js, P5.js
Vtk.js
Mapbox GL & Deck.gl

Canvas

A raster (pixels) surface with two interfaces:

Canvas API for drawing graphics via JavaScript and the HTML <canvas>


								<canvas id="canvas" width="150" height="150"></canvas>

								<script type="application/javascript">
									var canvas = document.getElementById("canvas");				
									if (canvas.getContext) {
										var ctx = canvas.getContext('2d');  //'2d' provides the Canvas API
										//use Canvas API to draw
									}
								</script>

WebGL API for rendering high-performance interactive 3D and 2D graphics


								<canvas id="glCanvas" width="100" height="100"></canvas>
								<script type="application/javascript">
									main();
									function main() {
										const canvas = document.querySelector("#glCanvas");
										const gl = canvas.getContext("webgl");  //initialize GL context
										//use WebGL API to draw
									}
								</script>

Canvas API example


					<canvas style="background-color: orangered;" width="200" height="200"></canvas>
					<canvas id="canvas" width="150" height="150"></canvas>
					<script type="application/javascript">
					  var canvas = document.getElementById("canvas");
					  if (canvas.getContext) {
					    var ctx = canvas.getContext('2d'); //initialize 2d context

					    ctx.fillStyle = 'rgb(200, 0, 0)';
					    ctx.fillRect(10, 10, 50, 50);

					    ctx.fillStyle = 'rgba(0, 0, 200, 0.5)';
					    ctx.fillRect(30, 30, 50, 50);
					  }
					</script>

See MDN Canvas API and Mike Bostock's World Tour observable with D3.

D3 with Canvas API example


					map = {
						const context = DOM.context2d(width, height);
						const path = d3.geoPath(projection, context);
						context.save();
						context.beginPath(), path(outline), context.clip(), context.fillStyle = "#fff", context.fillRect(0, 0, width, height);
						context.beginPath(), path(graticule), context.strokeStyle = "#ccc", context.stroke();
						context.beginPath(), path(land), context.fillStyle = "#000", context.fill();
						context.restore();
						context.beginPath(), path(outline), context.strokeStyle = "#000", context.stroke();
						return context.canvas;
					}

Example from World Map (Canvas) Observable

WebGL API example


						<canvas id="glCanvas" width="100" height="100"></canvas>
						<script type="application/javascript">
							main();
							function main() {
								const canvas = document.querySelector("#glCanvas");
								const gl = canvas.getContext("webgl");  //initialize GL context

								if (gl === null) {
									alert("Unable to initialize WebGL.");
									return;
								}

								gl.clearColor(0.0, 0.0, 0.0, 1.0);  //set clear color to black
								gl.clear(gl.COLOR_BUFFER_BIT);  //clear color buffer
							}
						</script>

Documentation: MDN WebGL API, example presented from MDN webgl creation sample.

Canvas vs. SVG

SVG (vector)

Simpler to use
SVG redraw inefficient for large datasets
SVG interactivity suffers with large datasets

Canvas (raster) advantages

More complex to use
Draw happens in immediate mode (DOM not involved)
Uses GPU for rendering
Stores data in graphic card memory for faster access

Outline

Overview
Canvas
WebGL
three.js
Processing, Processing.js, P5.js
Vtk.js
Mapbox GL & Deck.gl

WebGL

Web Graphics Library, a W3 standard
JS API for OpenGL ES 2.0 (OpenGL for mobile devices)
2D & 3D interactive rendering in HTML5 canvas
GPU accelerated rendering
Popular JS libraries based on WebGL:
- three.js
- PhiloGL
- GLGE
- P5.js

WebGL vs. SVG

	SVG	WebGL
Supports 3D content		✓
DOM Interaction	✓
Declarative scenegraph		✓
CSS Integration	✓
Scripting access	✓	✓

Scene geometry

A projection matrix converts world space coordinates into clip space coordinates
A model matrix, takes model data and moves it around in 3D world space
A view matrix is used to move objects in the scene to simulate the position of the camera being changed

Projection Matrix

The projection matrix is used to project the scene
A projection matrix is defined by a view frustrum and clipping planes
The view frustum defines the region whose contents are visible to the user
Clipping planes further limit what is visible in the view frustrum
Projection matrices are usually set to perspective or orthographic

Objects are built from primitives

WebGL uses the same primitives than OpenGL

Objects data

Defined as indexed arrays
Arrays are passed to GPU as buffers


					var vertices = [-1, -1, -1, 1, -1, -1, 1, 1, -1, -1, 1, -1,
					  -1, -1, 1, 1, -1, 1, 1, 1, 1, -1, 1, 1,
					  -1, -1, -1, -1, 1, -1, -1, 1, 1, -1, -1, 1,
					  1, -1, -1, 1, 1, -1, 1, 1, 1, 1, -1, 1,
					  -1, -1, -1, -1, -1, 1, 1, -1, 1, 1, -1, -1,
					  -1, 1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1];

					var colors = [5, 3, 7, 5, 3, 7, 5, 3, 7, 5, 3, 7,
					  1, 1, 3, 1, 1, 3, 1, 1, 3, 1, 1, 3,
					  0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 1,
					  1, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0,
					  1, 1, 0, 1, 1, 0, 1, 1, 0, 1, 1, 0,
					  0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0];

					var indices = [0, 1, 2, 0, 2, 3, 4, 5, 6, 4, 6, 7,
					  8, 9, 10, 8, 10, 11, 12, 13, 14, 12, 14, 15,
					  16, 17, 18, 16, 18, 19, 20, 21, 22, 20, 22, 23];

					var vertex_buffer = gl.createBuffer();
					gl.bindBuffer(gl.ARRAY_BUFFER, vertex_buffer);
					gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(vertices), gl.STATIC_DRAW);

					var color_buffer = gl.createBuffer();
					gl.bindBuffer(gl.ARRAY_BUFFER, color_buffer);
					gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(colors), gl.STATIC_DRAW);

					var index_buffer = gl.createBuffer();
					gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER, index_buffer);
					gl.bufferData(gl.ELEMENT_ARRAY_BUFFER, new Uint16Array(indices), gl.STATIC_DRAW);

Graphic pipeline

A fragment shader computes a color for each pixel of the primitive being drawn, operations that may include texture mapping and lighting


					var canvas = document.getElementById('canvas01');
					gl = canvas.getContext('webgl');

					var vertices = [-1, -1, -1, 1, -1, -1, 1, 1, -1, -1, 1, -1,
					  -1, -1, 1, 1, -1, 1, 1, 1, 1, -1, 1, 1,
					  -1, -1, -1, -1, 1, -1, -1, 1, 1, -1, -1, 1,
					  1, -1, -1, 1, 1, -1, 1, 1, 1, 1, -1, 1,
					  -1, -1, -1, -1, -1, 1, 1, -1, 1, 1, -1, -1,
					  -1, 1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1];

					var colors = [5, 3, 7, 5, 3, 7, 5, 3, 7, 5, 3, 7,
					  1, 1, 3, 1, 1, 3, 1, 1, 3, 1, 1, 3,
					  0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 1,
					  1, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0,
					  1, 1, 0, 1, 1, 0, 1, 1, 0, 1, 1, 0,
					  0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0];

					var indices = [0, 1, 2, 0, 2, 3, 4, 5, 6, 4, 6, 7,
					  8, 9, 10, 8, 10, 11, 12, 13, 14, 12, 14, 15,
					  16, 17, 18, 16, 18, 19, 20, 21, 22, 20, 22, 23];

					var vertex_buffer = gl.createBuffer();
					gl.bindBuffer(gl.ARRAY_BUFFER, vertex_buffer);
					gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(vertices), gl.STATIC_DRAW);

					var color_buffer = gl.createBuffer();
					gl.bindBuffer(gl.ARRAY_BUFFER, color_buffer);
					gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(colors), gl.STATIC_DRAW);

					var index_buffer = gl.createBuffer();
					gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER, index_buffer);
					gl.bufferData(gl.ELEMENT_ARRAY_BUFFER, new Uint16Array(indices), gl.STATIC_DRAW);

					var vertCode = 'attribute vec3 position;' +
					  'uniform mat4 Pmatrix;' +
					  'uniform mat4 Vmatrix;' +
					  'uniform mat4 Mmatrix;' +
					  'attribute vec3 color;' +
					  'varying vec3 vColor;' +
					  'void main(void) { ' +
					  'gl_Position = Pmatrix*Vmatrix*Mmatrix*vec4(position, 1.);' +
					  'vColor = color;' +
					  '}';

					var fragCode = 'precision mediump float;' +
					  'varying vec3 vColor;' +
					  'void main(void) {' +
					  'gl_FragColor = vec4(vColor, 1.);' +
					  '}';

					var vertShader = gl.createShader(gl.VERTEX_SHADER);
					gl.shaderSource(vertShader, vertCode);
					gl.compileShader(vertShader);

					var fragShader = gl.createShader(gl.FRAGMENT_SHADER);
					gl.shaderSource(fragShader, fragCode);
					gl.compileShader(fragShader);

					var shaderprogram = gl.createProgram();
					gl.attachShader(shaderprogram, vertShader);
					gl.attachShader(shaderprogram, fragShader);
					gl.linkProgram(shaderprogram);

					var _Pmatrix = gl.getUniformLocation(shaderprogram, "Pmatrix");
					var _Vmatrix = gl.getUniformLocation(shaderprogram, "Vmatrix");
					var _Mmatrix = gl.getUniformLocation(shaderprogram, "Mmatrix");

					gl.bindBuffer(gl.ARRAY_BUFFER, vertex_buffer);
					var _position = gl.getAttribLocation(shaderprogram, "position");
					gl.vertexAttribPointer(_position, 3, gl.FLOAT, false, 0, 0);
					gl.enableVertexAttribArray(_position);

					gl.bindBuffer(gl.ARRAY_BUFFER, color_buffer);
					var _color = gl.getAttribLocation(shaderprogram, "color");
					gl.vertexAttribPointer(_color, 3, gl.FLOAT, false, 0, 0);
					gl.enableVertexAttribArray(_color);
					gl.useProgram(shaderprogram);

					function get_projection(angle, a, zMin, zMax) {
					  var ang = Math.tan((angle * .5) * Math.PI / 180);
					  return [
					    0.5 / ang, 0, 0, 0,
					    0, 0.5 * a / ang, 0, 0,
					    0, 0, -(zMax + zMin) / (zMax - zMin), -1,
					    0, 0, (-2 * zMax * zMin) / (zMax - zMin), 0
					  ];
					}

					var proj_matrix = get_projection(40, canvas.width / canvas.height, 1, 100);
					var mo_matrix = [1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1];
					var view_matrix = [1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1];

					view_matrix[14] = view_matrix[14] - 6;

					var AMORTIZATION = 0.95;
					var drag = false;
					var old_x, old_y;
					var dX = 0, dY = 0;

					var mouseDown = function (e) {
					  drag = true;
					  old_x = e.pageX, old_y = e.pageY;
					  e.preventDefault();
					  return false;
					};

					var mouseUp = function (e) {
					  drag = false;
					};

					var mouseMove = function (e) {
					  if (!drag) return false;
					  dX = (e.pageX - old_x) * 2 * Math.PI / canvas.width,
					  dY = (e.pageY - old_y) * 2 * Math.PI / canvas.height;
					  THETA += dX;
					  PHI += dY;
					  old_x = e.pageX, old_y = e.pageY;
					  e.preventDefault();
					};

					canvas.addEventListener("mousedown", mouseDown, false);
					canvas.addEventListener("mouseup", mouseUp, false);
					canvas.addEventListener("mouseout", mouseUp, false);
					canvas.addEventListener("mousemove", mouseMove, false);

					function rotateX(m, angle) {
					  var c = Math.cos(angle);
					  var s = Math.sin(angle);
					  var mv1 = m[1], mv5 = m[5], mv9 = m[9];
					  m[1] = m[1] * c - m[2] * s;
					  m[5] = m[5] * c - m[6] * s;
					  m[9] = m[9] * c - m[10] * s;
					  m[2] = m[2] * c + mv1 * s;
					  m[6] = m[6] * c + mv5 * s;
					  m[10] = m[10] * c + mv9 * s;
					}

					function rotateY(m, angle) {
					  var c = Math.cos(angle);
					  var s = Math.sin(angle);
					  var mv0 = m[0], mv4 = m[4], mv8 = m[8];
					  m[0] = c * m[0] + s * m[2];
					  m[4] = c * m[4] + s * m[6];
					  m[8] = c * m[8] + s * m[10];
					  m[2] = c * m[2] - s * mv0;
					  m[6] = c * m[6] - s * mv4;
					  m[10] = c * m[10] - s * mv8;
					}
					var THETA = 0, PHI = 0;
					var time_old = 0;

					var anim0 = function (time) {
					  requestAnimationFrame(anim0);
					  var dt = time - time_old;
					  if (!drag) {
					    dX *= AMORTIZATION, dY *= AMORTIZATION;
					    THETA += dX, PHI += dY;
					  }
					  mo_matrix[0] = 1, mo_matrix[1] = 0, mo_matrix[2] = 0, mo_matrix[3] = 0,
					  mo_matrix[4] = 0, mo_matrix[5] = 1, mo_matrix[6] = 0, mo_matrix[7] = 0,
					  mo_matrix[8] = 0, mo_matrix[9] = 0, mo_matrix[10] = 1, mo_matrix[11] = 0,
					  mo_matrix[12] = 0, mo_matrix[13] = 0, mo_matrix[14] = 0, mo_matrix[15] = 1;
					  rotateY(mo_matrix, THETA);
					  rotateX(mo_matrix, PHI);
					  time_old = time;
					  gl.enable(gl.DEPTH_TEST);
					  // gl.depthFunc(gl.LEQUAL);
					  gl.clearColor(0.5, 0.5, 0.5, 0.9);
					  gl.clearDepth(1.0);
					  gl.viewport(0.0, 0.0, canvas.width, canvas.height);
					  gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT);
					  gl.uniformMatrix4fv(_Pmatrix, false, proj_matrix);
					  gl.uniformMatrix4fv(_Vmatrix, false, view_matrix);
					  gl.uniformMatrix4fv(_Mmatrix, false, mo_matrix);
					  gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER, index_buffer);
					  gl.drawElements(gl.TRIANGLES, indices.length, gl.UNSIGNED_SHORT, 0);
					}
					anim0(0);

WebGL Examples

Outline

Overview
Canvas
WebGL
three.js
Processing, Processing.js, P5.js
Vtk.js
Mapbox GL & Deck.gl

Three.js

High-level access to WebGL and graphical utilities:

Scene
Camera
Geometry

3D Model Loaders
Lights
Materials

Shaders
Particles
Animation
Math Utilities

Basic three.js example


							<canvas id="canvas" style="width: 600px; height: 600px;">

							<script type="application/javascript">
							  var scene = new THREE.Scene();
							  var camera = new THREE.PerspectiveCamera(75, 1, 0.1, 1000);
							  camera.position.set(0, 0, 2);  //camera.position.z = 2;
							  //camera.lookAt(0, 0, 0);

							  var renderer = new THREE.WebGLRenderer({ canvas: canvas });
							  renderer.setSize(600, 600);

							  var geometry = new THREE.BoxGeometry(1, 1, 1);
							  var material = new THREE.MeshLambertMaterial({color: 0xFFFFFF});
							  var cube = new THREE.Mesh(geometry, material);
							  scene.add(cube);

							  var light = new THREE.PointLight(0xFFFFFF);
							  light.position.set(2, 2, 2);
							  scene.add(light);

							  var anim1 = () => {
									requestAnimationFrame(anim1)
									renderer.render(scene, camera)

									cube.rotation.x += Math.PI / 180
									cube.rotation.y += Math.PI / 180
									cube.rotation.z += Math.PI / 180
							  }

							  anim1();
							</script>

three.js examples

three.js Kinect skeleton player demo (dsci-554-kinect-tutorial)
three.js gallery
PhiloGL projects & examples

Outline

Overview
Canvas
WebGL
three.js
Processing, Processing.js, P5.js
Vtk.js
Mapbox GL & Deck.gl

Processing implementations

Sketchbook and language for learning to code targeted at visual arts
All processing implementations support 2D and 3D <canvas> contexts

Implementations

Processing Simplified Java API for drawing and graphics [Fry & Reas 2001]
Processing.js JS API to use Processing code [Resig 2008]
P5.js HTML5 processing implementation Gallery [McCarthy 2015]

Processing sketch

Basic P5.js example


						<script src="p5.min.js"></script>

						<script>
						int[] angles = { 30, 10, 45, 35, 60, 38, 75, 67 };

						void setup() {
						  size(640, 360);
						  noStroke();
						  noLoop();  // Run once and stop
						}

						void draw() {
						  background(100);
						  pieChart(300, angles);
						}

						void pieChart(float diameter, int[] data) {
						  float lastAngle = 0;
						  for (int i = 0; i < data.length; i++) {
							float gray = map(i, 0, data.length, 0, 255);
							fill(gray);
							arc(width/2,
							  height/2,
							  diameter,
							  diameter,
							  lastAngle,
							  lastAngle+radians(data[i]));
							lastAngle += radians(data[i]);
						  }
						}
						</script>

Processing examples

Canvas, P5.js and circle packing (collision and cluster force) on a map on Observable
P5.js examples

Outline

Overview
Canvas
WebGL
three.js
Processing, Processing.js, P5.js
Vtk.js
Mapbox GL & Deck.gl

vtk.js

VTK from Kitware is an example of a component architecture
Adds a rendering abstraction layer over OpenGL
Adds a rendering abstraction layer over OpenGL
Targeted at medical imaging and engineering work
vtk.js is a re-implementation of VTK/C++ in JavaScript

vtk.js Examples

Outline

Overview
Canvas
WebGL
three.js
Processing, Processing.js, P5.js
Vtk.js
Mapbox GL & Deck.gl

Mapbox GL & Deck.gl

Mapbox GL

Open-source libraries for embedding customizable and responsive client-side maps in web
You can use Mapbox GL JS to display Mapbox maps in a web browser

deck.gl

Simplify high-performance, WebGL-based visualization of large data sets
deck.gl maps data (usually an array of JSON objects) into a stack of visual layers
Cartographic projections and integration with major basemap providers including Mapbox, Google Maps and ESRI
Part of the vis.gl framework suite

Commonalities between Mapbox GL and deck.gl

"GL" comes from OpenGL
Interactive event handling such as picking, highlighting and filtering
Renders at a high frame rate

Mapbox GL JS Example

Mapbox GL JS dive into large datasets with 3D shapes in Mapbox GL blog and full-screen demo

Mapbox GL & Deck.gl examples

deck.gl + Mapbox GL in Vue app demo (mapbox-deckgl-example)
deck.gl Examples
deck.gl Showcase
deck.gl Observable Getting Started

Questions?