Tensorflow in 3D

Recommended reading:

TensorFlow is not just a deep learning library - it is a library for performing manipulations on numbers, and as such it can perform tasks that many other libraries can. In this lesson, we take a look at using TensorFlow to perform manipulations on 3D objects.

A 3D object can be modelled as a series of triangles in three-dimensional space, that we normally refer to as (x, y, z). These names are not required, but are commonly used. A triangle is created from three of these 3D points. A point itself can be represented as a vector with size (3,). An array of these is a matrix of size (n, 3), where n is the number of points we have. Let’s dive in and see a basic cube. We will need this functionality later, so let’s create a function that plots out a basic shape:

from mpl_toolkits.mplot3d import Axes3D
import numpy as np
from matplotlib import cm
import matplotlib.pyplot as plt
from scipy.spatial import Delaunay

def plot_basic_object(points):
    """Plots a basic object, assuming its convex and not too complex"""
    tri = Delaunay(points).convex_hull
    fig = plt.figure(figsize=(8, 8))
    ax = fig.add_subplot(111, projection='3d')
    S = ax.plot_trisurf(points[:,0], points[:,1], points[:,2],
                        triangles=tri,
                        shade=True, cmap=cm.Blues,lw=0.5)
    ax.set_xlim3d(-5, 5)
    ax.set_ylim3d(-5, 5)
    ax.set_zlim3d(-5, 5)

    plt.show()

 

If you are using Jupyter Notebooks, I recommend running this line of code, which gives you an awesome interactive 3D plot. Left-click-and-drag moves around, right-click-and-drag zooms in or out.

%matplotlib notebook

Now let’s create a shape. The function below will return the six points that make up a cube. If you go back to the previous function, you will see the Delaunay line, which turns these points into triangles, so that we may render them.

import numpy as np
def create_cube(bottom_lower=(0, 0, 0), side_length=5):
    """Creates a cube starting from the given bottom-lower point (lowest x, y, z values)"""
    bottom_lower = np.array(bottom_lower)
    points = np.vstack([
        bottom_lower,
        bottom_lower + [0, side_length, 0],
        bottom_lower + [side_length, side_length, 0],
        bottom_lower + [side_length, 0, 0],
        bottom_lower + [0, 0, side_length],
        bottom_lower + [0, side_length, side_length],
        bottom_lower + [side_length, side_length, side_length],
        bottom_lower + [side_length, 0, side_length],
        bottom_lower,
    ])
    return points

 

Now let’s put those pieces together and see what it looks like:

cube_1 = create_cube(side_length=2)


plot_basic_object(cube_1)


I’m just showing an image here, but you can see the cube, and that is has been turned into triangles by our code and coloured differently (depending on the z-value). This is great, but now let’s use TensorFlow to do some operations on this.

Translation

A translation is a simple move: up/down, left/right, forward/backwards, or some combination of these. It is created by simply adding a vector to each point. If you add the same vector to all points, then the whole object will move in unison. Check out our lesson on broadcasting to understand what happens when we add our translation vector with size (3,) to our points matrix of size (n, 3).

import tensorflow as tf

def translate(points, amount):
    return tf.add(points, amount)


points = tf.constant(cube_1, dtype=tf.float32)

# Update the values here to move the cube around.
translation_amount = tf.constant([3, -3, 0], dtype=tf.float32)


translate_op = translate(points, translation_amount)

with tf.Session() as session:
    translated_cube = session.run(translate_op)


plot_basic_object(translated_cube)


Rotation

A rotation is formed by creating the dot product or a rotating matrix and the original points. Rotating an object first requires you to determine which axis you are rotating over. To rotate around a particular axis, set that axis’ value to zeros, with a 1 in the related axis. There are three matrices you need:

Rotating around the x-axis

[[1, 0, 0],
 [0, cos \theta, sin \theta],
 [0, -sin \theta, cos \theta]]
  

Rotating around the y-axis

[[cos \theta, 0, -sin \theta],
 [0, 1, 0],
 [sin \theta, 0, cos \theta]]
  

Rotating around the z-axis

[[cos \theta, sin \theta, 0],
 [-sin \theta, cos \theta, 0],
 [0, 0, 1]]
 

In the above, theta is the number of degrees to rotate the object. You take a point, compute the dot product with one of these matrices, and then the point is rotated around the relevant axis. We can also compute the dot product of one of these (3, 3) matrices against our (n, 3) points matrix. However, for a dot product to work, you need to have the inner dimensions matrix (and n is not necessarily 3). Therefore, you need to put the points matrix first - then it becomes a dot product of a (n, 3) matrix with a (3, 3) matrix.

def rotate_around_z(points, theta):
    theta = float(theta)
    rotation_matrix = tf.stack([[tf.cos(theta), tf.sin(theta), 0],
                                   [-tf.sin(theta), tf.cos(theta), 0],
                                   [0, 0, 1]])
    return tf.matmul(tf.to_float(points), tf.to_float(rotation_matrix))


with tf.Session() as session:
    result = session.run(rotate_around_z(cube_1, 75))


plot_basic_object(result)


With simple matrix manipulations like these, but combined and at scale, you can create a whole range of transformations to 3D objects like this. Shearing, scaling, intersections and more are possible using this concept. GPUs are very good at doing these transformations, which happen to relate to the same types of transformations needed for data analytic work such as deep learning. Due to this, TensorFlow works nicely with GPUs, which works nicely with 3D objects as well as for deep learning tasks.

Exercises

  1. Create a different object, such as a pyramid or hexagonal prism. If you aren’t sure how to start, start with a prism and create it in 2D first.
  2. Rotate the object around the x and y axis.
  3. You can combine rotations into a single transformation matrix. To do this, simply compute the dot product of the rotations.
  4. Does order matter for the question 3?
  5. A shear matrix is the identity matrix with a value in the off diagonal. An example is below. Create a shearing matrix and test different values.
[[1, 0.5, 0],
 [0, 1, 0],
 [0, 0, 1]]

Stuck? Get the exercise solutions here

If you are looking for solutions on the exercises, or just want to see how I solved them, then our solutions bundle is what you are after.

Buying the bundle gives you free updates for life - meaning when we add a new lesson, you get an updated bundle with the solutions.

It's just $9, and it also helps us to keep running the site with free lessons.

Grow your business with data analytics

Looking to improve your business through data analytics? Are you interested in implementing data mining, automation or artificial intelligence?

This book is the ultimate guide to getting started with using data in your business, with a non-technical view and focusing on achieving good outcomes. We don't get bogged down by technical detail or complex algorithms.

For additional offers, including a premium package, see this page.

Get updates

Sign up here to receive infrequent emails from us about updates to the site and when new lessons are released.



* indicates required

You can also support LearningTensorFlow.com by becoming a patron at Patreon. If we have saved you trawling through heavy documentation, or given you a pointer on where to go next, help us to create new lessons and keep the site running.

You'll also get access to extra content and updates not available on LearningTensorFlow.com!

Coming soon!

Note: despite the title, this book has no relationship to LearningTensorFlow.com

Learn More!