Tensorflow in 3D

Recommended reading:

TensorFlow is not just a deep learning library - it is a library for performing manipulations on numbers, and as such it can perform tasks that many other libraries can. In this lesson, we take a look at using TensorFlow to perform manipulations on 3D objects.

A 3D object can be modelled as a series of triangles in three-dimensional space, that we normally refer to as (x, y, z). These names are not required, but are commonly used. A triangle is created from three of these 3D points. A point itself can be represented as a vector with size (3,). An array of these is a matrix of size (n, 3), where n is the number of points we have. Let’s dive in and see a basic cube. We will need this functionality later, so let’s create a function that plots out a basic shape:

from mpl_toolkits.mplot3d import Axes3D
import numpy as np
from matplotlib import cm
import matplotlib.pyplot as plt
from scipy.spatial import Delaunay

def plot_basic_object(points):
    """Plots a basic object, assuming its convex and not too complex"""
    tri = Delaunay(points).convex_hull
    fig = plt.figure(figsize=(8, 8))
    ax = fig.add_subplot(111, projection='3d')
    S = ax.plot_trisurf(points[:,0], points[:,1], points[:,2],
                        triangles=tri,
                        shade=True, cmap=cm.Blues,lw=0.5)
    ax.set_xlim3d(-5, 5)
    ax.set_ylim3d(-5, 5)
    ax.set_zlim3d(-5, 5)

    plt.show()

 

If you are using Jupyter Notebooks, I recommend running this line of code, which gives you an awesome interactive 3D plot. Left-click-and-drag moves around, right-click-and-drag zooms in or out.

%matplotlib notebook

Now let’s create a shape. The function below will return the six points that make up a cube. If you go back to the previous function, you will see the Delaunay line, which turns these points into triangles, so that we may render them.

import numpy as np
def create_cube(bottom_lower=(0, 0, 0), side_length=5):
    """Creates a cube starting from the given bottom-lower point (lowest x, y, z values)"""
    bottom_lower = np.array(bottom_lower)
    points = np.vstack([
        bottom_lower,
        bottom_lower + [0, side_length, 0],
        bottom_lower + [side_length, side_length, 0],
        bottom_lower + [side_length, 0, 0],
        bottom_lower + [0, 0, side_length],
        bottom_lower + [0, side_length, side_length],
        bottom_lower + [side_length, side_length, side_length],
        bottom_lower + [side_length, 0, side_length],
        bottom_lower,
    ])
    return points

 

Now let’s put those pieces together and see what it looks like:

cube_1 = create_cube(side_length=2)


plot_basic_object(cube_1)


I’m just showing an image here, but you can see the cube, and that is has been turned into triangles by our code and coloured differently (depending on the z-value). This is great, but now let’s use TensorFlow to do some operations on this.

Translation

A translation is a simple move: up/down, left/right, forward/backwards, or some combination of these. It is created by simply adding a vector to each point. If you add the same vector to all points, then the whole object will move in unison. Check out our lesson on broadcasting to understand what happens when we add our translation vector with size (3,) to our points matrix of size (n, 3).

import tensorflow as tf

def translate(points, amount):
    return tf.add(points, amount)


points = tf.constant(cube_1, dtype=tf.float32)

# Update the values here to move the cube around.
translation_amount = tf.constant([3, -3, 0], dtype=tf.float32)


translate_op = translate(points, translation_amount)

with tf.Session() as session:
    translated_cube = session.run(translate_op)


plot_basic_object(translated_cube)


Rotation

A rotation is formed by creating the dot product or a rotating matrix and the original points. Rotating an object first requires you to determine which axis you are rotating over. To rotate around a particular axis, set that axis’ value to zeros, with a 1 in the related axis. There are three matrices you need:

Rotating around the x-axis

[[1, 0, 0],
 [0, cos \theta, sin \theta],
 [0, -sin \theta, cos \theta]]
  

Rotating around the y-axis

[[cos \theta, 0, -sin \theta],
 [0, 1, 0],
 [sin \theta, 0, cos \theta]]
  

Rotating around the z-axis

[[cos \theta, sin \theta, 0],
 [-sin \theta, cos \theta, 0],
 [0, 0, 1]]
 

In the above, theta is the number of degrees to rotate the object. You take a point, compute the dot product with one of these matrices, and then the point is rotated around the relevant axis. We can also compute the dot product of one of these (3, 3) matrices against our (n, 3) points matrix. However, for a dot product to work, you need to have the inner dimensions matrix (and n is not necessarily 3). Therefore, you need to put the points matrix first - then it becomes a dot product of a (n, 3) matrix with a (3, 3) matrix.

def rotate_around_z(points, theta):
    theta = float(theta)
    rotation_matrix = tf.stack([[tf.cos(theta), tf.sin(theta), 0],
                                   [-tf.sin(theta), tf.cos(theta), 0],
                                   [0, 0, 1]])
    return tf.matmul(tf.to_float(points), tf.to_float(rotation_matrix))


with tf.Session() as session:
    result = session.run(rotate_around_z(cube_1, 75))


plot_basic_object(result)


With simple matrix manipulations like these, but combined and at scale, you can create a whole range of transformations to 3D objects like this. Shearing, scaling, intersections and more are possible using this concept. GPUs are very good at doing these transformations, which happen to relate to the same types of transformations needed for data analytic work such as deep learning. Due to this, TensorFlow works nicely with GPUs, which works nicely with 3D objects as well as for deep learning tasks.

Exercises

  1. Create a different object, such as a pyramid or hexagonal prism. If you aren’t sure how to start, start with a prism and create it in 2D first.
  2. Rotate the object around the x and y axis.
  3. You can combine rotations into a single transformation matrix. To do this, simply compute the dot product of the rotations.
  4. Does order matter for the question 3?
  5. A shear matrix is the identity matrix with a value in the off diagonal. An example is below. Create a shearing matrix and test different values.
[[1, 0.5, 0],
 [0, 1, 0],
 [0, 0, 1]]

Keep going!

We have an increasing set of lessons that we hope guides you through learning this powerful library. Follow these links to keep going to our next lesson.

You can also use the nav menu at the top of the page to go directly to a specific lesson.

Coming soon (although not written by us):

Get updates

Sign up here to receive infrequent emails from us about updates to the site and when new lessons are released.



* indicates required

If you have any feedback, please see our page here. If you spot any errors with our lessons, please direct them to our Github page with the name of the lesson in which the error resides, so that we can resolve them and close them off there.

If you have larger questions that may involve consultancy, please contact us here