Arrays and working with Images

Recommended reading:

In this tutorial, we are going to work with an image, in order to visualise changes to an array. Arrays are powerful structures, as we saw briefly in the previous tutorial. Generating interesting arrays can be difficult, but images provide a great option.

First, download this image (Right Click, and look for an option like “Save Image As…”) to your computer.

This image comes from Wikimedia Commons, by user Uoaei1.

To work with images, we will need matplotlib. We will also need the pillow library, which overrides the deprecated PIL library for working with images. You can install both in your environment using Anaconda’s installation method:

conda install matplotlib pillow

To load the image, we use matplotlib’s image module:

import matplotlib.image as mpimg
import os
# First, load the image
dir_path = os.path.dirname(os.path.realpath(__file__))
filename = dir_path + "/MarshOrchid.jpg"

# Load the image
image = mpimg.imread(filename)

# Print out its shape
print(image.shape)

The above code reads in the image as a NumPy array, and prints out the size. Note that the filename needs to be a full path (absolute or relative) to the downloaded image file.

You’ll see the output, which is (5528, 3685, 3). This means the image is 5528 pixels high, 3685 pixels wide, and 3 colors “deep”.

You can view the current image using pyplot, like so:

import matplotlib.pyplot as plt
plt.imshow(image)
plt.show()

Now that we have our image, lets use TensorFlow to do some changes to it.

Geometric Manipulations

The first transformation we will perform is a transpose, turning the image 90 degrees counter-clockwise. The full program is below, most of which you have seen.

import tensorflow as tf
import matplotlib.image as mpimg
import matplotlib.pyplot as plt
import os

# First, load the image again
dir_path = os.path.dirname(os.path.realpath(__file__))
filename = dir_path + "/MarshOrchid.jpg"
image = mpimg.imread(filename)

# Create a TensorFlow Variable
x = tf.Variable(image, name='x')

model = tf.global_variables_initializer()

with tf.Session() as session:
    x = tf.transpose(x, perm=[1, 0, 2])
	session.run(model)
	result = session.run(x)


plt.imshow(result)
plt.show()

The result of the transpose operation:

The new bit is this line:

x = tf.transpose(x, perm=[1, 0, 2])

This line uses TensorFlow’s transpose method, swapping the axes 0 and 1 around using the perm parameter (axis 2 stays where it is).

The next manipulation we will do is a flip (left-right), swapping the pixels from one side to another. TensorFlow has a method for this called reverse_sequence, but the signature is a bit odd. Here is what the documentation says (from that page):

tf.reverse(tensor, axis, name=None)

Reverses specific dimensions of a tensor.

NOTE tf.reverse has now changed behavior in preparation for 1.0. tf.reverse_v2 is currently an alias that will be deprecated before TF 1.0. Given a tensor, and a int32 tensor axis representing the set of dimensions of tensor to reverse. This operation reverses each dimension i for which there exists j s.t. axis[j] == i. tensor can have up to 8 dimensions. The number of dimensions specified in axis may be 0 or more entries. If an index is specified more than once, a InvalidArgument error is raised.

For this function, it can be best thought of as:

  1. Iterate through the array according to batch_dim. Setting batch_dim=0 means we go through the rows (top to bottom).
  2. For each item in the iteration
    • Slice a second dimension, denoted by seq_dim. Setting seq_dim=1 means we go through the columns (left to right).
    • The slice for the nth item in the iteration is denoted by the nth item in seq_lengths

Lets see it in action:

import numpy as np
import tensorflow as tf
import matplotlib.image as mpimg
import matplotlib.pyplot as plt
import os
# First, load the image again
dir_path = os.path.dirname(os.path.realpath(__file__))
filename = dir_path + "/MarshOrchid.jpg"
image = mpimg.imread(filename)
height, width, depth = image.shape

# Create a TensorFlow Variable
x = tf.Variable(image, name='x')

model = tf.global_variables_initializer()

with tf.Session() as session:
    x = tf.reverse_sequence(x, [width] * height, 1, batch_dim=0)
    session.run(model)
    result = session.run(x)

print(result.shape)
plt.imshow(result)
plt.show()

The new bit is this line:

x = tf.reverse_sequence(x, np.ones((height,)) * width, 1, batch_dim=0)

It iterates over the image top to bottom (along its height), and slices left to right (along its width). From here, it then takes a slice of size width, where width is the width of the image.

The code np.ones((height,)) * width creates a NumPy array filled with the value width. This is not very efficient! Unfortunately, at time of writing, it doesn’t appear that this function allows you to specify just a single value.

The result of the “fliplr” operation:

Exercises

1) Combine the transposing code with the flip code to rotate clock wise.

2) Currently, the flip code (using reverse_sequence) requires width to be precomputed. Look at the documentation for the tf.shape function, and use it to compute the width of the x variable within the session.

3) Perform a “flipud”, which flips the image top-to-bottom.

4) Compute a “mirror”, where the first half of the image is copied, flipped (l-r) and then copied into the second half.

Stuck? Get the exercise solutions here

If you are looking for solutions on the exercises, or just want to see how I solved them, then our solutions bundle is what you are after.

Buying the bundle gives you free updates for life - meaning when we add a new lesson, you get an updated bundle with the solutions.

It's just $9, and it also helps us to keep running the site with free lessons.

Grow your business with data analytics

Looking to improve your business through data analytics? Are you interested in implementing data mining, automation or artificial intelligence?

This book is the ultimate guide to getting started with using data in your business, with a non-technical view and focusing on achieving good outcomes. We don't get bogged down by technical detail or complex algorithms.

For additional offers, including a premium package, see this page.

Get updates

Sign up here to receive infrequent emails from us about updates to the site and when new lessons are released.



* indicates required

You can also support LearningTensorFlow.com by becoming a patron at Patreon. If we have saved you trawling through heavy documentation, or given you a pointer on where to go next, help us to create new lessons and keep the site running.

You'll also get access to extra content and updates not available on LearningTensorFlow.com!

Coming soon!

Note: despite the title, this book has no relationship to LearningTensorFlow.com

Learn More!