Moving to AWS

Recommended reading:

In a lot of cases running code can be quite time consuming, especially if you are doing machine learning or anything to do with neural networks. Unless you have spent a large amount of money on your machine, going to a cloud based service might be the best way to go.

In this tutorial we are going to take some Tensorflow code and move it to a Amazon web service, (AWS) Elastic Compute Cloud instance (EC2).

Amazon Web Services (AWS) is a secure cloud services platform, which offers compute power, database storage, content delivery and other functionality to help businesses scale and grow. Additionally, Amazon Elastic Compute Cloud (Amazon EC2) is a web service that provides resizable compute capacity in the cloud. It is designed to make web-scale cloud computing easier for developers.

The advantage to this is, Amazon have a huge selection of cloud based servers with a lot of power behind them. This will allow you to run your code over the web in half the time you would be able to run it from your local machine. It also means that if it is a massive file that is going to take 5 - 8 hours to run, you can run it on the EC2 instance and leave it in the background without using your whole computers resources.

Creating an EC2 environment will cost you money, however it is a very minimal amount, probably about $4.00 for 8 hours. Further more once you stop using it, you will not be charged.
For prices visit this link

Creating an EC2 instances

To start with, go to the AWS console at:

https://console.aws.amazon.com/console/home?region=us-east-1

Log in with your Amazon account. If you don’t have one, you will be prompted to create one, which you will need to do in order to continue.

Next, go to the EC2 service console at: https://console.aws.amazon.com/ec2/v2/home?region=us-east-1.

Click on Launch Instance and choose your local region(i.e sydney, N california) as your location in the drop-down menu at the top-right.

Next go to community AMIs and search Ubuntu x64 AMI with TensorFlow (GPU), this is ready to run code via the GPU but it is also more then enough to run your basic or big Tensorflow scripts on it, and the great part is Tensorflow is already installed.

At this point, you will be charged, so please remember to shut down your machines when you are done with them. You can go to the EC2 service, select the machine, and stop it. You won't be charged for machines that are not running.

You’ll be prompted with some information on how to connect to your instance. If you haven’t used AWS before, you will probably need to create a new key pair to securely connect to your instance. In this case, give your key pair a name, download the pemfile, and store it in a safe place—if lost, you will not be able to connect to your instance again!

Click on Connect for information on using the pem file to connect to your instance. The most likely scenario is that you will use ssh with the following command:

ssh -i <certificante_name>.pem ubuntu@<server_ip_address>

Moving your code to AWS EC2

We will use this following example to move on to our EC2 instance, which is from our convergence lesson

import tensorflow as tf
import numpy as np

# x and y are placeholders for our training data
x = tf.placeholder("float")
y = tf.placeholder("float")
# w is the variable storing our values. It is initialised with starting "guesses"
# w[0] is the "a" in our equation, w[1] is the "b"
w = tf.Variable([1.0, 2.0], name="w")
# Our model of y = a*x + b
y_model = tf.mul(x, w[0]) + w[1]

# Our error is defined as the square of the differences
error = tf.square(y - y_model)
# The Gradient Descent Optimizer does the heavy lifting
train_op = tf.train.GradientDescentOptimizer(0.01).minimize(error)

# Normal TensorFlow - initialize values, create a session and run the model
model =  tf.global_variables_initializer()

with tf.Session() as session:
    session.run(model)
    for i in range(1000):
        x_value = np.random.rand()
        y_value = x_value * 2 + 6
        session.run(train_op, feed_dict={x: x_value, y: y_value})

    w_value = session.run(w)
    print("Predicted model: {a:.3f}x + {b:.3f}".format(a=w_value[0], b=w_value[1]))
    
    

There are many ways to get this file onto your EC2 instance, but one of the easiest is to just copy-and-paste the contents.

To start with, highlight all of the above code by pressing ctrl + A and then copy all of the code with ctrl + C

On the Amazon Virtual Machine, move to the home directory and open nano with a new filename, which we will call basic.py for this example: (The following are terminal commands)

$ cd~/

$ nano <nameofscript>.py

The nano program will open, which is a command-line text editor.

With this program open, paste the contents of your clipboard into this file. On some systems, you may need to use a file option of the ssh program, rather than pressing Ctrl+ V to paste. In nano, press Ctrl+ O to save the file on the disk, which we called basic.py and then Ctrl+ X to exit the program.

Once you have exited nano, type in python basic.py and you’re done!

You should now see the result of the code pop up in the terminal, as you could most likely see this could be a much better way of doing large data programs.

Exercises

Facenet is a Face recognition programing which utilizes Tensorflow, it offers pre-trained models for you to download and run to see how it works.

1) go to this link and download a pre-trained face recognition model

2) Using the above tutorial, Upload the code to the EC2 instances and get it running

Stuck? Get the exercise solutions here

If you are looking for solutions on the exercises, or just want to see how I solved them, then our solutions bundle is what you are after.

Buying the bundle gives you free updates for life - meaning when we add a new lesson, you get an updated bundle with the solutions.

It's just $9, and it also helps us to keep running the site with free lessons.

Grow your business with data analytics

Looking to improve your business through data analytics? Are you interested in implementing data mining, automation or artificial intelligence?

This book is the ultimate guide to getting started with using data in your business, with a non-technical view and focusing on achieving good outcomes. We don't get bogged down by technical detail or complex algorithms.

For additional offers, including a premium package, see this page.

Get updates

Sign up here to receive infrequent emails from us about updates to the site and when new lessons are released.



* indicates required

You can also support LearningTensorFlow.com by becoming a patron at Patreon. If we have saved you trawling through heavy documentation, or given you a pointer on where to go next, help us to create new lessons and keep the site running.

You'll also get access to extra content and updates not available on LearningTensorFlow.com!

Coming soon!

Note: despite the title, this book has no relationship to LearningTensorFlow.com

Learn More!