5.1 Google Cloud Platform
Google Cloud Platform (GCP) is a suite of cloud-based products that work together to provide robust and seamless solutions to high performance computing, big data storage, data analytics, machine learning and more. The platform is built on Google’s internal infrastructure and it’s known for it’s reliability, flexibility, speed, and a relatively low cost “pay-as-you” model. At emLab we mainly use three of GCP’s products: Cloud Storage, BigQuery, and Compute Engine and each year we have a limited amount of credits to cover the costs of using these tools for projects that require the storage and use of very large datasets, projects that require large computational power, or those that use Global Fishing Watch data.
When it comes to high performance computing, Compute Engine is a very useful tool. It allows us to easily create custom-made virtual machines with the storage, memory, and number of cores needed for a given task. Virtual machines can run public images of Linux, Windows Server, and can also be used to deploy Docker containers. Starting, stopping, and deleting virtual machines is easy and fast which means we have full control on the amount of resources we use and get billed for.
To get up and running with a virtual machine, Grant McDermott (SFG alumn and fellow) wrote this really good step by step tutorial. Here you will learn how to create, start, connect to, and stop a virtual machine in Compute Engine and how to install Rstudio server and Git. Importantly, you will also find a link that walks you through the installation of Google Cloud SDK command line utility (
gcloud) which is a prerequisite to be able to speak to your virtual machine from your local terminal. When you install
gcloud and autenthicate your credentials you will be able to set
emlab-gcp as your project which will link you to emlab’s billing account. If you have not joined
emlab-gcp please get in touch with and we will set you up!
General guidelines for creating and running virtual machines:
- Give your VM a descriptive name associated with the specific project you will be using it for.
- Give your VM a static IP address. That way you can add it to your bookmarks and access it easily.
- Always turn off your VM when not it use. Remember we get charged for every minute it is on.
- Delete the VM once the project is finished. That way we keep things tidy.