Google Cloud Storage Hosting w CI/CD and BigQuery Hit History Done In 5 Minutes!
Google Cloud Storage, HTTPS Load Balancing, Cloud CDN, Source Repositories, Cloud Build, Cloud Logging, and BigQuery.
This article walks through the steps to host a website on Google Cloud Storage, setup a Git based CI/CD system on Google Cloud Source Repositories w Cloud Build Triggers, and sink website hit records to BigQuery. In five minutes.
Prerequisites
— A domain name and access to add DNS records to it
— A Google Cloud Project with Billing enabled
— OpenSSH and Git installed on your computer
Five minutes. We will use the command line when possible to save time. Let’s get started.
Preliminary Tasks
Using your web browser, sign out of all google accounts then sign back in to the Google Cloud Console using the single account you wish to use.
- Browse to the Cloud Billing page and add a billing account.
- Browse to the Google Cloud Resource Manager. Create and name your project.
- Now browse to the Google Domain Verification page and verify ownership of your domain with Google Cloud.
Create Cloud Storage Bucket
Browse to the Google Cloud Console and launch a Cloud Shell (top right you will see an icon >
). The bucket name should be your domain name, e.g., mydomain.com. Enter the following command to create your Cloud Storage bucket that will store your website files.
gsutil mb -c standard -l us-central1 gs://yourdomain.com
Grant Storage Object Viewer
access to allUsers
to allow public access to the files in your bucket.
gsutil iam ch allUsers:objectViewer gs://yourdomain.com
Enter the command below to specify the main page and the error page to serve.
gsutil web set -m index.html -e 404.html gs://yourdomain.com
Create Cloud Source Repository
Enter the command below to create a git repository on Google Cloud. Name the repository whatever you want. The example below uses the name myrepository
.
gcloud source repos create myrepository
Create and Setup Your SSH keys
Generate Keys
Windows users launch another PowerShell window and navigate to their OpenSSH directory, e.g. cd C:\Users\myusername\.ssh
. Generate the keys by entering the command below. Leave the passphrase empty, and name the key google
.
ssh-keygen
Create a file namedconfig
in the ssh directory. Replace User me@mydomain.com
with your actual google cloud identity sign in email address. NOTE: the Add-Content
lines should have 4 spaces after the first quote. The below set of PowerShell commands produces the file.
Set-Content config 'Host source.developers.google.com'
Add-Content config ' HostName source.developers.google.com'
Add-Content config ' User me@mydomain.com'
Add-Content config ' IdentityFile ~/.ssh/google'
Register Public Key with Google Cloud Repositories
Browse to Google Source Repository Key Management, and register your public key. Name it laptop
, for example.
Setup Local Development Environment w/ Git
Create a development folder or directory on your computer called website
. Open a terminal window such as PowerShell and navigate there, e.g., cd C:\Users\myusername\Documents\mydomain\website
. Enter the following commands to initialize Git. Note that you are using an SSH address in setting up the git remote ssh://<your-email> @ source.developers.google.com/p/<project-name>/r/<repo-name>
.
git init
git config user.name "John Doe"
git config user.email me@mydomain.com
git remote add google ssh://me@mydomain.com@source.developers.google.com:2022/p/my-project-name/r/myrepository
git add .
git commit -m "initial push"
git push --all google
Setup HTTPS Load Balancer
Enable the Compute Engine API
Using Cloud Shell in the Google Cloud Console, enable the Cloud Build API by entering the following command.
gcloud services enable compute.googleapis.com
Now browse to the load balancer page and Click Create Load Balancer. Click Start Configuration under HTTP(S) Load Balancing. Choose From Internet to my VMs or serverless services, and Choose Global HTTP(S) Load Balancer. Click Continue.
Front End Configuration
Name it your new frontendfrontend
. Change protocol to https
. Change IP Address to Create New IP Address and name it public
. Click Reserve.
Click the Certificate
field. Click Create New Certificate. Name it whatever you want. Choose Create Google-managed certificate
and add your domain name to the Domains field. Click Create.
Enable HTTP to HTTPS redirect.
Backend Configuration
In the Backend Services and Backend Buckets
dropdown choose Create Backend Bucket
. Name it backend
. Click browse for bucket and highlight your bucket. Click Select. Enable Cloud CDN and Enable Cache Static Content.
Routing Rules
Leave to default settings.
Click Create. If prompted to, name the load balancer glb
.
Direct Domain To Load Balancer
Get Load Balancer IP Address
Browse to the Load Balancer page and click on your `glb` resource and make note of its IP address, e.g., 34.160.39.131. You will need this address to paste in to your domain name registrar’s system in the next step. Note: you do not want to copy the :443 or :80 at the end of the address.
Update DNS Records with Domain Name Registrar
Sign in to your domain name registrar’s admin page and update your DNS records. Add an A record pointing to your glb
IP address and a CNAME record pointing www
to yourdomain.com
.
Setup CI/CD with Cloud Build
Setup Biuld Trigger
Using Cloud Shell in the Google Cloud Console, enable the Cloud Build API by entering the following command.
gcloud services enable cloudbuild.googleapis.com
Browse to the Build Triggers page. Click Create Trigger. Name it push
. Choose the Region that your bucket is in. Under Source, choose the repository that you previously created. In the Cloud Build configuration file location, enter cloudbuild.yaml
. Leave the Service Account field blank. Click Create.
Place Cloud Build File in Website Folder
Create cloudbuild.yaml
on your local development machine in the website
folder. Change myrepository
and gs://mydomain.com/
to your specific values.
steps:
- name: gcr.io/google.com/cloudsdktool/cloud-sdk
args:
- rm
- '-rf'
- myrepository
- name: gcr.io/google.com/cloudsdktool/cloud-sdk
args:
- gcloud
- source
- repos
- clone
- myrepository
- name: gcr.io/google.com/cloudsdktool/cloud-sdk
args:
- gsutil
- cp
- myrepository/*
- 'gs://mydomain.com/'
options:
logging: CLOUD_LOGGING_ONLY
Create BigQuery Data Set
Using Cloud Shell in the Google Cloud Console, enter the following command to create a new BigQuery Dataset named hits
.
bq mk - location=us-central1 hits
Configure Cloud Logging Sink
— Browse to the Logs Router page and click Create Sink.
— Name the sink “bigquery” click Next.
— Select Sink Service BigQuery Dataset.
— Select the hits
BigQuery Dataset previously created.
— Paste the following into Logs To Send To Sink:
resource.type="http_load_balancer"
resource.labels.forwarding_rule_name="frontend"
resource.labels.url_map_name="glb"
httpRequest.requestUrl : ("mydomain.com")
Click Create.
Conclusion
You now have a website hosted on Cloud Storage, a Git based CI/CD system, and website hit logs being sent to BigQuery for historical analysis.