avatarGrig Gheorghiu

Free AI web copilot to create summaries, insights and extended knowledge, download it at here

10413

Abstract

set</span> instance_name dev pulumi<span class="hljs-built_in"> config </span><span class="hljs-built_in">set</span> instance_type n1-highmem-2 pulumi<span class="hljs-built_in"> config </span><span class="hljs-built_in">set</span> instance_image ubuntu-1604-xenial-v20191010 pulumi<span class="hljs-built_in"> config </span><span class="hljs-built_in">set</span> instance_disk_size 50</pre></div><p id="fe06">I then modified the sample __main__.py code so that it creates a GCP instance and also bootstraps it by means of an init/startup script. I based my code on on another <a href="https://github.com/pulumi/examples/blob/master/gcp-py-instance-nginx/__main__.py">Pulumi example</a>.</p><div id="11f8"><pre> cat main.py import pulumi from pulumi_gcp import compute with open(‘init_script.txt’, ‘r’) as init_script: data = init_script.read() <span class="hljs-keyword">script </span>= data <span class="hljs-built_in">config</span> = pulumi.Config() <span class="hljs-keyword">instance_name </span>= <span class="hljs-built_in">config</span>.require(‘<span class="hljs-keyword">instance_name’) </span><span class="hljs-keyword">instance_type </span>= <span class="hljs-built_in">config</span>.require(‘<span class="hljs-keyword">instance_type’) </span><span class="hljs-keyword">instance_image </span>= <span class="hljs-built_in">config</span>.require(‘<span class="hljs-keyword">instance_image’) </span><span class="hljs-keyword">instance_disk_size </span>= <span class="hljs-built_in">config</span>.require(‘<span class="hljs-keyword">instance_disk_size’)</span></pre></div><div id="2573"><pre><span class="hljs-attribute">addr</span> <span class="hljs-operator">=</span> compute.address.Address(instance_name)</pre></div><div id="6bd9"><pre>network = compute.Network(instance_name)<span class="hljs-built_in"> firewall </span>= compute.Firewall( instance_name, <span class="hljs-attribute">network</span>=network.self_link, allows=[ { “protocol”: “tcp”, “ports”: [“22”] } ] )</pre></div><div id="b9c0"><pre><span class="hljs-keyword">instance </span>= compute.<span class="hljs-keyword">Instance( </span> <span class="hljs-keyword">instance_name, </span> name=<span class="hljs-keyword">instance_name, </span> machine_type=<span class="hljs-keyword">instance_type, </span> <span class="hljs-keyword">boot_disk={ </span> “initializeParams”: { “image”: <span class="hljs-keyword">instance_image, </span> “size”: <span class="hljs-keyword">instance_disk_size </span> } }, network_interfaces=[ { “network”: network.id, “accessConfigs”: [{“nat_ip”: <span class="hljs-keyword">addr.address}] </span> } ], metadata_startup_script=<span class="hljs-keyword">script, </span>)</pre></div><div id="ab1b"><pre><span class="hljs-comment"># Export the DNS name of the bucket</span> pulumi.<span class="hljs-built_in">export</span>(“instance_name”, instance.name) pulumi.<span class="hljs-built_in">export</span>(“instance_network”, instance.network_interfaces) pulumi.<span class="hljs-built_in">export</span>(“external_ip”, addr.address)</pre></div><p id="917c">Note that I pass the data from a file called init_script.txt to the compute.Instance constructor, assigning it to the variable metadata_startup_script.</p><p id="b20a">Here is the init script, where I add my personal ssh public key on the remote instance, then I install docker and docker-compose:</p><div id="701e"><pre><span class="hljs-variable"></span> <span class="hljs-built_in">cat</span> init_script.txt <span class="hljs-comment"># add ssh pubkey</span></pre></div><div id="4ba5"><pre><span class="hljs-built_in">mkdir</span> -p /home/ggheo/.ssh <span class="hljs-built_in">chmod</span> 700 /home/ggheo/.ssh <span class="hljs-built_in">echo</span> “ssh-rsa contents of my ssh public key” &gt;&gt; /home/ggheo/.ssh/authorized_keys <span class="hljs-built_in">chown</span> -R ggheo:ggheo /home/ggheo/.ssh</pre></div><div id="84df"><pre><span class="hljs-meta"># install docker</span></pre></div><div id="0a6b"><pre>sudo apt update sudo apt install -y make apt-transport-https ca-certificates curl software-properties-common curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key<span class="hljs-built_in"> add </span>- sudo<span class="hljs-built_in"> add-apt-repository </span>“deb [arch=amd64] https://download.docker.com/linux/ubuntu bionic stable” sudo apt update sudo apt-cache policy docker-ce sudo apt install -y docker-ce</pre></div><div id="5e75"><pre><span class="hljs-meta"># download docker-compose</span></pre></div><div id="7276"><pre>sudo curl -L https:<span class="hljs-regexp">//gi</span>thub.com<span class="hljs-regexp">/docker/</span>compose<span class="hljs-regexp">/releases/</span>download<span class="hljs-regexp">/1.21.2/</span>docker-compose-`uname -s`-`uname -m` -o <span class="hljs-regexp">/usr/</span>local<span class="hljs-regexp">/bin/</span>docker-compose sudo chmod +x <span class="hljs-regexp">/usr/</span>local<span class="hljs-regexp">/bin/</span>docker-compose</pre></div><p id="ea84">To provision the <code>dev</code>stack, I ran :</p><div id="ffe1"><pre> pulumi up Previewing <span class="hljs-keyword">update</span> (<span class="hljs-built_in">dev</span>):</pre></div><div id="fe82"><pre><span class="hljs-keyword">Type</span> <span class="hljs-type">Name </span>Plan</pre></div><div id="f621"><pre><span class="hljs-bullet">+</span> pulumi:pulumi:Stack gcpinfra-dev create</pre></div><div id="6c83"><pre>+ ├─ gcp:<span class="hljs-built_in">compute</span>:Address dev create</pre></div><div id="fad3"><pre>+ ├─ gcp:<span class="hljs-built_in">compute</span>:Network dev create</pre></div><div id="e158"><pre>+ ├─ gcp:<span class="hljs-built_in">compute</span>:Firewall dev create</pre></div><div id="c9a6"><pre>+ └─ gcp:<span class="hljs-built_in">compute</span>:<span class="hljs-keyword">Instance</span> dev create</pre></div><div id="d58b"><pre><span class="hljs-symbol">Resources:</span></pre></div><div id="4421"><pre>+ <span class="hljs-number">5</span> <span class="hljs-built_in">to</span> <span class="hljs-built_in">create</span></pre></div><div id="9858"><pre><span class="hljs-keyword">Do</span> you want <span class="hljs-keyword">to</span> <span class="hljs-keyword">perform</span> this <span class="hljs-keyword">update</span>? yes</pre></div><div id="5547"><pre><span class="hljs-attribute">Updating (dev)</span><span class="hljs-punctuation">:</span></pre></div><div id="5051"><pre><span class="hljs-keyword">Type</span> <span class="hljs-keyword">Name</span> <span class="hljs-keyword">Status</span></pre></div><div id="5ece"><pre><span class="hljs-bullet">+</span> pulumi:pulumi:Stack gcpinfra-dev created</pre></div><div id="78a0"><pre>+ ├─ gcp:<span class="hljs-built_in">compute</span>:Network dev created</pre></div><div id="c2b5"><pre>+ ├─ gcp:<span class="hljs-built_in">compute</span>:Address dev created</pre></div><div id="6519"><pre>+ ├─ gcp:<span class="hljs-built_in">compute</span>:<span class="hljs-keyword">Instance</span> dev created</pre></div><div id="2aef"><pre>+ └─ gcp:<span class="hljs-built_in">compute</span>:Firewall dev created</pre></div><div id="b98c"><pre><span class="hljs-symbol">Outputs:</span></pre></div><div id="28a2"><pre><span class="hljs-attribute">external_ip</span> : “<span class="hljs-number">35.223.75.5</span>”</pre></div><div id="ee2a"><pre>instance_name : “<span class="hljs-type">dev</span>”</pre></div><div id="e79a"><pre><span class="hljs-symbol">instance_network:</span> [</pre></div><div id="360b"><pre>[<span class="hljs-symbol">0</span>]: <span class="hljs-link">{</span></pre></div><div id="fd63"><pre>accessConfigs : [</pre></div><div id="7149"><pre>[<span class="hljs-symbol">0</span>]: <span class="hljs-link">{</span></pre></div><div id="8540"><pre><span class="hljs-attribute">natIp</span> : “<span class="hljs-number">35.223.75.5</span>”</pre></div><div id="e03f"><pre>network_tier : “<span class="hljs-type">PREMIUM</span>”</pre></div><div id="d27f"><pre>}</pre></div><div id="2d6b"><pre>]</pre></div><div id="9918"><pre>name : “<span class="hljs-type">nic0</span>”</pre></div><div id="3590"><pre>network : “https:<span class="hljs-regexp">//</span>www.googleapis.com<span class="hljs-regexp">/compute/</span>v1<span class="hljs-regexp">/projects/my</span>project<span class="hljs-regexp">/global/</span>networks/dev-<span class="hljs-number">6</span>de02e1<span class="hljs-string">"</span></pre></div><div id="326e"><pre><span class="hljs-attribute">networkIp</span> : “<span class="hljs-number">10.128.0.2</span>”</pre></div><div id="ff26"><pre>subnetwork : “https:<span class="hljs-regexp">//</span>www.googleapis.com<span class="hljs-regexp">/compute/</span>v1<span class="hljs-regexp">/projects/my</span>project<span class="hljs-regexp">/regions/u</span>s-central1<span class="hljs-regexp">/subnetworks/</span>dev-<span class="hljs-number">6</span>de02e1<span class="hljs-string">"</span></pre></div><div id="d868"><pre><span class="hljs-symbol">subnetworkProject:</span> “myproject”</pre></div><div id="e53c"><pre>}</pre></div><div id="c103"><pre>]</pre></div><div id="7114"><pre><span class="hljs-symbol">Resources:</span></pre></div><div id="d4e5"><pre><span class="hljs-bullet">+</span> 5 created</pre></div><div id="dcf6"><pre><span class="hljs-attribute">Duration</span>: <span class="hljs-number">58</span>s</pre></div><div id="1c30"><pre>Permalink: https:<span class="hljs-regexp">//</span>app.pulumi.com<span class="hljs-regexp">/griggheo/g</span>cpinfra<span class="hljs-regexp">/dev/u</span>pdates/<span class="hljs-number">6</span></pre></div><p id="de2b">I was now able to list the new GCP instance and ssh into it with the gcloud CLI:</p><div id="3203"><pre> gcloud compute instances list NAME ZONE MACHINE_TYPE PREEMPTIBLE INTERNAL_IP EXTERNAL_IP STATUS dev us-central1-a n1-highmem-<span class="hljs-number">2 10.128.0</span>.<span class="hljs-number">2 35.223.75</span>.<span class="hljs-number">5</span> RUNNING</pre></div><div id="efab"><pre> gcloud compute ssh dev <span class="hljs-built_in">Warning</span>: Permanently added ‘compute<span class="hljs-number">.1672722981827980594</span>’ (ECDSA) <span class="hljs-keyword">to</span> the list <span class="hljs-keyword">of</span> known hosts. Welcome <span class="hljs-keyword">to</span> Ubuntu <span class="hljs-number">16.04</span><s

Options

pan class="hljs-number">.6</span> LTS (GNU/Linux <span class="hljs-number">4.15</span><span class="hljs-number">.0</span>–<span class="hljs-number">1046</span>-gcp x86_64)

  • Documentation: https://help.ubuntu.com
  • Management: https://landscape.canonical.com
  • Support: https://ubuntu.com/advantage <span class="hljs-number">40</span> packages can be updated. <span class="hljs-number">18</span> updates are <span class="hljs-keyword">security</span> updates. The programs included <span class="hljs-keyword">with</span> the Ubuntu <span class="hljs-keyword">system</span> are free software; the exact distribution terms <span class="hljs-keyword">for</span> <span class="hljs-keyword">each</span> program are described <span class="hljs-keyword">in</span> the individual files <span class="hljs-keyword">in</span> /usr/<span class="hljs-keyword">share</span>/doc<span class="hljs-comment">/*/copyright. Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by applicable law. ggheo@dev:</span></pre></div><p id="5fa7">At this point, I verified that docker, docker-compose and tutor were installed correctly. I was able to see the init script installation commands in /var/log/auth.log.</p><p id="bc2b">I was also able to ssh directly into the remote GCP instance, because my ssh key was added to authorized_files.</p><p id="856c">As an example of updating an existing Pulumi stack, let’s change the init script to add our user to the docker group. I added this line to the end of init_script.txt:</p><div id="643f"><pre>usermod -<span class="hljs-selector-tag">a</span> -G docker ggheo</pre></div><p id="65ba">When running <code>pulumi up</code>it will replace the instance because it detects that the startup script has changed:</p><div id="3f3e"><pre> pulumi up Previewing <span class="hljs-keyword">update</span> (<span class="hljs-built_in">dev</span>):</pre></div><div id="b21a"><pre><span class="hljs-keyword">Type</span> <span class="hljs-type">Name</span> Plan <span class="hljs-keyword">Info</span></pre></div><div id="656d"><pre><span class="hljs-symbol">pulumi:</span>pulumi:Stack gcpinfra-dev</pre></div><div id="1641"><pre>+- └─ gcp:<span class="hljs-built_in">compute</span>:<span class="hljs-keyword">Instance</span> dev <span class="hljs-built_in">replace</span> [diff: metadataStartupScript]</pre></div><div id="96b5"><pre><span class="hljs-symbol">Outputs:</span></pre></div><div id="6af0"><pre> instance_name : “dev” => <span class="hljs-keyword">output</span><<span class="hljs-keyword">string</span>></pre></div><div id="5791"><pre><span class="hljs-bullet">- </span>instance_network: [</pre></div><div id="d90b"><pre>- <span class="hljs-selector-attr">[0]</span>: {</pre></div><div id="a0ca"><pre>- accessConfigs : [</pre></div><div id="1b08"><pre>- <span class="hljs-selector-attr">[0]</span>: {</pre></div><div id="75c4"><pre>- natIp : “35.223.75.5”</pre></div><div id="57ff"><pre>- network_tier : “<span class="hljs-type">PREMIUM</span>”</pre></div><div id="b034"><pre>}</pre></div><div id="2e8d"><pre>]</pre></div><div id="c758"><pre>- name : “<span class="hljs-type">nic0</span>”</pre></div><div id="1e45"><pre>- network : “https:<span class="hljs-regexp">//</span>www.googleapis.com<span class="hljs-regexp">/compute/</span>v1<span class="hljs-regexp">/projects/my</span>project<span class="hljs-regexp">/global/</span>networks/dev-<span class="hljs-number">6</span>de02e1<span class="hljs-string">"</span></pre></div><div id="fab3"><pre>- networkIp : “10.128.0.2”</pre></div><div id="b298"><pre>- subnetwork : “https:<span class="hljs-regexp">//</span>www.googleapis.com<span class="hljs-regexp">/compute/</span>v1<span class="hljs-regexp">/projects/my</span>project<span class="hljs-regexp">/regions/u</span>s-central1<span class="hljs-regexp">/subnetworks/</span>dev-<span class="hljs-number">6</span>de02e1<span class="hljs-string">"</span></pre></div><div id="3602"><pre><span class="hljs-bullet">- </span>subnetworkProject: “myproject”</pre></div><div id="d285"><pre>}</pre></div><div id="1ad0"><pre>]</pre></div><div id="c73b"><pre>+ instance_network: <span class="hljs-keyword">output</span><<span class="hljs-keyword">string</span>></pre></div><div id="0ecf"><pre><span class="hljs-symbol">Resources:</span></pre></div><div id="c490"><pre>+<span class="hljs-number">-1</span> <span class="hljs-built_in">to</span> <span class="hljs-built_in">replace</span></pre></div><div id="4d3c"><pre><span class="hljs-symbol">4 </span>unchanged</pre></div><div id="e1d7"><pre><span class="hljs-keyword">Do</span> you want <span class="hljs-keyword">to</span> <span class="hljs-keyword">perform</span> this <span class="hljs-keyword">update</span>? yes</pre></div><div id="c58b"><pre><span class="hljs-attribute">Updating (dev)</span><span class="hljs-punctuation">:</span></pre></div><div id="3372"><pre><span class="hljs-keyword">Type</span> <span class="hljs-keyword">Name</span> <span class="hljs-keyword">Status</span> Info</pre></div><div id="20d2"><pre><span class="hljs-symbol">pulumi:</span>pulumi:Stack gcpinfra-dev</pre></div><div id="94d6"><pre>+- └─ gcp:<span class="hljs-built_in">compute</span>:<span class="hljs-keyword">Instance</span> dev replaced [diff: metadataStartupScript]</pre></div><div id="a8b7"><pre><span class="hljs-symbol">Outputs:</span></pre></div><div id="655c"><pre><span class="hljs-attribute">external_ip</span> : “<span class="hljs-number">35.223.75.5</span>”</pre></div><div id="f260"><pre>instance_name : “<span class="hljs-type">dev</span>”</pre></div><div id="f9b5"><pre><span class="hljs-symbol"> instance_network</span>: [</pre></div><div id="1ff0"><pre> <span class="hljs-selector-attr">[0]</span>: {</pre></div><div id="7ad1"><pre>accessConfigs : [</pre></div><div id="35b4"><pre>[<span class="hljs-symbol">0</span>]: <span class="hljs-link">{</span></pre></div><div id="3bd5"><pre><span class="hljs-attribute">natIp</span> : “<span class="hljs-number">35.223.75.5</span>”</pre></div><div id="dc2d"><pre>network_tier : “<span class="hljs-type">PREMIUM</span>”</pre></div><div id="d922"><pre>}</pre></div><div id="6d8c"><pre>]</pre></div><div id="fb7d"><pre>name : “<span class="hljs-type">nic0</span>”</pre></div><div id="4256"><pre>network : “https:<span class="hljs-regexp">//</span>www.googleapis.com<span class="hljs-regexp">/compute/</span>v1<span class="hljs-regexp">/projects/my</span>project<span class="hljs-regexp">/global/</span>networks/dev-<span class="hljs-number">6</span>de02e1<span class="hljs-string">"</span></pre></div><div id="0d9a"><pre>~ networkIp : “<span class="hljs-number">10.128.0.2</span>” => “<span class="hljs-number">10.128.0.3</span>”</pre></div><div id="0a5d"><pre>subnetwork : “https:<span class="hljs-regexp">//</span>www.googleapis.com<span class="hljs-regexp">/compute/</span>v1<span class="hljs-regexp">/projects/my</span>project<span class="hljs-regexp">/regions/u</span>s-central1<span class="hljs-regexp">/subnetworks/</span>dev-<span class="hljs-number">6</span>de02e1<span class="hljs-string">"</span></pre></div><div id="5ba2"><pre><span class="hljs-symbol">subnetworkProject:</span> “myproject”</pre></div><div id="ebc0"><pre>}</pre></div><div id="8a97"><pre>]</pre></div><div id="54eb"><pre><span class="hljs-symbol">Resources:</span></pre></div><div id="ffc3"><pre><span class="hljs-addition">+-1 replaced</span></pre></div><div id="986c"><pre><span class="hljs-symbol">4 </span>unchanged</pre></div><div id="9fbf"><pre><span class="hljs-attribute">Duration</span>: <span class="hljs-number">2</span>m12s</pre></div><p id="4246">My final example will be to show how to customize the environment on the remote instance by means of values passed to the instance via metadata. This can be useful for example when you want to create files containing certain custom values during the bootstrapping process on the remote instance.</p><p id="d4f2">Here for example is how to customize the “message of the day” (aka MOTD) greeting on the remote instance based on a custom value passed as metadata.</p><p id="6fe1">I first set a configuration variable called <code>motdgreeting</code>in pulumi:</p><div id="5d99"><pre> pulumi<span class="hljs-built_in"> config </span><span class="hljs-built_in">set</span> motdgreeting ‘Hello <span class="hljs-keyword">from</span> dev instance’</pre></div><p id="fcdb">I read the value of this configuration variable in __main__py and pass it to the <code>compute.Instance</code> constructor as the value of the <code>motdgreeting</code> key in the <code>metadata</code> dictionary:</p><div id="8499"><pre><span class="hljs-meta"># example of metadata variable</span></pre></div><div id="e586"><pre><span class="hljs-attr">motd_greeting</span> = config.require(‘motdgreeting’)</pre></div><div id="8185"><pre><span class="hljs-keyword">instance </span>= compute.<span class="hljs-keyword">Instance( </span> <span class="hljs-keyword">instance_name, </span> name=<span class="hljs-keyword">instance_name, </span> machine_type=<span class="hljs-keyword">instance_type, </span> <span class="hljs-keyword">boot_disk={ </span> “initializeParams”: { “image”: <span class="hljs-keyword">instance_image, </span> “size”: <span class="hljs-keyword">instance_disk_size </span> } }, network_interfaces=[ { “network”: network.id, “accessConfigs”: [{“nat_ip”: <span class="hljs-keyword">addr.address}] </span> } ], metadata_startup_script=<span class="hljs-keyword">script, </span> metadata={ “motdgreeting”: motd_greeting, }, )</pre></div><p id="09c7">Finally, I retrieve the value of the <code>motdgreeting</code> metadata variable in the init script and use it to set the MOTD message:</p><div id="441d"><pre>MOTD_GREETING=(curl http://metadata.google.internal/computeMetadata/v1/instance/attributes/motdgreeting -H “Metadata-Flavor: Google”) <span class="hljs-built_in">echo</span> “<span class="hljs-variable">MOTD_GREETING</span>” &gt; /etc/motd</pre></div><p id="8417">If you’ve been following along, you now have an instance running in GCP that you can ssh into, which is already running docker, and which you know how to customize it to your needs. If you don’t plan on using this instance, don’t forget to delete it so you don’t get billed for it. To do that, you can run:</p><div id="71a4"><pre><span class="hljs-variable"> </span>pulumi destroy</pre></div></article></body>

Infrastructure as code: using Pulumi to provision and bootstrap a GCP instance

This post will show you how to use Pulumi as the infrastructure-as-code tooling for provisioning an instance in the Google Cloud Platform. I will use Python for the Pulumi code examples. I ran the examples below on MacOS.

I used brew to install the pulumi CLI:

$ brew install pulumi

If you already had pulumi installed, you can upgrade it to the latest release:

$ brew upgrade pulumi

The next step was to download the gcloud SDK and CLI and to set it up following the setup instructions here:

https://www.pulumi.com/docs/intro/cloud-providers/gcp/setup/

$ gcloud auth login
You are now logged in as [myuser@example.com].
Your current project is [myproject]. You can change this setting by running:
$ gcloud config set project PROJECT_ID
$ gcloud auth application-default login
Credentials saved to file: [/Users/ggheo/.config/gcloud/application_default_credentials.json]
These credentials will be used by any library that requests
Application Default Credentials.
To generate an access token for other uses, run:
gcloud auth application-default print-access-token

I then created a new Pulumi project for Python in GCP. I named the project gcpinfra and I kept the default stack name which is dev.

$ pulumi new gcp-python
This command will walk you through creating a new Pulumi project.
Enter a value or leave blank to accept the (default), and press <ENTER>.
Press ^C at any time to quit.
project name: (gcpinfra)
project description: (A minimal Google Cloud Python Pulumi program) Provision GCP infrastructure resources
Created project ‘gcpinfra’
Please enter your desired stack name.
To create a stack in an organization, use the format <org-name>/<stack-name> (e.g. `acmecorp/dev`).
stack name: (dev) dev
Created stack ‘dev’
project: The Google Cloud project to deploy into: myproject
Saved config
Your new project is ready to go! ✨
To perform an initial deployment, run the following commands:
1. virtualenv -p python3 venv
2. source venv/bin/activate
3. pip3 install -r requirements.txt
Then, run ‘pulumi up’

I created and activated a virtual environment based on Python 3, then I installed the Pulumi requirements via pip3 install -r requirements.txt:

$ virtualenv -p /usr/local/bin/python3 venv
Running virtualenv with interpreter /usr/local/bin/python3
Using base prefix ‘/usr/local/Cellar/python/3.7.4/Frameworks/Python.framework/Versions/3.7’
New python executable in /Users/ggheo/code/mycode/codepraxis/mymooc/pulumi/gcpinfra/venv/bin/python3.7
Also creating executable in /Users/ggheo/code/pulumi/gcpinfra/venv/bin/python
Installing setuptools, pip, wheel…
done.
$ source venv/bin/activate
$ pip3 install -r requirements.txt

At this point, I was able to see the following files created:

$ ls -latotal 40
drwxr-xr-x 8 ggheo staff 256 Nov 16 15:53 .
drwxr-xr-x 4 ggheo staff 128 Nov 16 15:51 ..
-rw — — — — 1 ggheo staff 12 Nov 16 15:51 .gitignore
-rw-r — r — 1 ggheo staff 35 Nov 16 15:52 Pulumi.dev.yaml
-rw — — — — 1 ggheo staff 78 Nov 16 15:51 Pulumi.yaml
-rw — — — — 1 ggheo staff 203 Nov 16 15:51 __main__.py
-rw — — — — 1 ggheo staff 32 Nov 16 15:51 requirements.txt
drwxr-xr-x 6 ggheo staff 192 Nov 16 15:53 venv

The sample gcp-python project from Pulumi creates the following Pulumi code:

$ cat __main__.py
import pulumi
from pulumi_gcp import storage
# Create a GCP resource (Storage Bucket)
bucket = storage.Bucket(‘my-bucket’)
# Export the DNS name of the bucket
pulumi.export(‘bucket_name’, bucket.url)

My next step was to set a series of Pulumi configuration variables that I am going to use in my code:

GCP specific:

$ pulumi config set gcp:project myproject
$ pulumi config set gcp:region us-central1
$ pulumi config set gcp:zone us-central1-a

Application-specific:

$ pulumi config set instance_name dev
$ pulumi config set instance_type n1-highmem-2
$ pulumi config set instance_image ubuntu-1604-xenial-v20191010
$ pulumi config set instance_disk_size 50

I then modified the sample __main__.py code so that it creates a GCP instance and also bootstraps it by means of an init/startup script. I based my code on on another Pulumi example.

$ cat __main__.py
import pulumi
from pulumi_gcp import compute
with open(‘init_script.txt’, ‘r’) as init_script:
    data = init_script.read()
script = data
config = pulumi.Config()
instance_name = config.require(‘instance_name’)
instance_type = config.require(‘instance_type’)
instance_image = config.require(‘instance_image’)
instance_disk_size = config.require(‘instance_disk_size’)
addr = compute.address.Address(instance_name)
network = compute.Network(instance_name)
firewall = compute.Firewall(
  instance_name,
  network=network.self_link,
  allows=[
    {
      “protocol”: “tcp”,
      “ports”: [“22”]
    }
  ]
)
instance = compute.Instance(
  instance_name,
  name=instance_name,
  machine_type=instance_type,
  boot_disk={
    “initializeParams”: {
       “image”: instance_image,
       “size”: instance_disk_size
     }
  },
  network_interfaces=[
    {
      “network”: network.id,
      “accessConfigs”: [{“nat_ip”: addr.address}]
    }
  ],
  metadata_startup_script=script,
)
# Export the DNS name of the bucket
pulumi.export(“instance_name”, instance.name)
pulumi.export(“instance_network”, instance.network_interfaces)
pulumi.export(“external_ip”, addr.address)

Note that I pass the data from a file called init_script.txt to the compute.Instance constructor, assigning it to the variable metadata_startup_script.

Here is the init script, where I add my personal ssh public key on the remote instance, then I install docker and docker-compose:

$ cat init_script.txt
# add ssh pubkey
mkdir -p /home/ggheo/.ssh
chmod 700 /home/ggheo/.ssh
echo “ssh-rsa contents of my ssh public key” >> /home/ggheo/.ssh/authorized_keys
chown -R ggheo:ggheo /home/ggheo/.ssh
# install docker
sudo apt update
sudo apt install -y make apt-transport-https ca-certificates curl software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository “deb [arch=amd64] https://download.docker.com/linux/ubuntu bionic stable”
sudo apt update
sudo apt-cache policy docker-ce
sudo apt install -y docker-ce
# download docker-compose
sudo curl -L https://github.com/docker/compose/releases/download/1.21.2/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose

To provision the devstack, I ran :

$ pulumi up
Previewing update (dev):
Type Name Plan
+ pulumi:pulumi:Stack gcpinfra-dev create
+ ├─ gcp:compute:Address dev create
+ ├─ gcp:compute:Network dev create
+ ├─ gcp:compute:Firewall dev create
+ └─ gcp:compute:Instance dev create
Resources:
+ 5 to create
Do you want to perform this update? yes
Updating (dev):
Type Name Status
+ pulumi:pulumi:Stack gcpinfra-dev created
+ ├─ gcp:compute:Network dev created
+ ├─ gcp:compute:Address dev created
+ ├─ gcp:compute:Instance dev created
+ └─ gcp:compute:Firewall dev created
Outputs:
external_ip : “35.223.75.5”
instance_name : “dev”
instance_network: [
[0]: {
accessConfigs : [
[0]: {
natIp : “35.223.75.5”
network_tier : “PREMIUM”
}
]
name : “nic0”
network : “https://www.googleapis.com/compute/v1/projects/myproject/global/networks/dev-6de02e1"
networkIp : “10.128.0.2”
subnetwork : “https://www.googleapis.com/compute/v1/projects/myproject/regions/us-central1/subnetworks/dev-6de02e1"
subnetworkProject: “myproject”
}
]
Resources:
+ 5 created
Duration: 58s
Permalink: https://app.pulumi.com/griggheo/gcpinfra/dev/updates/6

I was now able to list the new GCP instance and ssh into it with the gcloud CLI:

$ gcloud compute instances list
NAME ZONE MACHINE_TYPE PREEMPTIBLE INTERNAL_IP EXTERNAL_IP STATUS
dev us-central1-a n1-highmem-2 10.128.0.2 35.223.75.5 RUNNING
$ gcloud compute ssh dev
Warning: Permanently added ‘compute.1672722981827980594’ (ECDSA) to the list of known hosts.
Welcome to Ubuntu 16.04.6 LTS (GNU/Linux 4.15.0–1046-gcp x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
40 packages can be updated.
18 updates are security updates.
The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.
ggheo@dev:~$

At this point, I verified that docker, docker-compose and tutor were installed correctly. I was able to see the init script installation commands in /var/log/auth.log.

I was also able to ssh directly into the remote GCP instance, because my ssh key was added to authorized_files.

As an example of updating an existing Pulumi stack, let’s change the init script to add our user to the docker group. I added this line to the end of init_script.txt:

usermod -a -G docker ggheo

When running pulumi upit will replace the instance because it detects that the startup script has changed:

$ pulumi up
Previewing update (dev):
Type Name Plan Info
pulumi:pulumi:Stack gcpinfra-dev
+- └─ gcp:compute:Instance dev replace [diff: ~metadataStartupScript]
Outputs:
~ instance_name : “dev” => output<string>
- instance_network: [
- [0]: {
- accessConfigs : [
- [0]: {
- natIp : “35.223.75.5”
- network_tier : “PREMIUM”
}
]
- name : “nic0”
- network : “https://www.googleapis.com/compute/v1/projects/myproject/global/networks/dev-6de02e1"
- networkIp : “10.128.0.2”
- subnetwork : “https://www.googleapis.com/compute/v1/projects/myproject/regions/us-central1/subnetworks/dev-6de02e1"
- subnetworkProject: “myproject”
}
]
+ instance_network: output<string>
Resources:
+-1 to replace
4 unchanged
Do you want to perform this update? yes
Updating (dev):
Type Name Status Info
pulumi:pulumi:Stack gcpinfra-dev
+- └─ gcp:compute:Instance dev replaced [diff: ~metadataStartupScript]
Outputs:
external_ip : “35.223.75.5”
instance_name : “dev”
~ instance_network: [
~ [0]: {
accessConfigs : [
[0]: {
natIp : “35.223.75.5”
network_tier : “PREMIUM”
}
]
name : “nic0”
network : “https://www.googleapis.com/compute/v1/projects/myproject/global/networks/dev-6de02e1"
~ networkIp : “10.128.0.2” => “10.128.0.3”
subnetwork : “https://www.googleapis.com/compute/v1/projects/myproject/regions/us-central1/subnetworks/dev-6de02e1"
subnetworkProject: “myproject”
}
]
Resources:
+-1 replaced
4 unchanged
Duration: 2m12s

My final example will be to show how to customize the environment on the remote instance by means of values passed to the instance via metadata. This can be useful for example when you want to create files containing certain custom values during the bootstrapping process on the remote instance.

Here for example is how to customize the “message of the day” (aka MOTD) greeting on the remote instance based on a custom value passed as metadata.

I first set a configuration variable called motdgreetingin pulumi:

$ pulumi config set motdgreeting ‘Hello from dev instance’

I read the value of this configuration variable in __main__py and pass it to the compute.Instance constructor as the value of the motdgreeting key in the metadata dictionary:

# example of metadata variable
motd_greeting = config.require(‘motdgreeting’)
instance = compute.Instance(
  instance_name,
  name=instance_name,
  machine_type=instance_type,
  boot_disk={
    “initializeParams”: {
       “image”: instance_image,
       “size”: instance_disk_size
     }
  },
  network_interfaces=[
    {
      “network”: network.id,
      “accessConfigs”: [{“nat_ip”: addr.address}]
    }
  ],
  metadata_startup_script=script,
  metadata={
    “motdgreeting”: motd_greeting,
  },
)

Finally, I retrieve the value of the motdgreeting metadata variable in the init script and use it to set the MOTD message:

MOTD_GREETING=$(curl http://metadata.google.internal/computeMetadata/v1/instance/attributes/motdgreeting -H “Metadata-Flavor: Google”)
echo “$MOTD_GREETING” > /etc/motd

If you’ve been following along, you now have an instance running in GCP that you can ssh into, which is already running docker, and which you know how to customize it to your needs. If you don’t plan on using this instance, don’t forget to delete it so you don’t get billed for it. To do that, you can run:

$ pulumi destroy
Docker
Pulumi
Infrastructure As Code
Cloud Computing
Google Cloud Platform
Recommended from ReadMedium
avatarAllan Alfonso
How to reduce your Google Cloud Compute Engine Bill by 50% with Committed Use Discounts — Part 2

In Part 1, I discussed how Committed Use Discounts (CUDs) can generate savings.

8 min read
avatarNikhil YN
Automating Instance Group Startup Scripts in Google Cloud with Terraform

Introduction: In this guide, we’ll explore how to leverage Terraform to automate the deployment of instance groups in Google Cloud and…

3 min read
avatarMatt Perreault
Setting Up Google Cloud Composer in a Shared VPC Network

Plenty of resources are available for setting up a Cloud Composer environment in a single GCP project. However, there is a lack of unified…

9 min read
avatarSiddharth Satpute
Automating Infrastructure as Code with Terraform and GitHub Actions in AWS

Introduction

5 min read
avatarDavid Bradshaw
Deployment of Corporate Training Portal on Google Cloud Platform using Deployment Manager, Google…

Project description:

2 min read
avatarJoel Wembo
Technical Guide: End-to-End CI/CD DevOps with Jenkins, Docker, Kubernetes, ArgoCD, Github Actions …

Building an end-to-end CI/CD pipeline for Django applications using Jenkins, Docker, Kubernetes, ArgoCD, AWS EKS, AWS EC2

48 min read