Automation Using Python on Google Cloud — IAM Service Accounts & Cloud Storage Integration
I am thrilled to have made it to the third cloud service provider as part of an intense multi-cloud boot camp, where I have been exposed to real-world enterprise use cases. This project, based on such scenarios, focused on using automation and Python to interact with Google Cloud Platform services. It highlighted how enterprises can leverage cloud automation to streamline their operations and manage cloud resources efficiently.
Project Description
This project underscores the vital role that automation plays in real enterprise environments, as companies increasingly rely on automated solutions to manage their cloud infrastructures at scale.
I acted as a Cloud Specialist, leveraging Google Cloud’s Identity and Access Management (IAM) services to create a Service Account. This Service Account enabled secure communication via scripts within the cloud architecture through the Google Cloud SDK and Python scripts. Through this project, I gained invaluable hands-on experience in configuring Service Accounts, managing IAM roles, and interacting with Google Cloud resources programmatically using Python.
Key Steps Performed:
Step 1: Installing Python
- Prerequisites: Installed Python 3.12.6 to support running automation scripts.
- Download Link: https://www.python.org/ftp/python/3.12.6/python-3.12.6-amd64.exe
For this use case, a Windows operating was used, as well as Git Bash and PowerShell terminal CLI. In these environments, I am able to execute Linux commands, which enhances my workflow by offering cross-platform compatibility. Additionally, I have downloaded and installed the Windows package for Python to ensure that I can run Python scripts seamlessly within this setup. If you running an Ubuntu and/or RHEL Linux, select the download option supported for Linux.
Step 2: Setting up Google Cloud SDK
To manage GCP resources programmatically, I installed the Google Cloud SDK.
Quickstart Guides:
- Google Cloud SDK on Windows: https://cloud.google.com/sdk/docs/quickstart#windows
- Google Cloud SDK on Linux: https://cloud.google.com/sdk/docs/quickstart#linux
Step 3: Creating a Service Account
- Path: IAM > Service Account
- Service Account Name: Automation
- Role Assigned: Storage Admin — to manage and interact with Google Cloud Storage.
This Service Account was used to execute scripts that interact with GCP services.
Step 4: Creating and Downloading the Service Account Key (JSON)
- Path: IAM > Service Account > automation > Added Key.
- Generated and downloaded the Service Account Key in JSON format to allow Python scripts to authenticate and interact with GCP resources.
Step 5: Downloading the Python Script
- Downloaded a pre-written Python script to list storage buckets within the GCP project and verified the lines using Visual Studio Code.
- The Python script is very basic and may be referenced below.
Pre-written Python script explained:
- Import the Google Cloud Storage library: The line item brings in the tools that are needed to interact with Google Cloud Storage.
- Import the sys module: Allows access to command-line arguments.
- Retrieve service account key: Gets the file path of the service account key used for authentication.
- Authenticate and create a client: Uses the key to set up a client for interacting with Google Cloud resources.
- List the buckets: Retrieves and stores a list of all Cloud Storage buckets associated with the account.
- Print the bucket list: This line outputs the list of buckets for validation or further actions.
Step 6: Installing Python Libraries for GCP Storage Services
To ensure that the Python script could interact with GCP storage services, I installed the necessary libraries.
Step 7: Creating a New Cloud Storage Bucket for Testing
Using the GCP console, I created a new Cloud Storage bucket for testing purposes, which would be listed by the Python script.
Step 8: Running the Python Script
The script returned a list of all the storage buckets in the current GCP project.
Conclusion
Having reached this stage of the multi-cloud boot camp, this project allowed me to strengthen my expertise in cloud automation with Google Cloud services. It has been a rewarding experience working on real-world enterprise use cases, which are critical as companies migrate to cloud environments. The hands-on experience I gained in automating tasks using Python, combined with the use of IAM services, adds great value for organizations looking to implement scalable cloud automation solutions.
