Skip to main content
December 17, 2024
Streamlining AWS Automation - Cloud Platform Engineering at CREALOGIX
CREALOGIX AWS
Management & Governance
Financial Services
Picture of Tamer Hunsu
Tamer Husnu
Senior Cloud Architect

In this blog post, we will explore how Crealogix combined ServiceNow, Terraform, GitLab and Backstage into one solution that enables AWS self-service for engineering teams.

Building and maintaining workloads in the cloud is slowed down when central teams manually provision AWS accounts - hampering development and innovation.

The solution developed by Crealogix not only automates AWS account vending, but also provisions fully configured environments, including Kubernetes clusters, databases, and other essential infrastructure components.

This automation approach ensures security, compliance, and consistency while reducing operational overhead and minimizing the risk of human error.

 

Introduction

Crealogix is a Swiss-based software company specializing in digital banking and wealth management solutions. Many financial organizations elevate their customer experience through their innovative digital services.

Operating in the Financial Services sector, security and compliance are top priorities for Crealogix and its clients. As the use of AWS expanded, managing and provisioning accounts became increasingly complex.

The central infrastructure team, responsible for creating, configuring, and maintaining cloud

environments, became a bottleneck as the company’s demands grew. Without a centralized, automated system, AWS accounts were sometimes provisioned inconsistently, leading to non-compliance risks, configuration issues, and challenges in enforcing security policies organization wide. This added unnecessary overhead for correcting mistakes and maintaining governance across accounts.

To address this, Crealogix developed a solution to not only automate cloud environment provisioning, but also AWS organizational unit (OU) and account vending, enabling full self-service capabilities for our Business Units. This empowers each Business Unit to manage their AWS usage independently, reducing reliance on central infrastructure team availability.

Solution overview

The solution is architected using the following 3rd party software:

  • Terraform – chosen as the infrastructure as code (IaC) tool because Crealogix’s solution engineering team had experience with it, and there are plenty of resources available online.
  • GitLab - GitLab hosts the IaC repositories for the solution and is the automation enabler thanks to its pipelines feature.
  • BackstageSoftware Templates are used to configure the environments on AWS, and the Software Catalog to track AWS OUs, accounts and resources.
  • ServiceNow – facilitates request/approval of new AWS OUs and accounts and triggers the automation pipeline in GitLab via the REST API. It was important for Crealogix that a senior Business Unit member should approve each new AWS account, as it involves costs.

and the following AWS services:

Technical details

Business Unit setup in AWS Organizations

Crealogix is structured into individual Business Units, each focusing on its own market segment, portfolio, and product(s). Business Units manage their own budgets, development, and sales. Core company functions, such as HR, administration, and internal IT, are shared as this remains both practical and economical.

Crealogix uses AWS Organizations to segregate AWS resources at the Business Unit level. This allows tracking AWS usage and costs per Business Unit, and to configure access to AWS resources easily. This setup supports the compliance requirements and simplifies the breakdown of AWS costs across Crealogix.

The setup of the Business Unit OUs in AWS Control Tower was performed manually. We created an OU under the root for each Business Unit, followed by the tooling and deployment OUs/accounts.

For each Business Unit, the solution includes:

We use Terraform to provision the resources in the tooling and deployment accounts, ensuring consistency.

Networking account

Access to the corporate data center is managed via the Global Networking AWS account. AWS Transit Gateway and Route 53 resolver rules are shared with the whole Organization using AWS Resource Access Manager, providing Workload accounts access to the corporate network resources (GitLab and Artifactory). The Route 53 Zone apex is managed in the Networking account, allowing us to delegate the root subdomain from the central networking account to the workload accounts. This lets each account manage its own DNS records independently.

Backstage details

The Backstage System Model is the basis for the platform model. The relevant entities for this solution are Groups, Domains and Systems.

The Backstage System Model is mapped to AWS entities as follows:

Business Units can use AWS organizational units to group their accounts into what Crealogix calls “Solutions”. The account naming takes into consideration the Business Unit name and the Solution name.

Backstage Software Templates are used to guide users step-by-step in configuring the resources needed for their account.

In the following screenshot from our “AWS Environment Provision/Update” template developers are asked to provide their EKS requirements.

They can specify the Kubernetes version, the worker node types (from a preconfigured list), the maximum number of nodes, all in a developer friendly manner.

 

Only supported resources are available in the templates (e.g., only PostgreSQL and Oracle as Relational DB options), meaning accounts cannot drift into unsupported configurations.

Once a user approves their options, Backstage creates a merge request (MR) in GitLab with the templated environment variables.

The pipeline triggered for the MR performs a terraform plan operation to allow the Platform Engineer to see which changes would be applied to the relevant account

 

Once the MR is reviewed and merged into the main branch, the main GitLab Pipeline applies the changes to the environment.

Software Templates have the added benefit of keeping the environments repeatable, compliant, and secure.

Process flow and usage

OU Provisioning (CREALOGIX Business Unit solution)

The process of creating a new OU within a Business Unit (BU) starts in ServiceNow.

Users submit a “New OU” service request, providing the OU name and associated BU. The request then moves through an approval process requiring sign-off from a senior BU member. Once approved, the GitLab Pipeline API is triggered with the necessary details to create the OU (Step 1 in the diagram).

 

 

The pipeline uses Terraform and AWS CLI to create the OU and apply a Control Tower Baseline for governance (Step 2 & 3).

 

Terraform also sets up a CloudFormation StackSet in the management account, which injects IAM roles into future accounts within the OU. This “bootstraps” new accounts with roles necessary for further automation (Step 4).

 

Finally, the Backstage Catalog is updated with a new Domain entry containing the OU details. This is achieved by using Terraform to create a catalog-info.yaml file, which the pipeline commits to the repository at the end of a successful run (Step 5).

 

Account Provisioning (CREALOGIX Solution Enviroment)

 

Similar to the process for creating a new OU, creating a new account starts in ServiceNow.

Users submit a “New account” service request, providing the environment name, account owner, and parent OU. The request goes through an approval process requiring sign-off from a senior BU member. Once approved, the GitLab Pipeline API is triggered with the necessary details to create the account (Step 1 in the diagram).

 

 

The pipeline, using Terraform and AWS CLI, triggers the Service Catalog API to create a new AWS Control Tower-managed account in the specified parent OU. The StackSet from the OU provisioning process detects the new account and deploys the required IAM roles for automation (Step 2 and 3).

 

Terraform also sets up a public Hosted Zone in the workload account and updates the Route 53 Zone Apex to enable DNS delegation to the subdomain (Step 4).

 

The Backstage Catalog is updated with a new System entry containing the account details, and assigned as a child of the Domain (OU) it belongs to (Step 5).

 

Enviroment provisioning

Crealogix developers can configure the resources they would like to have provisioned on an AWS account in Backstage.

The Backstage template "Provision AWS Environment" gathers the necessary parameters for configuring the developer-specified environment through a guided input process. From these inputs it generates a Terraform variables file, which is committed to a new branch by Backstage actions. Following this, an MR is automatically created (as shown in Step 1 of the diagram).

Crealogix Platform engineers are notified that there is a new MR in the environments repo, and review, approve or merge the change (Step 2).

 

The pipeline extracts the CI/CD variables, specifying which BU the job is for.

This allows the pipeline to set AWS environment variables relevant for the BU Deployment user, so the subsequent steps can assume the needed IAM role in the workload account (Step 3).

 

The terraform variables file (which was generated by Backstage based on the end-user input) defines which Terraform modules are to be deployed into the account (Step 4).

The Pipeline runs with the Deployment user credentials configured in its environment, allowing it to assume the IAM Role in the workload account.

This is the StackSet managed role which was injected during account provisioning (Step 5).

 

Resources within the AWS account are provisioned, including Amazon VPC (with TGW Attachments), EKS, EKS add-ons (and other necessary Kubernetes tools like external-dns, AWS Load Balancer Controller, …), RDS etc.. (Step 6).

 

Once the infrastructure has been provisioned, the application can be deployed via ArgoCD, pulling manifests from GitLab and images from Artifactory (Step 7).

 

Conclusion

 

This blog post explained how Crealogix has significantly enhanced its AWS infrastructure management by integrating Terraform, GitLab, ServiceNow, and Backstage.

This automated approach not only empowers engineering teams with self-service capabilities but also ensures that security, compliance, and consistency are maintained across the organization.

We hope this post inspires your organization's Platform Engineering initiatives, empowering your development teams to innovate by equipping them with robust tools and streamlined processes, all while upholding governance, security, and operational excellence.

To help speed up your adoption of platform engineering, a similar implementation has been created using GitHub actions, Terraform and Backstage:  

https://github.com/crealogix/aws_management

https://github.com/crealogix/aws_environments.

 

Related posts

Contact us