Archive for August 2018

Infrastructure as Code can Help with Compliance

IaC cycle

This was originally published on Amalgam Insights.


Companies struggle with all types of compliance issues. Failure to comply with government regulations, such as Dodd-Frank, EPA or HIPPA, is a significant business risk for many companies. Internally mandated compliance also represents problems as well. Security and cost control policies are just as vital as other forms of regulation since they protect the company from reputational, financial, the operational risks.

IT helps to manage compliance risks in two ways. First, by deploying systems that detect and assist the company in complying with risk. For example, an analytics system designed to discover Dodd-Frank violations in a bank is a way for IT to help remove some regulatory risk. The second way is to design systems that are compliant with internal and external regulations. That may mean configuring networks such that they address common security holes or data storage so that privacy is maintained. Systems must be configured to meet the needs of regulations and policies.

One of the more important methods of ensuring that systems are compliant is through audits. Different audits are conducted for different purposes, but they operate similarly. The state of a system – business or computer – is evaluated against a series of policies. Policies are statement that describe a required state in a system that ensures compliance. For example, a data privacy policy may require that customer data be only housed on encrypted drives. A security policy may require two-factor authentication for all external logins to a system.

Systems in place can be audited using a variety of tools that match the system state to policies and detect areas out of compliance. The problem with this approach is that it is post-hoc. Finding compliances issues in production is good but finding them before they are in production is better. Pre-production audits often rely on design documentation. This is a limited method because last minute changes or mistakes with configuration can lead to the final state diverging from production.

This is where Infrastructure as Code(IaC) can be helpful. Central to IaC is the idea that a plaintext script – the code part – describes the desired state of the system. The automation server then configures the system to the state indicated in the script. IaC systems may provision and configure both hardware and software components.

IaC presents an advantage when auditing for compliance. The script is the documentation and what is in the script will become the final system state. This makes it easy for compliance and security professionals to understand the system state before it is created. They can then analyze these scripts for potential violations of policy and prevent them from becoming part of the production system. Using IaC allows companies to more easily move from reactive compliance to proactive compliance. Non-compliance can be detected early and corrected before they become the production state. In rapidly changing environments, this is a safer and quicker approach than detecting problems after the fact and correcting them while in production.

Dealing with regulatory, security, and operational compliance of at scale is much becoming more difficult as policies proliferate and systems become more complex. IaC is a tool to deal with systems and their compliance issue in these environments.

Cloud Vendors Release CI/CD Tools

Cloud CI/CD Tools

This was also released under a slightly different name on Amalgam Insights.

Development organization continue to feel increasing pressure to produce better code more quickly. To help accomplish that faster-better philosophy, a number of methodologies have emerged that that help organizations quickly merge individual code, test it, and deploy to production. While DevOps is actually a management methodology, it is predicated on an integrated pipeline that drives code from development to production deployment smoothly. In order to achieve these goals, companies have adopted continuous integration and continuous deployment (CI/CD) tool sets. These tools, from companies such as Atlassian and GitLab, help developers to merge individual code into the deployable code bases that make up an application and then push them out to test and production environments.

Cloud vendors have lately been releasing their own CI/CD tools to their customers. In some cases, these are extensions of existing tools, such as Microsoft Visual Team Studio on Azure. Google’s recently announced Cloud Build as well as AWS CodeDeploy and CodePipeline are CI/CD tools developed specifically for their cloud environments. Cloud CI/CD tools are rarely all-encompassing and often rely on other open source or commercial products, such as Jenkins or Git, to achieve a full CI/CD pipeline.

These products represent more than just new entries into an increasingly crowded CI/CD market. They are clearly part of a longer-term strategy by cloud service providers to become so integrated into the DevOps pipeline that moving to a new vendor or adopting a multi-cloud strategy would be much more difficult. Many developers start with a single cloud service provider in order to explore cloud computing and deploy their initial applications. Adopting the cloud vendor’s CI/CD tools embeds the cloud vendor deeply in the development process. The cloud service provider is no longer sitting at the end of the development pipeline; They are integrated and vital to the development process itself. Even in the case where the cloud service provider CI/CD tools support hybrid cloud deployments, they are always designed for the cloud vendors own offerings. Google Cloud Build and Microsoft Visual Studio certainly follow this model.

There is danger for commercial vendors of CI/CD products outside these cloud vendors. They are now competing with native products, integrated into the sales and technical environment of the cloud vendor. Purchasing products from a cloud vendor is as easy as buying anything else from the cloud portal and they are immediately aware of the services the cloud vendor offers.  No fuss, no muss.

This isn’t a problem for companies committed to a particular cloud service provider. Using native tools designed for the primary environment offers better integration, less work, and ease of use that is hard to achieve with external tools. The cost of these tools is often utility-based and, hence, elastic based on the amount of work product flowing through the pipeline. The trend toward native cloud CI/CD tools also helps explain Microsoft’s purchase of GitHub. GitHub, while cloud agnostic, will be much for powerful when completely integrated into Azure – for Microsoft customers anyway.

Building tools that strongly embed a particular cloud vendor into the DevOps pipeline is clearly strategic even if it promotes monoculture. There will be advantages for customers as well as cloud vendors. It remains to be seen if the advantages to customers overcome the inevitable vendor lock-in that the CI/CD tools are meant to create.