Automating change management

Change management automation is an important part of the DevSecOps pipeline reference implementation. Developers, approvers, and auditors can monitor the compliance aspects of deployments. Every deployment must follow an organization's change management policy.

Change management automation can be visualized via the following flowchart. The following flowchart illustrates a standard change management automation, emergency change management, a manual change request flow and the change management automation flow when inline rollback is involved.

Before you begin

Be sure to become familiar with the process and terminology. For more information, see Automated change management.

Creating change requests for deployments

Use the pull request template that is provided in the inventory for promotion pull requests to populate the change request fields. Because you cannot automatically populate these fields, you must manually populate them to promote changes. By manually populating these fields, you trigger the deployment and continue automatic data collection for the rest of the change request.

The promotion pull request template contains the following fields:

Priority Required. The priority of the change. Valid values are: critical, high, moderate, low, and planning.
Change Request assignee Required. The email address of the person that the change request is assigned to.
Additional Description Describes the change process. Extra content from the automation process is appended here.
Purpose/Goal Describes the purpose of the change.
Explanation of Impact Describes the possible impact of the change.
Customer Impact Required. Describes the impact of the change for the customer. Valid values are: critical, high, moderate, low, no_impact
Deployment Impact Required. Descrives the impact of the change upon deployment of the change. Valid values are: small, large
Backout Plan Describes the rollback or backout plan.

Along with these fields, you must set two additional fields from the environment properties:

target-environment-purpose (Required) Valid values are: production, pre_prod. Any non-production deployment would qualify as pre_prod.
target-environment-detail (Required) A string which describes the target-environment where the change is deployed.

For more information on change request data, see Data included in change requests.

Types of change

Change Request Management supports two types of change: emergency or regular.

If the current change is an emergency change, add the emergency label to the promotion pull request.

There is no emergency flow on the CI pipeline side. However, setting a CI pipeline/trigger property skip-inventory-update-on-failure to an empty value or 0 will allow the inventory repository to be updated, even if issues are detected in the CI pipeline run. With this updated inventory, an emergency change can be activated.

Creating change requests for changes that require planned downtime

The continuous deployment pipeline automatically approves change requests based on the following conditions:

Implementing the change does not cause any downtime (outage duration is zero).
Deployment Readiness is true.

The Change Management automation in the DevSecOps reference implementation assumes that all changes are implemented without planned downtime. If your changes require planned downtime, you must create the change request manually and send it for approval. After it is approved, you can start deployment by providing the change request ID. The pipeline moves ahead to check its approval, and then runs the deployment. For more information, see Manually approving change requests.

Deployment Readiness is calculated from the evidence that is collected in the continuous integration and continuous deployment stages. If any of the collected evidence suggests a deviation, or an unsuccessful check, scan, or test that is related to the deployed set of artifacts, the Deployment Readiness is set to false, and the Change Request is not auto-approved. The deployment stops, and prevents further steps until the change request is approved.

You can read the created change request ID from the pipeline logs, wait for approval, and then start the deployment again by using the same change request ID. The pipeline moves ahead to check its approval, and then continues the deployment.

If the change is a type of emergency, the change request must be retroactively reviewed and approved.

Running deployments by using an existing change request ID

Running pipeline with a pre-approved change-request

You can use a pre-approved change-request(CR) for deployment. Here, two scenarios are possible:

When CD Pipeline recognizes the CR as created by some earlier CD Pipeline run, it fast-paths the deployment by:

Reusing the precomputed delta and evidence summary from the CR evidence.
Skipping peer review and signature verification steps.
Deploying the pre-computed delta.

Conversely, when CD Pipeline is unable to determine if the CR provided by the user, was not created by an earlier CD Pipeline run, or the provided CR does not match the deployment target of the current run, in such cases:

It does not reuse any precomputed delta or evidence summaries.
It does not skip peer-review verification or artifact signature verification.
Recomputes delta and summary from scratch.
The only thing it does not do is mint a new CR, since a CR was already supplied.

Rerunning pipeline against failed deployment

If you do not want to use automated change management, you can provide a previously created and approved change request instead. Rerun failed deployments in the following scenarios:

The latest automatically created change request is not ready for deployment and the change request was not auto-approved. You got approval for the change request and must start deployment again by using the change request again.
The deployment requires downtime. You created the change request, it was approved, and you followed the required steps in your organization's Change Management Policy.
No code or configuration changed. You created the change request, explained what changed (if the change was not in your app code), received approval, and started a deployment by using the approved change request.

The change request(CR) remains open after the end of CD Pipeline.

You can start the DevSecOps reference continuous deployment pipeline by using a pre-approved change request and entering the change request ID for the change-request-id property.

If the change-request-id property is set, the pipeline skips data collection for the change request and moves ahead to check the approval state of the change request. If the change-request-id is set to notAvailable by default, a change request is automatically created by the continuous deployment pipeline.