DevOps 101

What is DevOps?

• The merger of <Some Action> and IT operations with the purpose of increasing value to stakeholders.

Development + Operations = DevOps

• The merger of development and IT operations with the purpose of increasing value to stakeholders.
• DevOps has paved the way with well defined models to achieve the 3-ways of Devops…

The 3 Ways of DevOps

Work flows one direction, downstream.

• Systems and holistic thinking
• Focus on value streams
• Don’t pass defects downstream

Create, shorten, and amplify feedback loops.

• Increase Velocity
• Incremental Progress
• Fail Fast, Fix Fast

Continued experimentation and learning

• Continually Improve
• Build Knowledge
• Increase Resilience

CI/CD

This model is deceptively simple. It’s component parts are:

• Continuous Integration
• Continuous Delivery /Continuous Deployment (Automatic deployment to an environment)

The hidden CI

Continuous Improvement

If this is not happening then you are on a different model.

CICD - Crawl

• Version Control - Maybe
• Build Artifact
• Deploy Artifact - One Environment

CICD - Walk

• Version Control - Absolutely
• Build Artifact
• Tag Artifact
• Push - Docker/Artifactory/Helm
• Deploy Artifact
  • Multiple Environments

CICD - Run

• Version Control - Git Branch Model Established
• Unit Tests
• Build Artifact
• Tag Artifact
• Push - Docker/Artifactory/Helm
• Deploy Infrastructure - Terraform/Ansible/Pulumi/Other
• Deploy Artifact - Multiple Environments

CICD - Sprint

• Version Control - Git Branch Model Established
• Code Coverage/Quality
• Unit Testing
• Build Artifact
• Tag Artifact
• Push - Docker/Artifactory/Helm
• Deploy Infrastructure - Terraform/Ansible/Pulumi/Other
• Deploy Artifact - Multiple Scalable Environments

A great conceptual model (Jenkins-x).

DevOps Components

DevOps takes many forms but ultimately boils down to a handful of components working together;

1. Version Control
2. Pipelines
3. Artifacts

The components above work for simple software releases.

But there is always ‘More’…

Deployment Environments

Anything more advanced usually include

1. Service Endpoints (Secrets)
2. Metadata Management (Parameters)
3. Workload Sizing (autoscaling)
4. Gates/Approvals/Processes
5. Governance/Security

The DevOps engineer will wrap these together with CI/CD into Pipelines

Pipeline Products

One can weave together pipelines with a number of tools;

1. Jenkins/Jenkins-x
2. CircleCI
3. Semaphore
4. Tekton
5. Azure DevOps

Azure DevOps Pipelines

The Azure PaaS solution for creating pipelines is Azure DevOps.

• AKA -> VSTS Online

ADO Classic Pipelines

There are 2 parts to classic pipelines;

1. Build Pipeline
2. Release Pipeline

ADO Classic Pipelines

The build pipeline can reside as YAML code in the source repository (azure-pipeline.yaml).

The release pipeline is pretty much a manual affair (unless you are clever with json and apis)

Classic Pipeline Notes

• Build pipelines can be declarative YAML in the source repo
• Release pipelines are entirely manual (ClickOps!)
• There is a ‘hidden’ pipeline artifact repository that bridge the two
• It is smart to use your own artifact repository instead (just sayin’)
• Just way too… manual.

The ‘Multi-Stage pipeline’ has been released into preview.

It overcomes some of the deficiences of the classic pipeline.

I deploy these exclusively.

Multi-Stage Pipelines

• Both build and release code can be expressed as YAML and shifted left
• The introduction of ‘environments’ for deployments
• Can create templatized code for pipelines in another git repo
• Can reference, trigger, and pull artifacts from other pipelines

Multi-Stage Limitations

• To use the deployment element a pipeline artifact MUST have been created
• Manual intervention of deployments is limited to ‘Environments’
• It is bleeding edge

CI (Building)

The build process can be substantially more difficult than the deploy but is generally broken into the following parts

• Build
• Tag
• Push

Git Branch Strategies

Build operations will be closely aligned with the chosen git strategy.

Common models include:

• Single Master
• Git Flow*
• GitHub Flow
• Forking

Git Notes

• Your dev team will drive the git branching strategy
• GitFlow does not account for multiple environments
• Pull Request actions and branch policies are not native to git
• Submodules obfuscate a repo’s depth
• Git hooks are super handy (but I don’t use them)

Learning Git

Just read the manpages, it is easy!

Git man page generator -> https://git-man-page-generator.lokaltog.net/

Strong Pipelines

The stronger your pipeline is, the less you have to support it.

What makes a strong pipeline?

• Few outside dependencies
• Simplicity
• Immutability

Outside Dependencies

What is an outside dependency? (see if you can find the pattern);

• Public Docker Images
• Public Upstream Libraries
• Public Pipeline Tasks
• Public Helm Charts

Simpler Pipelines

What makes simpler pipelines?

• Pipeline as code
• Less delivery code
• Fewer Moving Parts

Immutability

What is Immutability in a pipeline?

• Clear division between CI and CD
• Less delivery code
• Eliminate 'ClickOps'

Lessons Learned

There are many best practices for DevOps.

Here are a few from the field…

• Work towards a regular release cadence
• Establish a git branch strategy early
• Cement a divide between CI & CD
• Learn some Agile
• 'Latest' tag is the devil

And a few more…

• Model the 12-factor app approach
• Build only once
• All pipeline as code if possible
• Immutable artifacts
• Shift left all that you can

And yet more…

• Create an umbrella repo
• Everything is a container
• Reduce outside dependencies
• Versioning, don't leave it until last
• Be able to build/deploy locally

Finally,

• If you implement DevOps correctly you WILL lose velocity, at first.

• It gets better

DataOps

Data schema is just as important as the code itself.

• Include a baseline and strategy for persistent data schema deployment in your pipeline.
• A database change should be subject to the same build/release cadence as the rest of the project.

Database Schemas

• Persistent data always has a schema
• This schema is closely bound to the application
• The versioned schema should be in version control
• This schema should be released with the app
• For obvious reasons, this should be idempotent

DataOps Terms

A few quick terms to know.

• Migration - Schema update
• Baseline - The ‘first’ schema

DataOps Tools

Some tools exist to facilitate CICD in pipelines. A few are;

• Liquibase (liquibase.org)
• Flyway (flywaydb.org)
• Sqitch (sqitch.org)

Model Breakers

Some IT not easily fit into the traditional DevOps CICD model:

• Data Science
• Data Engineering

References

• Three ways of devops - https://itrevolution.com/the-three-ways-principles-underpinning-devops/
• 12-factor app - https://12factor.net/

This presentation made with:

• Github - https://github.com/zloeber/deck.loeber.live
• ADO Pipelines - https://github.com/zloeber/azure-pipeline-library
• hugo - https://gohugo.io/
• reveal.js - https://github.com/hakimel/reveal.js
• render.com - Hosts https://decks.loeber.live