Devops With AWS
About This Course
Getting Started – DevOpsConcepts, Tools, and Technologies:
Understanding the DevOps movement
The waterfall model
The agile model
Collaboration
Introduction to Cloud computing
Why DevOps?
The benefits of DevOps
The DevOps lifecycle
Build automation
Continuous integration
Best Practices for Continuous integration
Cloud computing
Configuration management
Continuous delivery/continuous deployment
Best practices for continuous delivery
Continuous monitoring
Continuous feedback
Tools and technologies
source Code repositories – Git,GitHub
Build tools – Maven ,Gradle,Ant,MS-Build
Continuous integration tools – Jenkins,TFS,GitLab,GitHubActions
Configuration management tools – Ansible,Chef
Cloud service providers :Aws,Azure
Container technology:Docker ,Kubernetes
Monitoring tools:Zenoss Nagios Grafana,DataDog
Deployment orchestration/continuous delivery – Jenkins
End-to-end orchestration: Jenkins plugins
The DevOps dashboard
Linux-commands/ShellScripting/Admin:
____________________________________
*Linux-Basics: architecture,shells,
*commands:simple,compound and complex commands
*Administration:useradmin,service management,package management,volume managment,ssh configuration,process management
Automation tasks with cron,Apache Configuration ,FTP Configuration,DHCP Configuration,DNS
* Bash Scripting: variables,keywords,operators, conditional,looping ,jumping statements,arrays,file operators, functions
* focus on protocals: ftp,ssh,scp,
*connecting to remote using : putty,puttygen,winscp
Source CodeManagement, Repositories, Administration:Git, Github
_______________________________________________________
Introduction to Git:
Local Git
Remote Git
GitEssentials:
Filesystem interactions
Hisotry visualization
Advanced branching
rewriting history and disater recovery
GitHub pull requests
Hosting repostiory
GitHub:
account creation
usercreation
repositories creation
introduction to webhooks
Repository Migration
lock and unlock the repositories
Git Best Practises:
Creating clean History
Merging vs rebasing
recommended team workflows
Note: will focus on Git commands as well as Github Administration
: special focus on GitHub Actions for CI/CD
Introduction to Build Automation:
what is build
build lifecycle
build environment
build tools
introduction and diff between Ant,Maven and Gradle
Gradle
Introduction to Apache MAVEN:
introcution to java architecture
what is maven
maven architecture
maven installation
configuring the heap size
monitoring the build
remote debugging
convetions over configuration
trouble shooting
Introduction to Project Object Model:
pom hierarchy
super pom
maven coordinates
manging pom dependencies
transitive dependencies,scopes,elusion
Maven Archetypes:
Archetype quickstart
the batch mode
archetype catalogues
Introduction to Maven Life cycle:
lifeycles,phases,plug-ins ,goals
understanding the pom file,maven settings and command line options in maven.
understanding maven profiles,adding,activationg and deactivating a maven profiles
properties in maven
Introduction to Maven Plug-ins:
introduction to plug-ins
how to configure plug-ins
introduction to clean,compiler,surefire,fallssafe,resources,deploy,site,jar,release
MavenConfiguration:
proxy authenitications
secured repositories
encrypting credentials in settings.xml
source code management system
mirror repositories
Deploying artifacts
Deployment part in maven:
deploying web application to apache tomcat server
*Dependency Managment
code quality plug-ins:
introduction.
analyzing code coverage with: Jacoco, Cobertura, Sonarqube, PMD, checkstyle, Findbugs, jxr
*Multi-module projects in maven
Maven Repository Management:
maven repostiories
the update policy
multiple repositories
what is maven repostiory manager
introduction to nexus
configuration of nexus
deploying artifacts in nexus using maven
difference between nexus and artifactory-jfrog
Introduction tomcat server:
architecture
configuration
common problems and troubleshooting in installation
configuration and deployment
integration of tomcat with the apache webserver, loadbalancers, security, and proxy configuration.
logging in tomcat
introduction to clustering in tomcat
Note: complete focus on CI part with shell scripting, tomcat, maven, Github, nexus, sonarqube
Continuous-Integration-Jenkins
_______________________________________
What is Jenkins Best Practices
Installation and configuration
Prerequisites
Download & installation
Configuration tour
Managing Jenkins
Securing Jenkins
Managing Credentials
Plugin Management
Jenkins Backup
Create a Build Slave
Creating Application Builds
Anatomy of the build
Cloning sample project
Manual compilation with Maven
Manually Testing, Packaging and Running the App
Creating a Jenkins Job and configuring a Git Repo
Compiling in Jenkins
Browsing the workspace in Jenkins
App Packaging in Jenkins
Archiving artifacts
Cleaning up Past Builds
Build time trend
The jenkins Dashboard
Troubleshooting build failures
Importing Job config.xml
Anatomy of the job
Build linking upstream and downstream
Plugin
Plugin Architecture
Extension Points
Getting Plugins
Plugin Wiki
Useful Plugins Overview
Source Code Plugins,Trigger Plugins,Build Tool Plugins,Wrapper Plugins,Notifier Plugins
Reporting Plugins
Artifact & UI Plugins
Installing a plugin
Plugin configuration
Security Overview
Continuous Testing and Continuous Integration and Testing
Adding steps to Freestyle Project
Creating a Pipeline job to execute Maven
Archiving in a Pipeline
Checking out git repository in pipeline
The Master Agent Model
Allocating a node and workspace in Pipeline
High level progress with Pipeline stages
Triggering Automated Builds
Configuring an Email Server,Slack Channels and Integrating SNS
Notifications when a build fails
Duplicating a job
Executing unit tests
Visualizing Test Results
Finding and Managing Plugins
The need for plugins
Integrated Code Coverage
Assessing a plugin
Installing the HTML Publisher plugin
Publishing HTML Reports
Testing Plugins and Plugin Types
BlueOcean UI Plugin
Building Continuous Delivery Pipeline
Continuous Delivery
Backup and Restore
Starting point and Pipeline stashing
Browsing Workspaces in Pipeline Jobs
A Second Node Allocation
Adding an Agent Node
Setup parallel integration testing in a pipeline
Executing and Monitoring Parallel pipelines
Manual Approval for Deployments
Setup Deployment to staging
Executing a Deployment pipeline
Checkin pipeline script to Git
Note: Completely focus on DSL-Groovy
Introduction to container technology-docker:
_________________________________________________
Introduction to docker
Understanding Docker
Understanding Docker Overview
Docker Architecture Engine And Objects
docker client
docker server
docker daemon
dockerhub
docker registry
dockerlayer
docker repostiory
images and containers
Diff between Containerization And Virtualization
Handling docker containers and images
Docker Images
Docker Images and container CLI Commands
Introducing DockerHub.
Search and Pull Images from Docker Hub
Build Image using Dockerfile
Build Image using Commit
Push Images to Docker Hub
Docker file Instructions
working with GitHub and docker hub
building images using-docker file
publishing images
generating images and deploying images into docker hub with CI Practise-Jenkins
Docker Trusted Registry
Docker Trusted Registry
Introduction Deploy DTR
Backup Swarm UCP DTR
Miscellaneous Topic with DTR
Docker Storage and Volumes
Docker Storage and Volumes
Docker Storage Drivers
Selecting Storage Driver
Persistent Storage
Manage Application Data
Docker Volumes
Docker Bind Mounts
Docker tmpfs Mounts
External Storage
docker -cloud
docker workflows
Docker Networking
Docker Networking
None Network
Container Networking
Bridge Network
Host Network
Disconnect And Add Network
Introduction to Overlay Network
Create New Networks
Exercise: Use Custom Networks
Remove Networks
Multiple Networks In Container
Security
Docker Security Logs
Security Scanning
Docker Content Trust
Security With MTL
Logs and Logging Drivers
Docker Compose:
_________________
Orchestration Docker Swarm
Docker Swarm
Concepts of Swarm
Create Swarm
Maintain Swarm
Deploy Services to Swarm
Updates to Services
Managing Swarm Services
Secrets and Overlay Network
Docker Stack
Introduction to Kubernetes:
_______________________________
>understanding container orchestration
>Trend of MicroServices
>kubernetes Architecture
Kubernetes Componenents
Master Components
Api Server,Controller Manager ,etcd,Scheduer
Node Components
Kubelet,Kube-Proxy,Runtime-environment[containers]
>Configuration ,setups K8s on Target Servers
>Working with K8s Objects or resources
namespace
pods
label,annotation,labelselector
ReplicationController(RC)
ReplicaSet(RS)
Deployments
Sevices,
volumes
jobs
Secrets
ConfigMap
>Network and Security
>Monitoring and Logging
>Continuous Delivery
>Handling the K8s Package Manager-Helm
>Cluster Administration
>K8s on AWS
ECR,ECS,EKS
Note: we are separately focusing on CKA for offline-batch.
Introduction to Ansible:
____________________________
what is ansible
ansible architecture
configuring ansible
introduction to yaml file
Setup and Configuration:
_____________________________
Test Environment Setup
Download and Installation
Ansible Configuration File
Ansible Python Dependencies
The HOSTS File
Overriding the Default HOSTS File
Overriding the Default System Ansible.Cfg File
Overriding the Default Roles Path
Understanding the core components of Ansible
Ad-hoc commands in Ansible
Use both static and dynamic inventories to define groups of hosts
___________________________________________________________________
Overview of static and dynamic inventories in Ansible
Static Inventories
Dynamic Inventories
Ansible Playbooks
________________________
Configuring Your ‘Ansible’ Account
Ansible Command Line
System Facts
System Facts: Common Values for Playbooks
Our First Playbook
Variables: Inclusion Types
Target Section
Variable Section
Task Section
Handler Section
Outlining Playbook
Create a Playbook from Outline
Optimizing Playbook
Taking Playbook for a Dry Run
Asychronous Polling
Simple Variable Substitution
Lookups
RunOnce
Local Actions
Loops
Conditionals
Until
Notify
Vault
Prompt – Interactive Playbook
Basic Include Statements
Tags
Basic Error Handling
Includes – Breaking Your Playbook Into Discrete Plays
Starting At Task or Stepping Through All Tasks
Passing Variables Into Playbooks at the Command Line
Using Jinja2 Templates
LocalAction
DelegateTo
Use a playbook to copy a program and customize it for the target host
Ansible-modules:
_____________________
Commonly used Modules
Using modules in playbooks
‘Setup’ Module
‘File’ Module
‘Pause’ Module
‘WaitFor’ Module
‘Yum’ Module
‘Apt’ Module
‘Service’ Module
‘Copy’ Module
‘Command’ Module
‘Cron’ Module
‘Debug’ Module
‘Fetch’ Module
‘User’ Module and more modules will discuss
Create and use templates to create customized configuration files
______________________________________________________________
Introduction
Templates
Working with Ansible facts and variables.
_______________________________________________
Let see how we get ansible facts and how we use facts
Using Ansible facts
Using variables to gather server info
working with ansible playbooks
Roles
_______________
Introduction to Roles
Roles – The Directory Structure
Role Based Tasks
Task Order – Pre and Post Tasks
Roles – Conditional Execution
Roles – Variable Substitution
Roles – Handlers
Roles – Using Notification
error handling ,rollback and reporing
introduction to inventories
introcution to ansible galaxy,tower
deployment with ansible
ansible tower and ci/cd
ansible with docker
ansible with aws
* will focus on nagios,elk, splunk,cloud logs with demonstration on cloud aws.
note: it always advisable to practise on cloud environment
* will demonstrate orchestarting application deployment.
plan:
Real-Time Usecases and pipelines:
_________________________________
1)TroubleShooting examples
2)deployment using shell scripting
3)Manual deployments
4)Deployment Pipelines using GitLab
5)CI/CD-pipelines using tools with groovy
jira+github+maven+jfrog+sonarqube+tomcat+docker+k8s+ansible+jenkins with groovy pipeline
6) focus on docker and k8s deployment pipelines
7)aws Developertools-CI/CD
>code commit
>code artifacts
>code build
>code deploy
>code pipeline
8) ecr,ecs and eks deployments
9) Deployments using Terraform
Introduction to AWS-Services:
____________________________
EC2,EBS,EFS,S3,AutoScaling,ELB,CloudWatch,vpc,Route53,IAM,CFT,ECR,ECS,EKS,RDS
Introduction Terraform:
_________________________
what is IAC-Infrastructure as code
AdHoc Scripts
configuration Management Tools
server Templating Tools
server provisioning Tools
Benefits of IAC
How Terraform Works
How Terraform Compares to other IAC Tools
configuration Management Versus Provisioning
Mutable Infrastructure Versus Immutable Infrastructure
Procedural Language Versus Declarative Language
Master Versus Masterless
Agent Versus Agentless
Getting Strated with Terraform:
________________________________
Setup Aws Account
Install Terraform
Deploy a Single Server
Deploy a Single Web Server
Deploy a Configurable Web Server
Deploy a Cluster of Web Servers
Deploy a Load Balancer
cleanup
How to Manage Terraform State:
_______________________________
what is Terraform State
shared stoage for state Files
Locking State Files
Isolating State Files
File Layout
Read-Only State
How to Create Reusable Infrastructure with Terraform Modules:
_______________________________________________________________
Module Basics
Module Inputs
Module Outputs
Module File paths
Inline Blocks
Module Versioning
Terraform Tips and Trciks: loops,if-statements,Delployments
_____________________________________________________________
loops
if-statements
simple if-statements
complicated if-statements
if-else statements
simple,complicated if-else statements
zero downtime Deployments
How to use Terraform as a Team:
_________________________________
Version Control
A Repository for Modules
A Repository for live Infrastructure
golden rule of Terraform
Automated Tests
prepare your code
write the Test code
use Multiple Types of Automated Tests
coding guidelines
Workflow
Plan
statging
code review
production
some Types of Terraform Changes Can be Automated
some Types of Terraform Changes Lead to conflicts
Larger Teams May Need to use a Development Pipeline